Prophecy Fabrics provide a structured way to govern data access by inheriting permissions from external platforms such as Databricks, while offering flexibility to design teams with varying levels of access to develop, test, and deploy workflows.
This article covers the core concepts of Fabrics and Teams, and walks through three common scenarios: interactive development, pipeline deployment with service principals, and deployment with service accounts.
Note: Prophecy provides you with compute and data storage for Free and Professional Editions (with Prophecy Managed Fabrics). Users on Express and Enterprise Editions need to connect to their own compute and data storage.
Core Concepts
What is a Fabric?
A fabric is a Prophecy entity that contains all of the connection information you need to connect to external compute and data storage. It provides a structured way to separate working environments (such as Production and Development) and stores a collection of data sources. Fabrics are the foundation for controlling how users access different compute environments.
What are Teams?
Teams organize user groups and projects within Prophecy. Each Team has designated Team Admins who manage the team, its Fabrics, and associated resources. Teams determine which Fabrics and projects are accessible by their members, and users can belong to multiple teams.
Best Practice: Separate your Teams by privilege level. Use a lower-privilege team for interactive development and a higher-privilege team for access to a deployment environment, using service principal credentials with broader data access.
Scenario 1: Interactive Development
For day-to-day development work with multiple developers, configure your Fabric to use OAuth authentication at the individual user level. This lets Prophecy identify exactly who is accessing which compute resources.
How It Works
With Databricks OAuth configured, a short-lived personal access token is generated each time a user attaches to a cluster. This token communicates the individual’s role-based access control (RBAC) privileges assigned in Databricks — ensuring users can only access the compute resources assigned to them. OAuth also simplifies credential management: instead of managing separate tokens per user, each developer authenticates once and Prophecy handles token generation automatically.
Best For
-
Teams with multiple active developers working in parallel.
-
Environments where individual compute utilization tracking is required.
-
Interactive cluster usage on Databricks.
Scenario 2: Pipeline Deployment with Service Principals
When scheduling and deploying pipelines, short-lived OAuth tokens are insufficient — you need long-lasting credentials. A dedicated deployment Fabric using a Databricks service principal is recommended for this purpose.
How It Works
A service principal is a non-human identity in Databricks with its own set of permissions. By configuring your deployment Fabric to authenticate as a service principal, pipelines run with consistent, durable credentials that are not tied to any individual user. Because service principal credentials typically carry broader data access permissions, the deployment Fabric should be assigned to a more privileged team to restrict who can trigger deployments.
Best For
-
Scheduled pipeline runs that require unattended execution.
-
Teams that centrally manage deployment credentials.
Scenario 3: Deployment with Service Accounts
An alternative deployment approach uses a service account — a dedicated user account created by the governance team specifically for automation. This account owns jobs and serves as the permissions center for production-level deployments.
How It Works
Unlike a service principal (a first-class identity in Databricks), a service account is a regular Databricks user account created solely for automation purposes. It uses Personal Access Token (PAT) authentication. The governance team copies the PAT from the service account in Databricks and configures it in the Fabric.
Security Note: The service account PAT is a long-lived credential. Treat it like a password — rotate it regularly and restrict access to only those who manage the deployment Fabric.
Best For
-
Organizations that prefer user-based identities over service principals.
-
Governance teams that centrally manage production credentials.
Adding New Users
Once your Fabrics and Teams are configured, onboarding new users is straightforward:
-
Add the user to Prophecy.
-
Assign them to the appropriate team(s) based on their role (e.g., individual_dev for developers, deployment_team for DevOps engineers).
-
The user automatically inherits access to all Fabrics associated with their team(s) — no additional configuration required.
Scenario Comparison
| Scenario | Auth Method | Best For |
|---|---|---|
| Interactive Development | OAuth (individual) | Multi-developer teams, usage tracking |
| Service Principal Deployment | OAuth (service principal) | Scheduled pipelines, production deployments |
| Service Account Deployment | Personal Access Token | Job ownership, centralized governance |
Learn more about Prophecy Fabrics in the help docs here.