AI over governed enterprise data

Problem

Business users wanted natural-language access to governed data. A chatbot on top would have bypassed existing access groups, lineage and curated marts. Genie spaces also had no supported way to be source-controlled, and the space metadata was edited by hand in the UI by the product owner.

Work

• Designed the Genie semantic layer: business metadata, join criteria and metric views, so queries resolve to the same definitions as dashboards.
• Built Databricks-backed MCP servers for Claude, scoped to curated data.
• Kept access control in Unity Catalog: Databricks group grants down to tables, with row-level security and column masking beneath them.
• Built a CLI in one sprint, run from GitHub Actions, that source-controls Genie spaces. Instructions, join criteria, sample questions, filters and data values are defined in files and deployed to the space.
• Deployed the metric views as Databricks Asset Bundle resources from GitHub Actions.
• Rolled it out on two tracks. A human-maintained space served a pilot group; I mined their questions, agent outputs and logs and fed them to Claude to build better evaluation sets and instructions. Those went into a CLI-maintained space, auto-deployed to the pilot group and the remaining users. The original space was then retired and everyone moved to the source-controlled, CI/CD-deployed space. The product owner kept a space for sandboxing, maintained with engineering help, and the CLI switches between and deploys spaces.
• Built a CI/CD evaluation harness on the Genie eval-run API. Golden SQL queries and questions with evaluation criteria run against each space.
• Built a feedback loop: a separate pipeline uses an agent to cluster questions and aggregate thumbs-up and thumbs-down responses, and a dedicated chat channel receives traces.
• Closed the loop on data defects: Genie diagnosed the defect behind a bad answer and wrote a handoff prompt, and Claude opened a PR against the pipeline that builds the affected curated table. When a fix needed a backfill, one was triggered; backfills were expensive, so they were avoided where possible.
• Worked with business leaders on answer quality. For questions such as policies in force on a date versus over a period, I instructed the agent to reconcile across the time frame.

Request path

• Access is enforced in Unity Catalog and does not depend on what the model was told. Databricks groups hold grants on catalogs, schemas and tables, with row-level security and column masking beneath them.
• Row-level security limits each user to the rows they are entitled to see.

Result

• Business users queried governed data through these endpoints, and the pilot group moved to the source-controlled space.
• Latency was the main limitation, from the serverless SQL warehouse and the Genie API.
• No ungoverned copy of the data was created.