Use caseReporting and analyticsAI product feature
Plain-English questions over your own database, with the SQL shown and checked
A text-to-SQL assistant for operations teams: plain-English questions answered from a read-only replica, with the SQL, the definitions used and a chart.
A blueprint, not a client story. The business described is illustrative; the architecture, integrations and trade-offs are real, and this is how I would build it. By Ergini, .
The short version
A plain-English question box over your own database, for operations teams who wait days for an analyst. A model maps each question to metric definitions kept in code; certified metrics compile to SQL, and anything else is written against curated views on a read-only replica. Every query is checked before it runs, and the database enforces each person's row permissions. Answers show the SQL, the definitions and a chart. Ambiguous questions get a clarifying question, and an analyst reviews uncertified queries.
- Best for
- Companies with a Postgres or BigQuery database of a few hundred tables, an operations team full of questions, and one or two people who write SQL.
- Connects to
- Postgres or BigQuery, dbt or plain SQL views, Metabase, Slack, Excel export, Google Workspace or Microsoft Entra ID
- The AI does
- Works out which defined metric, filters and dates a question means, writes SQL where no definition exists, and states the result in a sentence.
- People do
- The analyst owns the definitions and the evaluation set and reviews uncertified queries; managers decide who may see which rows.
- Built as
- AI Integration, usually $20K - $45K
Two hundred tables and one person who can read them
Take a freight forwarder with an operations team of twelve, a transport management system on Postgres with around 200 tables, and one analyst who also looks after the Metabase dashboards. The ops managers have questions every day. How many shipments to Poland missed their promised date last month, by carrier? Which customers shipped a third less this quarter? How long do pallets wait at the cross-dock?
Each question joins the analyst's queue in a Slack channel. The simple ones take ten minutes to write and two days to reach the top, and half come back with a follow-up that restarts the wait. By the time the number arrives, the carrier decision it was meant to inform has been made on instinct.
Text-to-SQL demos promise to fix this and look impressive on eight tidy sample tables. On the real schema they fall over: tables called shp_evt and cust_xref, three columns named status, soft-deleted rows, and words that mean different things in different departments. A wrong answer in ten seconds is worse than a right one in two days, because the person asking cannot tell it is wrong.
The appetite is real: in Anthropic's own 2026 survey of more than 500 US technical leaders, data analysis and report generation was the most-cited high-impact use of agents outside coding, named by 60 percent. What separates a useful build from a demo is everything around the model, so I build this as an AI integration into tools the team already uses, starting with Slack.
One word, three metrics
Most wrong answers are definition mistakes, not SQL mistakes. The fix is to write down, in code, what the business means by its words, and let the model choose between definitions instead of inventing one. In this company, 'active customer' alone means three things.
| Metric in the semantic layer | Definition, as the SQL implements it | Who means it |
|---|---|---|
| customers_active_shipping_90d | At least one shipment picked up in the last 90 days, cancelled and test bookings excluded | Sales and account managers |
| customers_under_contract | A signed rate agreement valid today | Finance and pricing |
| customers_active_portal_30d | At least one customer portal login in the last 30 days | Customer service |
| shipments_late | Delivered after the promised date, compared by calendar day in the consignee's local time. Shipments without a promised date are counted separately, never as on time | Operations |
| on_time_rate | Delivered on or before the promised date, divided by delivered shipments that had one | Operations and carrier management |
From a Slack question to a checked query
The model does two jobs: understanding the question, and writing SQL when no definition covers it. Everything that protects the data, and everything that decides whether an answer can be trusted, is code around it.
01 Trigger · Slack Events API
Someone asks
A mention in Slack, or the question box in the internal tool. The Slack user is mapped to a company account through SSO, which fixes their database role before anything else happens.
02 AI model · Structured output
Map the words to definitions
A model maps the question onto the semantic layer's catalog (metrics, dimensions, filters, time range) under a strict output schema, or returns a clarifying question instead.
03 Decision
Can it be answered as asked?
Code checks the mapping against the catalog.
- Every term maps to one certified metric, with a time range then compile the SQL from the definitions
- A term matches several metrics, or there is no time range then ask the clarifying question in the thread
- No definition covers it then draft exploratory SQL over the curated views, labeled uncertified
- It asks for regions or columns outside the asker's role then decline, and say who has access
04 AI model
Write SQL where no definition exists
Uncertified questions only. The model sees the curated views' documentation and a few dozen reviewed example queries, never the raw 200-table schema. Certified metrics skip this step, because code compiles their SQL.
05 Plain code
Validate before anything runs
A SQL parser confirms a single SELECT over allowed views and functions. Code adds a row limit, checks EXPLAIN against a cost ceiling (on BigQuery, a dry run and a byte cap) and sets a statement timeout. A failing query returns its reason, never a partial result.
06 System · Postgres read replica
Run on the replica, as the asker's role
A read-only role with row-level security policies limits the query to the asker's regions and customers. The database enforces it, so no wording of a question can widen it.
07 Plain code
Assemble the answer
Code renders the chart, the SQL, the definitions used and a certified or uncertified label, and a model adds one sentence from the aggregated result. Row-level results go to the asker as a table or an Excel file without passing through the model.
08 Person
The analyst reviews what was not certified
Uncertified queries and flagged answers land in a weekly review. Good ones become certified metrics; wrong ones become evaluation cases with the correct SQL attached.
09 Result
Answer in the thread, and log it
The answer posts with buttons to save it as a Metabase question or download it. Question, mapping, SQL, validation result and row count are logged per user.
A question from the ops channel, answered with its working
An illustrative exchange. The first question is ambiguous, so the assistant asks before it queries. The metric that follows is certified, so its SQL is compiled by code, then checked and run.
Slack, #ops-questions, Tuesday 09:12
Marta, operations · Slack
how many active customers do we have in poland vs a year ago?
- resolve_terms(question, catalog: "semantic_layer v14")'active customers' matches 3 metrics / 'poland' = customer_country 'PL' / 'vs a year ago' = same window, 365 days earlier / clarification needed
Data assistant · Slack
Which 'active' do you mean? 1. Shipped with us in the last 90 days 2. Has a signed rate agreement valid today 3. Logged into the customer portal in the last 30 days
Marta, operations · Slack
1
- compile_metric(metric: "customers_active_shipping_90d", filter: { customer_country: "PL" }, compare: "minus_365_days")certified metric / SQL compiled from the definition / one view, no joins
- validate_and_run(sql, role: "ops_cee")single SELECT on an allowed view / no personal columns / EXPLAIN cost within the ceiling / row policy for ops_cee applied by Postgres / 1 row in 380 ms on the replica
Data assistant · Slack
184 customers in Poland shipped with us in the last 90 days, against 161 in the same window a year ago, up 14%. Definition: customers_active_shipping_90d, at least one pickup in the last 90 days, cancelled and test bookings excluded. Certified by the analytics team. SQL: SELECT COUNT(DISTINCT customer_id) FILTER (WHERE picked_up_at >= CURRENT_DATE - 90 AND picked_up_at < CURRENT_DATE) AS active_now, COUNT(DISTINCT customer_id) FILTER (WHERE picked_up_at >= CURRENT_DATE - 455 AND picked_up_at < CURRENT_DATE - 365) AS active_year_ago FROM analytics.shipments_v WHERE customer_country = 'PL' AND booking_status NOT IN ('cancelled', 'test'); Chart: active customers in Poland by month, last 13 months. Save to Metabase or download as Excel.
- Certified metric, so no analyst review. The definition is printed with every answer, so nobody has to remember which 'active' was meant.
Fan-out joins, slow scans, personal data and other traps
These are the failures that make people stop trusting a text-to-SQL tool. Each has a specific guard, and most of the guards are ordinary engineering.
Joins that multiply rows
Join shipments to their legs or invoice lines, and a count of shipments silently becomes a count of legs while freight revenue doubles on every two-leg shipment. Each curated view has one declared grain, the validator flags aggregates across joins that change it, and the evaluation set includes questions only a fan-out would get wrong.
A slow query on the wrong database
A model will happily write a query that scans three years of tracking events. Queries run only on the replica, or on BigQuery under a maximum bytes billed cap, behind a statement timeout and an EXPLAIN ceiling. An expensive query comes back with a suggestion to narrow the date range.
Personal data where it is not needed
Consignee names, phone numbers and addresses are personal data under GDPR, and almost no operational question needs them. Curated views leave those columns out unless a role requires them, row-level results skip the model, and the model runs in the EU under a data processing agreement, as in any GDPR-compliant AI build.
Last month, in whose timezone?
Timestamps are stored in UTC, but a delivery at 23:30 in Warsaw is on time or late depending on which calendar day counts. Date logic lives in the metric definitions, so the model never improvises timezone arithmetic for a certified metric.
The schema moves under it
A migration renames a column and yesterday's example queries break. The semantic layer is tested in CI against the replica, a nightly job compares information_schema with the documented schema and alerts the analyst to drift, and the evaluation set runs on every change.
Instructions hidden in the data
Free-text fields such as delivery notes can hold text written to steer an AI. The model sees aggregated results only, never free-text columns, the role cannot write, and anything but a single SELECT is rejected, so an injected instruction has nothing to act on. The prompt injection guide covers the wider pattern.
The evaluation set that decides whether it can be trusted
This is the part most text-to-SQL projects skip. LangChain's own 2026 survey of about 1,300 agent builders found that only about half run offline evaluations at all. Here the set is built before launch and gates every change after it.
- Collect 150 to 200 real questions from the Slack channel and the analyst's queue, badly worded ones included.
- For each, the analyst writes the correct SQL and stores the expected result against a fixed snapshot of the replica, so expected answers do not drift as data arrives.
- Tag each question by type: certified metric, exploratory, ambiguous (the right response is a clarifying question) or out of bounds (the right response is a refusal).
- Score result sets, not SQL text. Two different queries that return the same numbers are both right; a plausible query that returns different numbers is wrong.
- Run the whole set on every change to a view, a definition, the prompt or the model version. A drop in accuracy on any question type blocks the release.
- Each week, add the questions that went wrong in production, once the analyst has written their correct answers.
The same set makes switching models safe: a cheaper or newer model is adopted only when it matches the current one on these questions. The evaluation framework comparison covers the tooling.
Who decides what, from question to chart
The AI model
Map the question to metrics, filters and dates
Understanding loosely worded questions is what language models do well, and the catalog keeps the options finite.
Write SQL for questions no definition covers
Useful for exploration, always labeled uncertified, always validated.
Say the result in one sentence
From the aggregated figures only, printed next to the SQL that produced them.
Plain code
Compile certified metrics into SQL
A definition that compiles the same way every time cannot be misremembered.
Validate every query before it runs
Parsing, allowlists, limits and cost ceilings are rules, not judgement calls.
Enforce row-level permissions
Done by the database role, so a cleverly phrased question cannot widen access.
A person
Own and certify the definitions
What 'active' means is a business decision, made once, by the analyst and the department that uses it.
Review uncertified queries every week
Good ones become certified metrics; wrong ones become evaluation cases.
Decide who sees which rows
Access follows the org chart and the data protection assessment, not the tool's defaults.
Your BI tool's own AI, or a build on your own definitions?
Check what you already pay for first. Metabase has Metabot, which turns plain-language questions into charts and SQL, and Looker, Power BI with Copilot, ThoughtSpot with Spotter and Hex all offer natural-language questions in some form. They work best on data already modeled inside them, so if your definitions live in LookML, a Power BI semantic model or Metabase models, switching the feature on is the cheapest experiment you can run. For many teams it is enough.
A build wins when definitions and permissions are the hard part: metrics that span systems your BI tool does not model, row access that must follow rules from your own application, answers that need to live in Slack or an internal tool rather than behind a BI seat, or an evaluation set and logs you control. The same governed queries can later be offered to Claude or Copilot through an MCP server.
The hybrid is usually right: keep Metabase for dashboards, move the definitions into dbt or SQL views that both the BI tool and the assistant read, and let the assistant save good answers back into Metabase. If the questions keep circling a spreadsheet everyone edits by hand, the better first project may be turning that spreadsheet into an internal tool.
How you would know it is working
A blueprint has no results to report, so here is what I would measure from the first week instead, on your own data.
- Evaluation accuracy
- Share of the evaluation set answered with the correct result, per question type and per release.
- Clarification rate
- How often the assistant asks before answering. Very low suggests it is guessing; very high means definitions are missing.
- Analyst queue
- Questions waiting for the analyst each week, and the median wait, before and after launch.
- Uncertified share
- Share of questions answered with exploratory SQL. A falling share means the catalog is catching up with what people actually ask.
- Flagged answers
- Answers marked wrong by the asker or the analyst, each traced to its cause: the definition, the SQL, the data or the question itself.
What a build like this costs
This is built as AI Integration, which runs $3.5K - $45K overall. A build like this one usually lands in the agentic feature tier: $20K - $45K, 4-8 weeks. The first working version runs on your real data well before the end of that window.
What it costs to run
Model costs are a few cents per question: one or two calls with the catalog in context, and prompt caching keeps the repeated schema text cheap. The larger variable is the database, either a Postgres read replica or bytes scanned on BigQuery, and the per-query caps keep that predictable.
What moves the price
- The state of the schema: a documented dbt project is a head start, while 200 undocumented tables mean weeks of definitions work with the analyst
- Row-level security: mapping SSO groups to database roles and testing that each role sees exactly its rows
- How many certified metrics are needed at launch, and how many departments must agree on them
- Where it lives: a Slack assistant is quickest, and an internal tool or customer portal adds interface and login work
- The size of the evaluation set, and the analyst time its reference answers take
Who this is for
- Logistics and SaaS companies with a Postgres or BigQuery database of a few hundred tables and one or two people who write SQL
- Operations teams whose questions wait days in an analyst's queue
- Companies that tried a text-to-SQL tool on their real schema and got confident wrong answers
- Teams where departments mean different things by 'active', 'late' or 'revenue'
- Businesses that need row-level permissions, so a country manager sees only their own customers
Questions people ask about this
Can AI query a database accurately in plain English?
On a well-defined layer of your data, yes; on a raw schema of hundreds of tables, not reliably. Accuracy comes from curated views and metric definitions for the model to choose from, certified metrics compiled in code, every query validated before it runs, and an evaluation set of your own questions. It also has to ask when a question is ambiguous.
Is it safe to let an AI run SQL against our production database?
Not against the production primary, and never with write access. Queries run on a read replica or a separate warehouse dataset, under a role that can only read curated views, with a statement timeout and a cost ceiling. A parser rejects anything but a single SELECT, and row-level security in the database limits each person to their own rows, whatever the question says.
What is the difference between text-to-SQL and a semantic layer?
Text-to-SQL means a model writes SQL straight from the question and the schema. A semantic layer is a set of named metrics and dimensions defined once in code, in dbt, Cube or plain SQL views, that compile to SQL. This build uses both: questions map onto the semantic layer, and raw SQL is written only when no definition exists, labeled uncertified.
Does Metabase, Power BI Copilot or ThoughtSpot already do this?
All three offer natural-language questions over data modeled inside them, and if your definitions already live there, try that first. A custom build is worth it when definitions span systems the BI tool does not model, when permissions come from your own application, or when answers need to live in Slack or your internal tool with an evaluation set you control.
How long does it take to build an AI SQL assistant?
Usually four to eight weeks, and most of that goes on definitions, permissions and the evaluation set rather than the model. A Slack assistant over the first twenty or so certified metrics can answer real questions early in the project, and exploratory SQL is switched on once the evaluation set shows it holds up. The price tier is shown on this page.
Can the results be exported to Excel?
Yes. Row-level results come as an .xlsx or CSV file generated by code, not by the model, with the SQL and the definitions on a second sheet so the numbers can be traced later. Exports follow the same row-level permissions as the answer in Slack, and every download is logged.
Sources