Question-Driven Development

Treat your users' questions like a test suite for your semantic layer.

Question-Driven Development is a way of working for dbt teams in which the questions colleagues actually ask act as the test suite for the semantic layer: an unverified answer is a failing test, and a metric or saved query added in dbt makes it pass.

  1. Begin with the question. The real questions colleagues ask in Teams, not a list written in advance.
  2. An unverified answer (⚠️) is the failing test. The question isn't covered by a governed definition in the semantic layer yet.
  3. The definition added in dbt makes it pass (✅). The data team adds a metric or saved query, reviewed like any other code change.
  4. The coverage scan is the test run. The scan asks every question on your list through Mark Your Data Assistant and reports each one as ✅ governed (with the metric used), ⚠️ unverified, ❓ asked back or ⛔ refused. Run the scan before and after a dbt change to see the difference per question.

The scan ends with a summary such as Governed: 6 of 7 (before: 5 of 7).

✅ means “answered from your definition”, not “proven correct”. The data team checks correctness separately, with a set of reference questions and expected results that it maintains.

Where the questions come from

The questions come from Teams. In a pilot, Mark Your Data Assistant runs on your own cloud and colleagues ask their real questions in Teams from the first day.

The assistant's log shows every question, whether it was governed, and the 👎 reasons. The data team picks the unverified questions marked “Data is missing or incomplete” as the next definitions to add, and runs the coverage scan on the questions colleagues asked to see coverage go up after each dbt change.

The loop

  1. Ask in Teams
  2. ⚠️ Unverified
  3. 👎 with a reason
  4. Data team adds the definition in dbt
  5. Publish
  6. ✅ Governed for everyone
  7. The scan confirms coverage went up

The data team chooses the definitions and writes them. Mark Your Data Assistant labels the answers, collects the feedback and measures the coverage.

Questions and answers about the feedback loop

See Mark Your Data Assistant

Working with dbt and want to start with conversational analytics?

Start with a fit check to see whether Mark Your Data Assistant fits your stack, or with a one-quarter pilot on your own cloud.

Send me a message