Are you a datamaxxer?
A free AI data readiness test your agent runs on your own data. Give it read-only credentials, and it runs 8 quick checks on the data behind one question your business asks (29 if you go deeper), then gives you a score and what to fix first.
Paste this into your agent
Why ask your agent? It won't sugarcoat it. Every point needs evidence, and when it's unsure, it takes the lower score. We never see your data or your score.
The copied prompt carries the whole test (about 7,300 words), so your agent doesn't need this page or internet access. Read it as plain text.
8
checks, about 10 minutes
29
checks to go deeper
1–2
questions for you
Human? You can stop here.
Everything below is for your agent. It’ll tell you whether you’re a datamaxxer.
Curious anyway? Scroll on. It’s the whole test, nothing hidden.
Agent, start here
Ground rules
- 01Read-only. Read catalogs, schemas, query and job logs, bills, incident records and access policies. Never write, delete, rerun a production job or change a permission while taking this test. This is a rule you keep, not a lock: if your access could write or delete, tell your human in Step 0 and suggest read-only credentials.
- 02Evidence or zero. Every score of 1 or 2 cites its evidence: the query you ran, the file, log or policy you read, or the person who confirmed it. Each check names where to look. If you could read that place and the thing isn't there, it scores 0. In the rubrics, "unknown" means nobody can tell even though you could read where the answer would be.
- 03No access is not a zero. The first place in a check's "Look in" line is its main place. Settle the status in this order. (1) Anything you read, anywhere, already settles the status: score it rather than marking it no access. A 0-level fact, such as "computed differently in places", settles the score at 0 even when your view was partial; a positive finding on a partial view is still capped at 1, as in (3). One exception: a fact about one backup doesn't settle a check about backups while another backup of the same data may exist, so read the backup tool's own records first. (2) You couldn't read the main place: no access, left out of the score and listed as an access gap. (3) You could read only part of it, such as the IaC but not the live policy, or only the design and code but not the live system: that is a partial view. Score what you saw, at most 1, and list the rest as an access gap. (4) You could read it and the thing isn't there: 0, with any other places you couldn't read listed as access gaps. A main place that doesn't exist counts the same: once the cloud account or your human shows there is no backup tool, the checks that look there first (E2, E3, R2 to R5) are scored from the rest of their "Look in" line, and 0 where nothing there has it, not no access. An empty result from a metadata or metering view usually means a missing privilege: treat that place as unreadable unless you can show the object exists.
- 04Your own work counts, up to a point. What you measure, find or estimate during the test fills any part of a rubric that asks for something to be measured, known, found or estimated. Everything else a rubric asks for (tracked, tested, written, owned, recomputed, retired) must already be true without you. Your work alone can lift a check to 1, never to 2. The score describes the data and the team, not your effort today.
- 05Ask first. Ask your human before requesting access, running anything that costs money, writing anywhere (even a scratch copy), restoring anything, or contacting another team. Batch the asks into one message. Read-only queries on a warehouse you were given are fine; ask before anything that provisions capacity or runs for long. When a check needs one of these and you have no answer, use the check's fallback; where it names none, score what you could see, at most 1, and say what was missing.
- 06Stay on the question. Every check is about the data behind one business question (Step 1), not the whole estate. That data runs through layers, from the source system through loaders and copies to the main table. Pick the layer per check and name it in the finding: Lean checks usually land on the loader, Understood on the main table and its docs, Reversible on the layer you would restore from.
- 07Keep data where it is. Report counts, names and timings. Don't copy rows, credentials or personal data into your report.
- 08One reading, at the finish. Your score is how things stand when your run finishes. Suggest scheduled follow-ups where they help, such as a re-run once fixes land; they never change this run's score.
- 09Time-box. Keep queries cheap: filter by date, sample with limits, don't scan large tables in full. When a check would take far longer than the rest, score what you found, say what's missing and move on.
Modes
Quick first, full to go deeper
- Quick mode is the default. 8 checks to a tier in about 10 minutes: M1, M2, U1, U3, E3, E4, R2 and R4, marked Quick in Step 2. Run Steps 0 to 5 on those 8 only. Run the full test only when your human asks for it.
- Full mode goes deeper. All 29 checks. After a quick run, keep the same business question and definition, run the other 21 checks and score all 29 as a new full run; a quick finding carries over only if it is still true when the full run finishes.
- Label the mode. Step 3 says how a quick run is scored. Put the mode next to the tier in the report and in the JSON's
mode. A quick tier is a first reading: say so, offer the full test as the next step, and compare a quick run only with another quick run.
Step 0
Check what you can read
Before Step 1, find out what you can actually reach, so your human hears what's missing once, at the start, and not after the checks.
- List what you can read. A meaningful run needs read access to the warehouse and the repo, plus read-only cloud access (billing, IAM and the backup console where possible). Mark each place the checks look in as readable, partly readable or not readable: the warehouse with its query and load history, the orchestrator and pipeline code, the bill or metering views, the incident tracker, access policies, the backup tool and the cloud's resource inventory, and the catalog, docs, chat and tickets. Having no backup tool at all is a finding, not an access gap (rule 3).
- Estimate coverage. From the checks' "Look in" lines, count how many of the checks in your mode (8 quick, or all 29) you could score.
- Ask once. Send your human one message: what you can read, the estimated coverage (below 60% means a low-confidence result), whether your access could write or delete (rule 1), and the access that would open the most checks, with who could grant it. Set expectations in the same message: about 1 or 2 questions in all (this one if access is missing, and which business question to use in Step 1), and where nobody answers you go on with a marked assumption.
- Adapt or stop. Go on with what you can read while you wait, and pick up the checks any new access opens. If you can't see the bill, IAM or the backups, score from the repo, the IaC and the warehouse's metadata instead: metering and job history for cost, grants and IaC policies for access, and IaC and backup configs for backups. That is usually a partial view, so 1 at most, and what stays unreadable becomes an access gap (rule 3). If you can read neither the data nor any docs about it, stop here and report what you checked and the access you'd need, with no score.
Step 1
Anchor on one business question
Datamaxxing is measured against a question the business actually asks. Pick one, write it down, and run every check against the data behind it.
Path A
Your human names it
Ask: "Which question should our data answer better: one a decision depends on, that broke at least once this year, and that someone senior noticed?" Then sharpen it until it names a table and a number, like "which accounts are about to churn this quarter?" If they name the question but not the decision it drives, infer the decision and mark it assumed.
Path B
You infer it from usage
- Questions your human asked you more than once.
- The most-run queries and most-read tables of the last 30 days.
- The most-viewed dashboards and the reports with a meeting attached.
- Recurring data questions in tickets or chat, and the datasets with the most incidents.
Leave your own queries and earlier self-test runs out of usage history; if that is all you can see, use the fallbacks. If all the history you can see is under your own identity, treat it all as yours. No usage logs you can read? Use what points at usage instead, in this order: dashboards and BI or semantic-layer definitions, roles and services granted read access, references in code and docs.
Propose up to three candidates with the evidence for each, and ask your human to pick. If nobody answers, take the one with the most evidence of use and mark it assumed, meaning inferred and not confirmed.
Question
One sentence, answered by a number or a ranking.
Decision
What someone does differently with the answer.
Main table
The table the answer is read from.
Consumers and layers
Who reads it, and the path from source system to the main table.
Today's answer
The value, and its exact definition: population filter, numerator and denominator for a rate, time column, window, unit, how open records count. Always decide attempts or entities: does a retried job count once or every time? List that and every other choice that moves the number, with its alternative value, and when you read it. A later run must reproduce the definition; for a table that changes by the hour, it needn't reproduce the value.
Source
Named by the human, or inferred, with the evidence and any assumptions.
Step 2
The checklist
29 checks in five groups. Each says where to look, the evidence to collect, what scores 0, 1 or 2, and the fix pattern to reach for if it falls short. The 8 marked Quick are quick mode: M1, M2, U1, U3, E3, E4, R2 and R4.
Field note
Loaders that rebuild the whole table
Many warehouse copies are dropped and reloaded from the source on a schedule. The warehouse's table history lists each reload and the dropped versions it still keeps. Six checks read differently there:
- M2: there is no per-event landing time. Event to queryable is roughly half the reload interval plus the load time at p50, and the full interval plus the load time at p95, on top of any delay before the source has the event.
- L2: rerunning a past day can't reproduce it once the source has changed. Score the dedupe and update rules from the design, 1 at most.
- L3: each reload picks up late rows in the copy. Whether closed days are corrected is then a question about the source; score the source if you can read it.
- L4: a reload carries any schema change straight through unless a check sits between source and copy.
- L5: compare rows written per reload with rows that actually changed. That ratio is the waste.
- R3: a bad write to the copy heals on the next reload, so that isn't a restore path. Score the source, where a bad write would stay.
Group 1 · 6 checks
Measured
You can't datamaxx what you haven't measured: freshness, jobs, cost and breaks, with a target.
M1Quick
Can you list every dataset the answer reads, with an owner for each?
Evidence: lineage from the catalog, or the from and join chain traced back from the main table. An owner is a team or person accountable for fixing the dataset; an owning role, a reviewer group or a project name is not.
Look in: the data catalog or lineage tool, warehouse grants, ownership files in the repo.
Fix pattern: lineage in the catalog, one named owner per dataset.
0
Can't list them, or no dataset has an owner.
1
Listed, and some but not all have owners.
2
Complete, every dataset owned, found without asking anyone.
M2Quick
Do you know how long an event takes to become queryable, and how fresh the question needs it?
Evidence: landing times joined to the finish time of the first build that includes them; p50 and p95, split into landing delay and pipeline delay. For a loader that replaces the whole table, its load history gives cadence and duration.
Look in: ingest or landing timestamps, the warehouse's load or table history, the orchestrator's run log.
Fix pattern: a scheduled freshness query; fix the schedule before the source when the delay is the pipeline's.
0
Unknown.
1
Estimated, or measured once.
2
p50 and p95 measured and split, against a stated need, and tracked.
M3
Is there a complete inventory of the jobs between the sources and the main table?
Evidence: the orchestrator's job list walked back from the main table; per job its schedule, full rebuild or incremental, owner, and whether it only cleans up raw data.
Look in: the orchestrator and any ELT tool configs, loader code in the repo.
Fix pattern: a job inventory generated from the orchestrator, kept as code.
0
None.
1
Partial, from memory, or rebuilt by you from metadata.
2
Complete, generated, cleanup-only jobs marked.
M4
Can last month's compute and storage cost for those jobs be attributed?
Evidence: the bill grouped by job tag; for the biggest line item, what runs on it (big batch, many small jobs, streaming).
Look in: the cloud bill or cost export, warehouse metering views.
Fix pattern: cost-allocation tags per job; until then, node-hours per run times runs times rate.
0
Can't be attributed.
1
Estimated from the run log.
2
Tagged per job and split by workload type.
M5
Are the last 30 days of data breaks on record: what broke, who noticed first, how long the fix took?
Evidence: one incident log, each break typed as wrong, late or missing data, with hours to notice and to fix, and whether a customer found it first. A break is a wrong, late or missing value of the answer; count it if it was live at any point in the last 30 days.
Look in: the incident tracker or on-call tool, postmortems.
Fix pattern: a written definition of a break and one incident template.
0
No record.
1
Scattered across chat and tickets.
2
One log, typed, with notice and fix times.
M6
Is there one numeric target for this data, with a guardrail?
Evidence: a written target like "event to queryable p95 under 2 hours", next to the baseline, plus "cost and breaks don't get worse". A display threshold in code isn't a target unless someone committed to it.
Look in: the table doc or catalog entry for the main table; also planning docs and the tracker.
Fix pattern: one target, one guardrail, both next to the baseline numbers.
0
No target.
1
A target, but not a number or no guardrail.
2
Numeric target, guardrail and baseline written down.
Group 2 · 6 checks
Lean
Do the work once: no repeat cleaning, no silent drops, no nightly rebuild of what didn't change.
L1
Do you know which fields the question reads, and which the pipelines process anyway?
Evidence: fields in the query logs against fields in the pipeline code. Processed but never read is cost; read but no longer parsed, often after a rename, is a silent bug.
Look in: query history, the code of each consumer, the pipeline's transforms.
Fix pattern: drop unread fields; parse the renamed ones.
0
Unknown.
1
Partly known.
2
Computed from logs and code; no field read but unparsed.
L2
Is there one rule for duplicates, updates and deletes, and does rerunning a day give the same output?
Evidence: the dedupe key each team uses on the same source; a past day rebuilt in a scratch location and diffed against what was published. Without approval to write a scratch copy, score from the design: a single key with idempotent upserts is a 1 at most.
Look in: pipeline code (merge keys, upserts), load history, a past day's published output.
Fix pattern: one dedupe key; idempotent writes that merge on it.
0
Unknown, or reruns change the output.
1
Written down but untested, or teams differ.
2
One key; a rerun reproduces the published day.
L3
What share of records arrive after the processing window closes, and does anything correct them?
Evidence: event time against landing time against the window; whether any job recomputes a closed day.
Look in: event and landing times in the data, job code that recomputes closed days.
Fix pattern: a watermark on landing time; recompute only the partitions late rows touch.
0
Unknown.
1
Measured, never corrected.
2
Measured, and closed days are recomputed.
L4
Would a source renaming a field, or adding a new value, be caught before wrong numbers go live?
Evidence: the last schema change on a source, what broke, who found it and how long wrong numbers were live. A new status or type value can silently change a rate.
Look in: source schema migrations or history, ingest contracts, the incident tracker.
Fix pattern: a schema contract checked at ingest.
0
No; a reader would find it.
1
Monitoring catches it after it lands.
2
A contract or check stops it at ingest.
L5
Has a nightly full rebuild been replaced by an incremental update that is proven to match?
Evidence: rows scanned per run, runs per month, and an equality check between the incremental and full versions. Distinct counts, like weekly actives, are the usual trap.
Look in: job definitions, table history (rows written per load).
Fix pattern: recompute only what new rows touch; keep the equality check as a test.
0
All full rebuilds, none assessed.
1
A candidate found, savings estimated.
2
At least one incremental, equality check in place.
L6
When a record fails processing, is it kept with a reason, does someone hear about it, and can it be replayed?
Evidence: one failed record traced end to end; record counts in and out of each step. A gap nobody can explain means silent drops. A consumer that swallows a failed query with only a log line is a silent drop too.
Look in: pipeline code and error tables, alert rules, dead-letter queues.
Fix pattern: a dead-letter queue with a reason, an alert, and a replay path that needs no ticket.
0
Dropped silently, or unknown.
1
Kept, but no alert or no replay.
2
Kept, alerted, replayable; counts reconcile.
Group 3 · 7 checks
Understood
A colleague, or you, can answer from the main table without asking the person who built it.
U1Quick
Is it written down what one row of the main table is, and is that true?
Evidence: the one-sentence grain ("one account per day") and a query that finds zero duplicate keys. Group by the full key and count; some engines' COUNT(DISTINCT a, b) skips rows with a null in any column.
Look in: the table doc or catalog, and the table itself.
Fix pattern: a one-page table doc; a uniqueness test on the key.
0
Not written.
1
Written, or checked once.
2
Written, and a uniqueness test runs on every load.
U2
Are the five columns people ask about most defined, with meaning, unit and business term?
Evidence: the five chosen from real questions (query logs, chat, tickets), and their definitions where readers find them.
Look in: catalog column comments and docs; query history, chat and tickets for what gets asked.
Fix pattern: column definitions in the catalog, next to the table.
0
None defined.
1
Some, or picked from memory or consumer code rather than real questions.
2
Five, picked from real questions, in the catalog.
U3Quick
Is each key metric's formula written, with additive and point-in-time metrics marked?
Evidence: the formulas, and every place the metric is computed. Summing a daily balance like ARR over a month overstates it about 30 times. For a ratio, the numerator and denominator must count the same population.
Look in: the metric layer, model code, BI definitions, docs.
Fix pattern: one metric definition, used everywhere it is reported.
0
Not written, or computed differently in places.
1
Written for some.
2
All written, marked, one definition in use.
U4
Are the table's freshness, null rates and distributions profiled, with an alert when they shift?
Evidence: the alert rule and a backtest on last month: would it have caught the last incident early, and how often would it have fired for nothing? An alert whose history you can't read is a 1 at most.
Look in: data-quality tests and monitors on the main table; also alert rules.
Fix pattern: alert on the daily inputs against a trailing median, not on rolling averages that hide a shift for days.
0
No profile, no alert.
1
A profile, or an alert on rolling metrics.
2
Alert on daily inputs, backtested.
U5
Are the segments that behave differently written down?
Evidence: regions with their own rules or upload schedule, accounts with outsized volume, and what each means for the target.
Look in: the table doc, and the data itself.
Fix pattern: a segments section in the table doc, with per-segment targets where needed.
0
Not known.
1
Known informally.
2
In the doc, with their effect on targets.
U6
Do schema changes, pipeline changes and shifts in the data reach the people and agents that read the table?
Evidence: where definitions live, who or what updates them, and how a breaking change is announced or blocked.
Look in: the catalog, docs, the channel where data changes are announced.
Fix pattern: definitions kept in the catalog next to the table, so they reach agents the way they reach people; a named owner.
0
Nothing reaches readers.
1
Ad hoc announcements.
2
In the catalog, with a named owner or automation.
U7
The outsider test: can you answer questions about the data from the docs, catalog and semantic layer, without looking at the data?
Evidence: five questions about the data behind the business question: what the main table and its key columns mean, how the key metric is defined, where a given fact lives, the grain, and how fresh it is. Answer each from the docs, catalog and semantic layer alone and write down the answer with the doc it came from, before you query anything. Then check each answer against the data with a read-only query and count how many held up. An answer that needed code or a guess didn't hold up.
Look in: the docs, catalog and semantic layer for the answers; the data only to check them.
Fix pattern: every answer that didn't hold up, or that you had to guess, goes into the table doc or the catalog.
0
No docs to answer from, or 2 or fewer of the 5 held up.
1
3 or 4 held up.
2
All 5 held up against the data.
You are the outsider here: answer first, look second. Answer from what the metadata says, not from what you saw in the data or code earlier in this run. Cite the doc behind each answer and the query that checked it. If you can't read the data, score 1 at most when the docs answered all five, mark the answers unverified and list the data as an access gap.
Group 4 · 5 checks
Ready and cheap
What it costs to keep the data useful: volume, retention, copies, formats and peaks.
E1
Do you know which events make up most of the volume, and is any sampling rule proven safe?
Evidence: the volume mix; if sampling, the answer per account or segment before and after, not just the total.
Look in: the data itself, ingest config.
Fix pattern: sample with weights; exempt small segments and critical events.
0
Volume mix unknown.
1
Known; no rule, or a rule never validated.
2
A validated rule, or a written decision not to sample.
Billing, financial and audit data are usually never sampled. That is a 2 once the decision is written down, and a 1 until then.
E2
Is there a retention period for raw events, summary tables and old versions, each with an approver?
Evidence: days kept for each of the three, who approves deleting them (a named person or team who must say yes first), and whether deletion requests reach summary tables and snapshots too.
Look in: the backup tool's policies and retention settings; also storage and warehouse settings and the deletion-request procedure.
Fix pattern: a retention table with approvers; deletion requests that cover every copy.
0
None of the three set on purpose: kept forever, or on tool defaults.
1
At least one set on purpose, but an approver or deletion-request coverage is missing.
2
All three, approvers named, deletion requests covered.
E3Quick
How many copies of this data exist, and does any pipeline copy what a backup already holds?
Evidence: team copies, dev clones, exports and backups. Exports and spreadsheets can only be estimated, and an estimate is fine. A copy that serves another engine or team still counts; it is redundant if one shared copy could serve both.
Look in: the backup tool's inventory; also warehouse object lists, storage buckets and export jobs.
Fix pattern: retire unread copies; let one shared copy serve both backup and analytics.
0
Unknown.
1
Counted or estimated.
2
Counted, and no redundant copy left: each is retired or serves more than one job.
E4Quick
Can every query engine you use read these tables through one catalog, in an open format, from storage in your own account?
Evidence: the same table read by two engines with matching row counts; file counts and sizes per partition. Score the main table, and mention any open copy elsewhere in the finding.
Look in: table format and catalog config, each engine's connection settings.
Fix pattern: an open table format in a shared catalog; scheduled compaction.
0
Locked to one engine, or unknown.
1
Open format, but catalog gaps or small files.
2
Two engines agree; files compacted.
E5
What happened at the last traffic peak, and what does the capacity held for it cost?
Evidence: peak lag, hot keys, behaviour when a worker failed; (capacity provisioned minus capacity needed at the median hour) times hours times rate.
Look in: lag and throughput metrics, capacity config, the bill.
Fix pattern: capacity that scales with load; spread hot keys. Hot keys survive a move to incremental.
0
Unknown.
1
Peak known, its cost not.
2
Both known, and the cost is a deliberate choice.
When you can see that nothing is sized up for a peak (shared or serverless capacity, or a scheduled full reload with no traffic peak), mark this not applicable with the reason. If you can't see the capacity and the load pattern doesn't rule out a peak, it's no access. E5 is the only check that can be not applicable.
Group 5 · 5 checks
Reversible
Whether the data is protected at all, how fast a bad write is noticed and undone, and who can make one. You are one of the agents this group is about.
R1
Is every agent and automation that can write to or delete this data listed, with who approves its risky actions?
Evidence: roles and service accounts with write or delete rights on the main table and its sources, read from access policies. Include yourself. Count individual people; don't name them. An approver is a named person or team who must say yes before the action runs; whoever reviews the code that does it is not.
Look in: access policies: cloud IAM, warehouse grants, service accounts, agent configs.
Fix pattern: least privilege; an approval step for deletes and bulk writes.
0
Unknown.
1
Partial list.
2
Complete, each with an approver.
R2Quick
Do the backups have their own credentials, so that nothing with production access can delete them?
Evidence: the backup's access policies read directly, for the layer you would restore the main table from. Warehouse time travel is not a backup. A lock that blocks deletion beats delete rights: if the policies allow deletion but you can't read whether a lock is on, that is a partial view, so 1 at most. This is often an access review with a security team; a partial answer still counts.
Look in: the backup tool's own records of where its copies live, who can delete them and whether they're locked; also the access policies and the IaC behind them.
Fix pattern: backups in a separate account or vault, immutable for their retention period.
0
Unknown, or production can delete them.
1
Separate, not verified.
2
Verified from the policies.
R3
How long does it take to put back 50 bad rows, without restoring the whole database?
Evidence: a timed restore of 50 rows from snapshots, time travel or backups into a scratch location, or the documented procedure and the time of the last real restore. Restore history that includes row- or record-level restores with their durations counts as a timed drill.
Look in: the backup tool's restore history, with job types and durations; also restore runbooks and snapshot tooling.
Fix pattern: row-level restore from snapshots, drilled before the incident.
0
Full restore only, or unknown.
1
Possible, but untimed, taking hours, older than 90 days, or mostly failing.
2
A timed drill or restore history from the last 90 days, mostly succeeding, in minutes or less.
If a loader recreates the table on every run, warehouse time travel holds only the versions within its retention; list them from table history, and look one layer down. Never restore into production for this test. Run the drill into a scratch location only with your human's approval; otherwise score it from the procedure and the last real restore. "We can only do a full restore, and it took N hours" is a valid answer.
R4Quick
Is every source behind the question backed up, and how old is the last good backup?
Evidence: the sources' resources listed from the cloud's own inventory, not from tags, matched against what the backup policies cover; the time of the last successful backup of each, against how much data the question can afford to lose.
Look in: the backup tool's inventory and policies; also the cloud's resource inventory.
Fix pattern: policies that match on what exists rather than on tags, and an alert when a source is left uncovered.
0
Some sources have no backup, or nobody can tell.
1
All covered, but through tags or a hand-kept list, or the last good backup is older than the question allows.
2
All covered by policies that pick up new resources, each with a recent enough good backup.
R5
Would a mass delete, overwrite or encryption of this data be noticed before the clean copies age out?
Evidence: what watches for sudden drops in rows or objects, mass changes or encrypted content, on the source or in the backups; when it last fired; how long clean copies are kept, against how long such damage could go unnoticed.
Look in: the backup tool's anomaly or threat alerts; also data-quality monitors and alert rules.
Fix pattern: anomaly checks on every backup, with retention longer than the time to notice.
0
Nothing watches for it, or clean copies would age out first.
1
Counts or the source are watched, but not the backups, or the alert has never fired or been tested.
2
Every backup is checked for mass deletion and encryption, alerts reach someone, and retention outlasts the time to notice.
Step 3
Score yourself
Score how things stand when your run finishes. Each check scores 0, 1 or 2, or is marked no access and left out. Each group counts the same, however many checks it has, so the five reversibility checks can't be outvoted by the rest. A quick run applies the same rules to its 8 checks.
0–34
Datasitter
The data sits around doing nothing, and nobody can say what it costs or how it breaks.
35–59
Datacurious
Some numbers exist. Most answers still live in one person's head.
60–79
Datamaxxer in training
Measured and mostly understood. A few fixes away from running on its own.
80–100
Datamaxxer
Measured, lean, documented for agents, cheap to keep and quick to undo.
assessed = checks scored 0, 1 or 2 (not "no access" or "not applicable") group score = points in the group / (2 × assessed checks in the group) → whole %, 0 to 100 overall score = average of the groups with at least one assessed check → whole %, 0 to 100 coverage = assessed checks / (checks in the run − not applicable) → below 60%: low confidence mode = quick: the 8 quick checks, coverage out of 8; full: all 29 rounding = average the unrounded group scores, then round once partial = checks scored on a partial view; report the count next to coverage
Gates
- No "Datamaxxer" while any Reversible check in the run scores 0 or wasn't assessed: R2 or R4 in quick mode. One bad write you can't undo outweighs everything else.
- No score at all without the Step 1 fields filled in.
- A group with no assessed checks shows "not assessed" and stays out of the average. A group with fewer than half its checks assessed is marked low confidence. In quick mode, Lean has none of the 8 checks: it shows "not in quick mode", stays out of the average, and the overall is the average of the other four groups.
- Low confidence is reported next to the tier, never hidden, and so is the mode: a quick tier is labeled quick.
Be strict
- "Someone said so" is a 1 at most, unless you can see it.
- A 2 means measured or verified, and still true without the person who set it up.
- When unsure between two scores, take the lower and say why.
- A part of a check that doesn't exist for this data (no summary tables, no cleanup-only jobs) is skipped, not held against the check.
Calibration
Checks scored on real datasets, to anchor your own calls:
- M6, 0. The table doc was readable and has no target; the planning docs were not. The main place was readable and empty, so 0, with the planning docs listed as an access gap.
- U1, 1. The grain is stated in a column comment, and a uniqueness query run today found no duplicates. Nothing runs that check on each load, so 1.
- L3, 2. 3% of rows land after the day closes (measured today), and the source recomputes closed days on every run. The measuring is yours; the recompute is the system's. 2.
- R2, 1. The IaC shows the backups in a separate account; the live policies were unreadable. Partial access, so 1 at most, with the live policies listed as an access gap.
- R2, 1, not 0. The IaC shows native backup vaults in the production account with no lock, but the backup tool's inventory shows the same databases also backed up by a separate tool, whose vault policies you couldn't read. The native vaults don't settle it: a partial view of the backup that matters, so 1.
- R3, 2. The backup tool's restore history shows record-level restores of this database every few days, each finishing in minutes. That is a timed drill, even though you didn't run one.
Step 4
Find strengths and gaps
Strengths are what to keep. Gaps are ranked by what they do to the answer of the business question, not by how easy they are.
Strengths
- Any group at 75% or more, and any check scored 2 that the question depends on directly. If none qualify, say so and name the closest.
- For each, what keeps it true: a test, an alert, an owner. A strength nobody maintains is next quarter's gap.
Gaps
- Every check at 0 or 1, with its evidence or what was unknown.
- Impact, highest first: the answer can be wrong → data can be lost → the answer is late → it costs too much. Defaults: Understood, L2, L4 and L6 make answers wrong; Reversible and E2 lose data; M1, M2, M3, M5, M6 and L3 make them late; M4, L1, L5, E1, E3, E4 and E5 cost money. Change it when the finding says otherwise.
- Effort: S (a day or less), M (a week or less), L (longer).
- Unlocks: some checks come before others. M2 to M5 before any savings claim; U1 to U3 before U7; M3 before L5; R4 before R2 and R3. They order the recipe without blocking an item; say what each item depends on.
Access gaps
Every check marked no access, grouped by the access that would unblock it: which system, what level (read-only is almost always enough), who can grant it, and which checks it opens. Access gaps don't lower the score, but they cap the confidence, and the ones that block Reversible checks come first.
Step 5
Write the datamaxxing recipe
Turn the gaps into a short plan your human can act on. Every Prep and Cook item names the check it moves, the number it should change, and how you'll prove it did. Every Skip item says why not. When the Reversible gate fails, the first item addresses it, in Prep if it's small and in Cook if not.
Prep · this week
Up to three S-effort fixes with the highest impact. Usually: write the definition, add the tag, set the alert.
Cook · this month
Two or three M-effort changes that move a scorecard number: freshness, jobs, cost, breaks, or a check's score.
Serve · keep it
The tests, alerts and owners that stop today's strengths from sliding back.
Skip · not now
What isn't worth doing for this question, and why. Saying no is part of the recipe.
Report to your human
# Datamaxxer self-test (<quick or full>): <question> Finished <time> by <agent>. Read-only: I made no writes, deletes, reruns or permission changes. ## Business question Question / decision / main table / consumers and layers / today's answer (window, population, unit) / source ## Assumptions and questions for you - <question I would have asked> -> assumed <default> ## Score Overall <n>% · <tier> (<quick or full>) · assessed <n> of <8, or 29 minus n/a> (<coverage>%, <n> on a partial view) | group | score | scored 0 | partial | no access | n/a | |-------------|-------|----------|---------|-----------|-----| | Measured | | | | | | | Lean | | | | | | | Understood | | | | | | | Ready/cheap | | | | | | | Reversible | | | | | | Checks: M1 <s> M2 <s> ... R5 <s> (s = 0, 1, 2, no-access or n/a; quick: only its 8 checks, Lean "not in quick mode") ## Strengths (keep) - <check> [<layer>]: <evidence>; kept true by <test/alert/owner> (none qualify: say so, and name the closest) ## Gaps (by impact) - <check> <score> [<layer>]: <finding>; impact <wrong/lost/late/cost>; effort <S/M/L> ## Recipe Prep: <fix> -> moves <check>, changes <number>, proof <how you'll show it> Cook: ... Serve: ... Skip: <item> -> why not ## Access gaps (no access, or only a partial view) - <system, read-only> from <whom>, unblocks <checks> ## Follow-ups (optional) - after a quick run: go deeper with the full test, same question and definition - <scheduled re-run once Prep and Cook land, same question and definition> - next dataset: <candidate and why>
Save for the next run
{
"version": 1,
"mode": "one of: quick, full",
"finished_at": "<ISO 8601 time the run finished>",
"question": {
"text": "",
"decision": "",
"main_table": "",
"consumers": [],
"layers": [],
"todays_answer": "",
"definition": "",
"read_at": "",
"source": "one of: named, inferred, assumed",
"candidates": [],
"assumptions": []
},
"checks": [
{
"id": "M2",
"status": "one of: scored, no_access, not_applicable",
"score": 1,
"partial_view": false,
"layer": "warehouse copy",
"evidence": "query or file or person",
"finding": "p95 25 h; landing 0.1 h, pipeline 24.9 h",
"impact": "one of: wrong, lost, late, cost",
"effort": "one of: S, M, L"
}
],
"groups": {
"measured": 0, "lean": 0, "understood": 0,
"ready_cheap": 0, "reversible": 0
},
"overall": 0,
"tier": "one of: Datasitter, Datacurious, Datamaxxer in training, Datamaxxer",
"gates_failed": ["any of: reversible, step_1"],
"strengths": [
{ "check": "", "evidence": "", "kept_true_by": "" }
],
"assessed": 0,
"coverage": 0,
"partial": 0,
"access_gaps": [
{ "system": "", "level": "read-only",
"grantor": "", "unblocks": ["M4"] }
],
"recipe": {
"prep": [
{ "item": "", "moves": ["U3"], "changes": "",
"proof": "", "depends_on": [] }
],
"cook": [], "serve": [],
"skip": [ { "item": "", "why_not": "" } ]
},
"follow_ups": [ "" ]
}Group scores, overall and coverage are whole percentages from 0 to 100; a group with no assessed checks is null, a check's score is null unless its status is scored, and impact and effort are null unless the check is a gap. The mode is quick or full; a quick run lists only its 8 checks, and groups.lean is null. The tier is the one left after the gates, or null with no score; gates_failed lists reversible when any Reversible check in the run scores 0 or wasn't assessed, and step_1 when a Step 1 field is missing. On a later run, load the last file, run the same checks in the same mode against the same question and window rule, and report what moved and why.
Questions about the self-test
Not if you run it with read-only credentials. The test tells your agent to stay read-only, but that’s a rule it follows, not a lock. Anything that needs new access, costs money, restores data, or involves another team waits for your approval, and the recipe is a plan for you to approve, not changes already made.
No. Your agent runs the test wherever you run it, and the report stays with you. Nothing is sent to us.
Those checks are marked "no access." They don't count against the score, and the report lists exactly what read access would unblock them.
Quick mode takes about 10 minutes to a tier: 8 checks, plus your answers to one or two questions. Ask your agent to go deeper for all 29 checks.
Because "is our data good?" has no answer, and "can we trust which accounts churn this quarter?" does. A score tied to a question can be checked, improved and re-tested.
Because one bad write you can't undo outweighs everything else on this list, and agents make more writes every month. Our Rogue AI agent tracker counts the public cases. Also, we make backup software, so read those five checks with that in mind: each one names a fix pattern, not a product, and any backup tool can pass them.
Whenever you want a reading. Save the JSON at the end, and the next run reports what moved and why.
Gilad Pagi, a forward deployed engineer at Eon. It's v1, and it'll change as more people run it.
You are my AI agent. Run the Datamaxxer self-test below on our data, in quick mode (8 checks) unless I ask for the full test. Stay read-only, ask me before anything that needs new access or costs money, and come back with the report and the JSON from Step 5. A meaningful run needs read access to our warehouse and repo, plus read-only cloud access (billing, IAM and the backup console where possible).
# The Datamaxxer self-test
## Agent, start here: Ground rules
1. Read-only. Read catalogs, schemas, query and job logs, bills, incident records and access policies. Never write, delete, rerun a production job or change a permission while taking this test. This is a rule you keep, not a lock: if your access could write or delete, tell your human in Step 0 and suggest read-only credentials.
2. Evidence or zero. Every score of 1 or 2 cites its evidence: the query you ran, the file, log or policy you read, or the person who confirmed it. Each check names where to look. If you could read that place and the thing isn't there, it scores 0. In the rubrics, "unknown" means nobody can tell even though you could read where the answer would be.
3. No access is not a zero. The first place in a check's "Look in" line is its main place. Settle the status in this order. (1) Anything you read, anywhere, already settles the status: score it rather than marking it no access. A 0-level fact, such as "computed differently in places", settles the score at 0 even when your view was partial; a positive finding on a partial view is still capped at 1, as in (3). One exception: a fact about one backup doesn't settle a check about backups while another backup of the same data may exist, so read the backup tool's own records first. (2) You couldn't read the main place: no access, left out of the score and listed as an access gap. (3) You could read only part of it, such as the IaC but not the live policy, or only the design and code but not the live system: that is a partial view. Score what you saw, at most 1, and list the rest as an access gap. (4) You could read it and the thing isn't there: 0, with any other places you couldn't read listed as access gaps. A main place that doesn't exist counts the same: once the cloud account or your human shows there is no backup tool, the checks that look there first (E2, E3, R2 to R5) are scored from the rest of their "Look in" line, and 0 where nothing there has it, not no access. An empty result from a metadata or metering view usually means a missing privilege: treat that place as unreadable unless you can show the object exists.
4. Your own work counts, up to a point. What you measure, find or estimate during the test fills any part of a rubric that asks for something to be measured, known, found or estimated. Everything else a rubric asks for (tracked, tested, written, owned, recomputed, retired) must already be true without you. Your work alone can lift a check to 1, never to 2. The score describes the data and the team, not your effort today.
5. Ask first. Ask your human before requesting access, running anything that costs money, writing anywhere (even a scratch copy), restoring anything, or contacting another team. Batch the asks into one message. Read-only queries on a warehouse you were given are fine; ask before anything that provisions capacity or runs for long. When a check needs one of these and you have no answer, use the check's fallback; where it names none, score what you could see, at most 1, and say what was missing.
6. Stay on the question. Every check is about the data behind one business question (Step 1), not the whole estate. That data runs through layers, from the source system through loaders and copies to the main table. Pick the layer per check and name it in the finding: Lean checks usually land on the loader, Understood on the main table and its docs, Reversible on the layer you would restore from.
7. Keep data where it is. Report counts, names and timings. Don't copy rows, credentials or personal data into your report.
8. One reading, at the finish. Your score is how things stand when your run finishes. Suggest scheduled follow-ups where they help, such as a re-run once fixes land; they never change this run's score.
9. Time-box. Keep queries cheap: filter by date, sample with limits, don't scan large tables in full. When a check would take far longer than the rest, score what you found, say what's missing and move on.
### Modes: Quick first, full to go deeper
- Quick mode is the default. 8 checks to a tier in about 10 minutes: M1, M2, U1, U3, E3, E4, R2 and R4, marked Quick in Step 2. Run Steps 0 to 5 on those 8 only. Run the full test only when your human asks for it.
- Full mode goes deeper. All 29 checks. After a quick run, keep the same business question and definition, run the other 21 checks and score all 29 as a new full run; a quick finding carries over only if it is still true when the full run finishes.
- Label the mode. Step 3 says how a quick run is scored. Put the mode next to the tier in the report and in the JSON's mode. A quick tier is a first reading: say so, offer the full test as the next step, and compare a quick run only with another quick run.
## Step 0: Check what you can read
Before Step 1, find out what you can actually reach, so your human hears what's missing once, at the start, and not after the checks.
1. List what you can read. A meaningful run needs read access to the warehouse and the repo, plus read-only cloud access (billing, IAM and the backup console where possible). Mark each place the checks look in as readable, partly readable or not readable: the warehouse with its query and load history, the orchestrator and pipeline code, the bill or metering views, the incident tracker, access policies, the backup tool and the cloud's resource inventory, and the catalog, docs, chat and tickets. Having no backup tool at all is a finding, not an access gap (rule 3).
2. Estimate coverage. From the checks' "Look in" lines, count how many of the checks in your mode (8 quick, or all 29) you could score.
3. Ask once. Send your human one message: what you can read, the estimated coverage (below 60% means a low-confidence result), whether your access could write or delete (rule 1), and the access that would open the most checks, with who could grant it. Set expectations in the same message: about 1 or 2 questions in all (this one if access is missing, and which business question to use in Step 1), and where nobody answers you go on with a marked assumption.
4. Adapt or stop. Go on with what you can read while you wait, and pick up the checks any new access opens. If you can't see the bill, IAM or the backups, score from the repo, the IaC and the warehouse's metadata instead: metering and job history for cost, grants and IaC policies for access, and IaC and backup configs for backups. That is usually a partial view, so 1 at most, and what stays unreadable becomes an access gap (rule 3). If you can read neither the data nor any docs about it, stop here and report what you checked and the access you'd need, with no score.
## Step 1: Anchor on one business question
Datamaxxing is measured against a question the business actually asks. Pick one, write it down, and run every check against the data behind it.
### Path A: Your human names it
Ask: "Which question should our data answer better: one a decision depends on, that broke at least once this year, and that someone senior noticed?" Then sharpen it until it names a table and a number, like "which accounts are about to churn this quarter?" If they name the question but not the decision it drives, infer the decision and mark it assumed.
### Path B: You infer it from usage
- Questions your human asked you more than once.
- The most-run queries and most-read tables of the last 30 days.
- The most-viewed dashboards and the reports with a meeting attached.
- Recurring data questions in tickets or chat, and the datasets with the most incidents.
Leave your own queries and earlier self-test runs out of usage history; if that is all you can see, use the fallbacks. If all the history you can see is under your own identity, treat it all as yours. No usage logs you can read? Use what points at usage instead, in this order: dashboards and BI or semantic-layer definitions, roles and services granted read access, references in code and docs.
Propose up to three candidates with the evidence for each, and ask your human to pick. If nobody answers, take the one with the most evidence of use and mark it assumed, meaning inferred and not confirmed.
- Question: One sentence, answered by a number or a ranking.
- Decision: What someone does differently with the answer.
- Main table: The table the answer is read from.
- Consumers and layers: Who reads it, and the path from source system to the main table.
- Today's answer: The value, and its exact definition: population filter, numerator and denominator for a rate, time column, window, unit, how open records count. Always decide attempts or entities: does a retried job count once or every time? List that and every other choice that moves the number, with its alternative value, and when you read it. A later run must reproduce the definition; for a table that changes by the hour, it needn't reproduce the value.
- Source: Named by the human, or inferred, with the evidence and any assumptions.
## Step 2: The checklist
29 checks in five groups. Each says where to look, the evidence to collect, what scores 0, 1 or 2, and the fix pattern to reach for if it falls short. The 8 marked Quick are quick mode: M1, M2, U1, U3, E3, E4, R2 and R4.
### Field note: Loaders that rebuild the whole table
Many warehouse copies are dropped and reloaded from the source on a schedule. The warehouse's table history lists each reload and the dropped versions it still keeps. Six checks read differently there:
- M2: there is no per-event landing time. Event to queryable is roughly half the reload interval plus the load time at p50, and the full interval plus the load time at p95, on top of any delay before the source has the event.
- L2: rerunning a past day can't reproduce it once the source has changed. Score the dedupe and update rules from the design, 1 at most.
- L3: each reload picks up late rows in the copy. Whether closed days are corrected is then a question about the source; score the source if you can read it.
- L4: a reload carries any schema change straight through unless a check sits between source and copy.
- L5: compare rows written per reload with rows that actually changed. That ratio is the waste.
- R3: a bad write to the copy heals on the next reload, so that isn't a restore path. Score the source, where a bad write would stay.
### Group 1: Measured
You can't datamaxx what you haven't measured: freshness, jobs, cost and breaks, with a target.
#### M1 (quick). Can you list every dataset the answer reads, with an owner for each?
Evidence: lineage from the catalog, or the from and join chain traced back from the main table. An owner is a team or person accountable for fixing the dataset; an owning role, a reviewer group or a project name is not.
Look in: the data catalog or lineage tool, warehouse grants, ownership files in the repo.
Fix pattern: lineage in the catalog, one named owner per dataset.
- 0 · Can't list them, or no dataset has an owner.
- 1 · Listed, and some but not all have owners.
- 2 · Complete, every dataset owned, found without asking anyone.
#### M2 (quick). Do you know how long an event takes to become queryable, and how fresh the question needs it?
Evidence: landing times joined to the finish time of the first build that includes them; p50 and p95, split into landing delay and pipeline delay. For a loader that replaces the whole table, its load history gives cadence and duration.
Look in: ingest or landing timestamps, the warehouse's load or table history, the orchestrator's run log.
Fix pattern: a scheduled freshness query; fix the schedule before the source when the delay is the pipeline's.
- 0 · Unknown.
- 1 · Estimated, or measured once.
- 2 · p50 and p95 measured and split, against a stated need, and tracked.
#### M3. Is there a complete inventory of the jobs between the sources and the main table?
Evidence: the orchestrator's job list walked back from the main table; per job its schedule, full rebuild or incremental, owner, and whether it only cleans up raw data.
Look in: the orchestrator and any ELT tool configs, loader code in the repo.
Fix pattern: a job inventory generated from the orchestrator, kept as code.
- 0 · None.
- 1 · Partial, from memory, or rebuilt by you from metadata.
- 2 · Complete, generated, cleanup-only jobs marked.
#### M4. Can last month's compute and storage cost for those jobs be attributed?
Evidence: the bill grouped by job tag; for the biggest line item, what runs on it (big batch, many small jobs, streaming).
Look in: the cloud bill or cost export, warehouse metering views.
Fix pattern: cost-allocation tags per job; until then, node-hours per run times runs times rate.
- 0 · Can't be attributed.
- 1 · Estimated from the run log.
- 2 · Tagged per job and split by workload type.
#### M5. Are the last 30 days of data breaks on record: what broke, who noticed first, how long the fix took?
Evidence: one incident log, each break typed as wrong, late or missing data, with hours to notice and to fix, and whether a customer found it first. A break is a wrong, late or missing value of the answer; count it if it was live at any point in the last 30 days.
Look in: the incident tracker or on-call tool, postmortems.
Fix pattern: a written definition of a break and one incident template.
- 0 · No record.
- 1 · Scattered across chat and tickets.
- 2 · One log, typed, with notice and fix times.
#### M6. Is there one numeric target for this data, with a guardrail?
Evidence: a written target like "event to queryable p95 under 2 hours", next to the baseline, plus "cost and breaks don't get worse". A display threshold in code isn't a target unless someone committed to it.
Look in: the table doc or catalog entry for the main table; also planning docs and the tracker.
Fix pattern: one target, one guardrail, both next to the baseline numbers.
- 0 · No target.
- 1 · A target, but not a number or no guardrail.
- 2 · Numeric target, guardrail and baseline written down.
### Group 2: Lean
Do the work once: no repeat cleaning, no silent drops, no nightly rebuild of what didn't change.
#### L1. Do you know which fields the question reads, and which the pipelines process anyway?
Evidence: fields in the query logs against fields in the pipeline code. Processed but never read is cost; read but no longer parsed, often after a rename, is a silent bug.
Look in: query history, the code of each consumer, the pipeline's transforms.
Fix pattern: drop unread fields; parse the renamed ones.
- 0 · Unknown.
- 1 · Partly known.
- 2 · Computed from logs and code; no field read but unparsed.
#### L2. Is there one rule for duplicates, updates and deletes, and does rerunning a day give the same output?
Evidence: the dedupe key each team uses on the same source; a past day rebuilt in a scratch location and diffed against what was published. Without approval to write a scratch copy, score from the design: a single key with idempotent upserts is a 1 at most.
Look in: pipeline code (merge keys, upserts), load history, a past day's published output.
Fix pattern: one dedupe key; idempotent writes that merge on it.
- 0 · Unknown, or reruns change the output.
- 1 · Written down but untested, or teams differ.
- 2 · One key; a rerun reproduces the published day.
#### L3. What share of records arrive after the processing window closes, and does anything correct them?
Evidence: event time against landing time against the window; whether any job recomputes a closed day.
Look in: event and landing times in the data, job code that recomputes closed days.
Fix pattern: a watermark on landing time; recompute only the partitions late rows touch.
- 0 · Unknown.
- 1 · Measured, never corrected.
- 2 · Measured, and closed days are recomputed.
#### L4. Would a source renaming a field, or adding a new value, be caught before wrong numbers go live?
Evidence: the last schema change on a source, what broke, who found it and how long wrong numbers were live. A new status or type value can silently change a rate.
Look in: source schema migrations or history, ingest contracts, the incident tracker.
Fix pattern: a schema contract checked at ingest.
- 0 · No; a reader would find it.
- 1 · Monitoring catches it after it lands.
- 2 · A contract or check stops it at ingest.
#### L5. Has a nightly full rebuild been replaced by an incremental update that is proven to match?
Evidence: rows scanned per run, runs per month, and an equality check between the incremental and full versions. Distinct counts, like weekly actives, are the usual trap.
Look in: job definitions, table history (rows written per load).
Fix pattern: recompute only what new rows touch; keep the equality check as a test.
- 0 · All full rebuilds, none assessed.
- 1 · A candidate found, savings estimated.
- 2 · At least one incremental, equality check in place.
#### L6. When a record fails processing, is it kept with a reason, does someone hear about it, and can it be replayed?
Evidence: one failed record traced end to end; record counts in and out of each step. A gap nobody can explain means silent drops. A consumer that swallows a failed query with only a log line is a silent drop too.
Look in: pipeline code and error tables, alert rules, dead-letter queues.
Fix pattern: a dead-letter queue with a reason, an alert, and a replay path that needs no ticket.
- 0 · Dropped silently, or unknown.
- 1 · Kept, but no alert or no replay.
- 2 · Kept, alerted, replayable; counts reconcile.
### Group 3: Understood
A colleague, or you, can answer from the main table without asking the person who built it.
#### U1 (quick). Is it written down what one row of the main table is, and is that true?
Evidence: the one-sentence grain ("one account per day") and a query that finds zero duplicate keys. Group by the full key and count; some engines' COUNT(DISTINCT a, b) skips rows with a null in any column.
Look in: the table doc or catalog, and the table itself.
Fix pattern: a one-page table doc; a uniqueness test on the key.
- 0 · Not written.
- 1 · Written, or checked once.
- 2 · Written, and a uniqueness test runs on every load.
#### U2. Are the five columns people ask about most defined, with meaning, unit and business term?
Evidence: the five chosen from real questions (query logs, chat, tickets), and their definitions where readers find them.
Look in: catalog column comments and docs; query history, chat and tickets for what gets asked.
Fix pattern: column definitions in the catalog, next to the table.
- 0 · None defined.
- 1 · Some, or picked from memory or consumer code rather than real questions.
- 2 · Five, picked from real questions, in the catalog.
#### U3 (quick). Is each key metric's formula written, with additive and point-in-time metrics marked?
Evidence: the formulas, and every place the metric is computed. Summing a daily balance like ARR over a month overstates it about 30 times. For a ratio, the numerator and denominator must count the same population.
Look in: the metric layer, model code, BI definitions, docs.
Fix pattern: one metric definition, used everywhere it is reported.
- 0 · Not written, or computed differently in places.
- 1 · Written for some.
- 2 · All written, marked, one definition in use.
#### U4. Are the table's freshness, null rates and distributions profiled, with an alert when they shift?
Evidence: the alert rule and a backtest on last month: would it have caught the last incident early, and how often would it have fired for nothing? An alert whose history you can't read is a 1 at most.
Look in: data-quality tests and monitors on the main table; also alert rules.
Fix pattern: alert on the daily inputs against a trailing median, not on rolling averages that hide a shift for days.
- 0 · No profile, no alert.
- 1 · A profile, or an alert on rolling metrics.
- 2 · Alert on daily inputs, backtested.
#### U5. Are the segments that behave differently written down?
Evidence: regions with their own rules or upload schedule, accounts with outsized volume, and what each means for the target.
Look in: the table doc, and the data itself.
Fix pattern: a segments section in the table doc, with per-segment targets where needed.
- 0 · Not known.
- 1 · Known informally.
- 2 · In the doc, with their effect on targets.
#### U6. Do schema changes, pipeline changes and shifts in the data reach the people and agents that read the table?
Evidence: where definitions live, who or what updates them, and how a breaking change is announced or blocked.
Look in: the catalog, docs, the channel where data changes are announced. Fix pattern: definitions kept in the catalog next to the table, so they reach agents the way they reach people; a named owner. - 0 · Nothing reaches readers. - 1 · Ad hoc announcements. - 2 · In the catalog, with a named owner or automation. #### U7. The outsider test: can you answer questions about the data from the docs, catalog and semantic layer, without looking at the data? Evidence: five questions about the data behind the business question: what the main table and its key columns mean, how the key metric is defined, where a given fact lives, the grain, and how fresh it is. Answer each from the docs, catalog and semantic layer alone and write down the answer with the doc it came from, before you query anything. Then check each answer against the data with a read-only query and count how many held up. An answer that needed code or a guess didn't hold up. Look in: the docs, catalog and semantic layer for the answers; the data only to check them. Fix pattern: every answer that didn't hold up, or that you had to guess, goes into the table doc or the catalog. - 0 · No docs to answer from, or 2 or fewer of the 5 held up. - 1 · 3 or 4 held up. - 2 · All 5 held up against the data. Note: You are the outsider here: answer first, look second. Answer from what the metadata says, not from what you saw in the data or code earlier in this run. Cite the doc behind each answer and the query that checked it. If you can't read the data, score 1 at most when the docs answered all five, mark the answers unverified and list the data as an access gap. ### Group 4: Ready and cheap What it costs to keep the data useful: volume, retention, copies, formats and peaks. #### E1. Do you know which events make up most of the volume, and is any sampling rule proven safe? Evidence: the volume mix; if sampling, the answer per account or segment before and after, not just the total. Look in: the data itself, ingest config. Fix pattern: sample with weights; exempt small segments and critical events. - 0 · Volume mix unknown. - 1 · Known; no rule, or a rule never validated. - 2 · A validated rule, or a written decision not to sample. Note: Billing, financial and audit data are usually never sampled. That is a 2 once the decision is written down, and a 1 until then. #### E2. Is there a retention period for raw events, summary tables and old versions, each with an approver? Evidence: days kept for each of the three, who approves deleting them (a named person or team who must say yes first), and whether deletion requests reach summary tables and snapshots too. Look in: the backup tool's policies and retention settings; also storage and warehouse settings and the deletion-request procedure. Fix pattern: a retention table with approvers; deletion requests that cover every copy. - 0 · None of the three set on purpose: kept forever, or on tool defaults. - 1 · At least one set on purpose, but an approver or deletion-request coverage is missing. - 2 · All three, approvers named, deletion requests covered. #### E3 (quick). How many copies of this data exist, and does any pipeline copy what a backup already holds? Evidence: team copies, dev clones, exports and backups. Exports and spreadsheets can only be estimated, and an estimate is fine. A copy that serves another engine or team still counts; it is redundant if one shared copy could serve both. Look in: the backup tool's inventory; also warehouse object lists, storage buckets and export jobs. Fix pattern: retire unread copies; let one shared copy serve both backup and analytics. - 0 · Unknown. - 1 · Counted or estimated. - 2 · Counted, and no redundant copy left: each is retired or serves more than one job. #### E4 (quick). Can every query engine you use read these tables through one catalog, in an open format, from storage in your own account? Evidence: the same table read by two engines with matching row counts; file counts and sizes per partition. Score the main table, and mention any open copy elsewhere in the finding. Look in: table format and catalog config, each engine's connection settings. Fix pattern: an open table format in a shared catalog; scheduled compaction. - 0 · Locked to one engine, or unknown. - 1 · Open format, but catalog gaps or small files. - 2 · Two engines agree; files compacted. #### E5. What happened at the last traffic peak, and what does the capacity held for it cost? Evidence: peak lag, hot keys, behaviour when a worker failed; (capacity provisioned minus capacity needed at the median hour) times hours times rate. Look in: lag and throughput metrics, capacity config, the bill. Fix pattern: capacity that scales with load; spread hot keys. Hot keys survive a move to incremental. - 0 · Unknown. - 1 · Peak known, its cost not. - 2 · Both known, and the cost is a deliberate choice. Note: When you can see that nothing is sized up for a peak (shared or serverless capacity, or a scheduled full reload with no traffic peak), mark this not applicable with the reason. If you can't see the capacity and the load pattern doesn't rule out a peak, it's no access. E5 is the only check that can be not applicable. ### Group 5: Reversible Whether the data is protected at all, how fast a bad write is noticed and undone, and who can make one. You are one of the agents this group is about. #### R1. Is every agent and automation that can write to or delete this data listed, with who approves its risky actions? Evidence: roles and service accounts with write or delete rights on the main table and its sources, read from access policies. Include yourself. Count individual people; don't name them. An approver is a named person or team who must say yes before the action runs; whoever reviews the code that does it is not. Look in: access policies: cloud IAM, warehouse grants, service accounts, agent configs. Fix pattern: least privilege; an approval step for deletes and bulk writes. - 0 · Unknown. - 1 · Partial list. - 2 · Complete, each with an approver. #### R2 (quick). Do the backups have their own credentials, so that nothing with production access can delete them? Evidence: the backup's access policies read directly, for the layer you would restore the main table from. Warehouse time travel is not a backup. A lock that blocks deletion beats delete rights: if the policies allow deletion but you can't read whether a lock is on, that is a partial view, so 1 at most. This is often an access review with a security team; a partial answer still counts. Look in: the backup tool's own records of where its copies live, who can delete them and whether they're locked; also the access policies and the IaC behind them. Fix pattern: backups in a separate account or vault, immutable for their retention period. - 0 · Unknown, or production can delete them. - 1 · Separate, not verified. - 2 · Verified from the policies. #### R3. How long does it take to put back 50 bad rows, without restoring the whole database? Evidence: a timed restore of 50 rows from snapshots, time travel or backups into a scratch location, or the documented procedure and the time of the last real restore. Restore history that includes row- or record-level restores with their durations counts as a timed drill. Look in: the backup tool's restore history, with job types and durations; also restore runbooks and snapshot tooling. Fix pattern: row-level restore from snapshots, drilled before the incident. - 0 · Full restore only, or unknown. - 1 · Possible, but untimed, taking hours, older than 90 days, or mostly failing. - 2 · A timed drill or restore history from the last 90 days, mostly succeeding, in minutes or less. Note: If a loader recreates the table on every run, warehouse time travel holds only the versions within its retention; list them from table history, and look one layer down. Never restore into production for this test. Run the drill into a scratch location only with your human's approval; otherwise score it from the procedure and the last real restore. "We can only do a full restore, and it took N hours" is a valid answer. #### R4 (quick). Is every source behind the question backed up, and how old is the last good backup? Evidence: the sources' resources listed from the cloud's own inventory, not from tags, matched against what the backup policies cover; the time of the last successful backup of each, against how much data the question can afford to lose. Look in: the backup tool's inventory and policies; also the cloud's resource inventory. Fix pattern: policies that match on what exists rather than on tags, and an alert when a source is left uncovered. - 0 · Some sources have no backup, or nobody can tell. - 1 · All covered, but through tags or a hand-kept list, or the last good backup is older than the question allows. - 2 · All covered by policies that pick up new resources, each with a recent enough good backup. #### R5. Would a mass delete, overwrite or encryption of this data be noticed before the clean copies age out? Evidence: what watches for sudden drops in rows or objects, mass changes or encrypted content, on the source or in the backups; when it last fired; how long clean copies are kept, against how long such damage could go unnoticed. Look in: the backup tool's anomaly or threat alerts; also data-quality monitors and alert rules. Fix pattern: anomaly checks on every backup, with retention longer than the time to notice. - 0 · Nothing watches for it, or clean copies would age out first. - 1 · Counts or the source are watched, but not the backups, or the alert has never fired or been tested. - 2 · Every backup is checked for mass deletion and encryption, alerts reach someone, and retention outlasts the time to notice. ## Step 3: Score yourself Score how things stand when your run finishes. Each check scores 0, 1 or 2, or is marked no access and left out. Each group counts the same, however many checks it has, so the five reversibility checks can't be outvoted by the rest. A quick run applies the same rules to its 8 checks. ``` assessed = checks scored 0, 1 or 2 (not "no access" or "not applicable") group score = points in the group / (2 × assessed checks in the group) → whole %, 0 to 100 overall score = average of the groups with at least one assessed check → whole %, 0 to 100 coverage = assessed checks / (checks in the run − not applicable) → below 60%: low confidence
mode = quick: the 8 quick checks, coverage out of 8; full: all 29
rounding = average the unrounded group scores, then round once
partial = checks scored on a partial view; report the count next to coverage
```
- 0–34: Datasitter. The data sits around doing nothing, and nobody can say what it costs or how it breaks.
- 35–59: Datacurious. Some numbers exist. Most answers still live in one person's head.
- 60–79: Datamaxxer in training. Measured and mostly understood. A few fixes away from running on its own.
- 80–100: Datamaxxer. Measured, lean, documented for agents, cheap to keep and quick to undo.
### Gates
- No "Datamaxxer" while any Reversible check in the run scores 0 or wasn't assessed: R2 or R4 in quick mode. One bad write you can't undo outweighs everything else.
- No score at all without the Step 1 fields filled in.
- A group with no assessed checks shows "not assessed" and stays out of the average. A group with fewer than half its checks assessed is marked low confidence. In quick mode, Lean has none of the 8 checks: it shows "not in quick mode", stays out of the average, and the overall is the average of the other four groups.
- Low confidence is reported next to the tier, never hidden, and so is the mode: a quick tier is labeled quick.
### Be strict
- "Someone said so" is a 1 at most, unless you can see it.
- A 2 means measured or verified, and still true without the person who set it up.
- When unsure between two scores, take the lower and say why.
- A part of a check that doesn't exist for this data (no summary tables, no cleanup-only jobs) is skipped, not held against the check.
### Calibration
Checks scored on real datasets, to anchor your own calls:
- M6, 0. The table doc was readable and has no target; the planning docs were not. The main place was readable and empty, so 0, with the planning docs listed as an access gap.
- U1, 1. The grain is stated in a column comment, and a uniqueness query run today found no duplicates. Nothing runs that check on each load, so 1.
- L3, 2. 3% of rows land after the day closes (measured today), and the source recomputes closed days on every run. The measuring is yours; the recompute is the system's. 2.
- R2, 1. The IaC shows the backups in a separate account; the live policies were unreadable. Partial access, so 1 at most, with the live policies listed as an access gap.
- R2, 1, not 0. The IaC shows native backup vaults in the production account with no lock, but the backup tool's inventory shows the same databases also backed up by a separate tool, whose vault policies you couldn't read. The native vaults don't settle it: a partial view of the backup that matters, so 1.
- R3, 2. The backup tool's restore history shows record-level restores of this database every few days, each finishing in minutes. That is a timed drill, even though you didn't run one.
## Step 4: Find strengths and gaps
Strengths are what to keep. Gaps are ranked by what they do to the answer of the business question, not by how easy they are.
### Strengths
- Any group at 75% or more, and any check scored 2 that the question depends on directly. If none qualify, say so and name the closest.
- For each, what keeps it true: a test, an alert, an owner. A strength nobody maintains is next quarter's gap.
### Gaps
- Every check at 0 or 1, with its evidence or what was unknown.
- Impact, highest first: the answer can be wrong → data can be lost → the answer is late → it costs too much. Defaults: Understood, L2, L4 and L6 make answers wrong; Reversible and E2 lose data; M1, M2, M3, M5, M6 and L3 make them late; M4, L1, L5, E1, E3, E4 and E5 cost money. Change it when the finding says otherwise.
- Effort: S (a day or less), M (a week or less), L (longer).
- Unlocks: some checks come before others. M2 to M5 before any savings claim; U1 to U3 before U7; M3 before L5; R4 before R2 and R3. They order the recipe without blocking an item; say what each item depends on.
### Access gaps
Every check marked no access, grouped by the access that would unblock it: which system, what level (read-only is almost always enough), who can grant it, and which checks it opens. Access gaps don't lower the score, but they cap the confidence, and the ones that block Reversible checks come first.
## Step 5: Write the datamaxxing recipe
Turn the gaps into a short plan your human can act on. Every Prep and Cook item names the check it moves, the number it should change, and how you'll prove it did. Every Skip item says why not. When the Reversible gate fails, the first item addresses it, in Prep if it's small and in Cook if not.
- Prep (this week): Up to three S-effort fixes with the highest impact. Usually: write the definition, add the tag, set the alert.
- Cook (this month): Two or three M-effort changes that move a scorecard number: freshness, jobs, cost, breaks, or a check's score.
- Serve (keep it): The tests, alerts and owners that stop today's strengths from sliding back.
- Skip (not now): What isn't worth doing for this question, and why. Saying no is part of the recipe.
**Report to your human**
```
# Datamaxxer self-test (<quick or full>): <question>
Finished <time> by <agent>. Read-only: I made no writes, deletes, reruns or permission changes.
## Business question
Question / decision / main table / consumers and layers /
today's answer (window, population, unit) / source
## Assumptions and questions for you
- <question I would have asked> -> assumed <default>
## Score
Overall <n>% · <tier> (<quick or full>) · assessed <n> of <8, or 29 minus n/a> (<coverage>%, <n> on a partial view)
| group | score | scored 0 | partial | no access | n/a |
|-------------|-------|----------|---------|-----------|-----|
| Measured | | | | | |
| Lean | | | | | |
| Understood | | | | | |
| Ready/cheap | | | | | |
| Reversible | | | | | |
Checks: M1 <s> M2 <s> ... R5 <s>
(s = 0, 1, 2, no-access or n/a; quick: only its 8 checks, Lean "not in quick mode")
## Strengths (keep)
- <check> [<layer>]: <evidence>; kept true by <test/alert/owner>
(none qualify: say so, and name the closest)
## Gaps (by impact)
- <check> <score> [<layer>]: <finding>; impact <wrong/lost/late/cost>; effort <S/M/L>
## Recipe
Prep: <fix> -> moves <check>, changes <number>, proof <how you'll show it>
Cook: ...
Serve: ...
Skip: <item> -> why not
## Access gaps (no access, or only a partial view)
- <system, read-only> from <whom>, unblocks <checks>
## Follow-ups (optional)
- after a quick run: go deeper with the full test, same question and definition
- <scheduled re-run once Prep and Cook land, same question and definition>
- next dataset: <candidate and why>
```
**Save for the next run**
```
{
"version": 1,
"mode": "one of: quick, full",
"finished_at": "<ISO 8601 time the run finished>",
"question": {
"text": "",
"decision": "",
"main_table": "",
"consumers": [],
"layers": [],
"todays_answer": "",
"definition": "",
"read_at": "",
"source": "one of: named, inferred, assumed",
"candidates": [],
"assumptions": []
},
"checks": [
{
"id": "M2",
"status": "one of: scored, no_access, not_applicable",
"score": 1,
"partial_view": false,
"layer": "warehouse copy",
"evidence": "query or file or person",
"finding": "p95 25 h; landing 0.1 h, pipeline 24.9 h",
"impact": "one of: wrong, lost, late, cost",
"effort": "one of: S, M, L"
}
],
"groups": {
"measured": 0, "lean": 0, "understood": 0,
"ready_cheap": 0, "reversible": 0
},
"overall": 0,
"tier": "one of: Datasitter, Datacurious, Datamaxxer in training, Datamaxxer",
"gates_failed": ["any of: reversible, step_1"],
"strengths": [
{ "check": "", "evidence": "", "kept_true_by": "" }
],
"assessed": 0,
"coverage": 0,
"partial": 0,
"access_gaps": [
{ "system": "", "level": "read-only",
"grantor": "", "unblocks": ["M4"] }
],
"recipe": {
"prep": [
{ "item": "", "moves": ["U3"], "changes": "",
"proof": "", "depends_on": [] }
],
"cook": [], "serve": [],
"skip": [ { "item": "", "why_not": "" } ]
},
"follow_ups": [ "" ]
}
```
Group scores, overall and coverage are whole percentages from 0 to 100; a group with no assessed checks is null, a check's score is null unless its status is scored, and impact and effort are null unless the check is a gap. The mode is quick or full; a quick run lists only its 8 checks, and groups.lean is null. The tier is the one left after the gates, or null with no score; gates_failed lists reversible when any Reversible check in the run scores 0 or wasn't assessed, and step_1 when a Step 1 field is missing. On a later run, load the last file, run the same checks in the same mode against the same question and window rule, and report what moved and why.
