Generating plausible data is only half the job. The other half is proving it, because “looks fine” is exactly the failure mode that ships broken fixtures and biased training sets. After every run, CrossRow re-reads what it produced with a separate validator and reports two numbers plus a detailed breakdown. The scores are computed from the actual rows, not the plan that was supposed to produce them.
STRUCTURE 98.6/100 structural correctness REALISM 9.4/10 does each row make sense ✓ foreign keys resolve 100% ✓ formula accuracy 100% (balance = prev + amount) ✓ uniqueness txn_id 100% ✓ cross-row: running_ledger reconciles on all groups ✓ distribution: amount lognormal, within bounds
Structure Score: is the data well-formed?
The Structure Score, out of 100, is the objective half. No language model is involved; these are deterministic checks you could run yourself if you had the time to write them:
- Referential integrity: every foreign key resolves to a row that exists, self-referential keys included.
- Formula accuracy: derived columns are recomputed and compared. If the plan says
tax = gross × 0.22, the validator checks that it holds on every row and reports the percentage that do. - Uniqueness and null rates: primary keys are unique; columns hold null at the rate the plan intended, not by accident.
- Type conformance: values match their declared column types, so the data loads without coercion surprises.
- Distribution checks: numeric columns are tested against the shape they were asked for (lognormal, Benford, normal) and their bounds.
A perfect Structure Score means the data will load, join, and compute exactly as a real database would. A dented one points at the specific column or constraint that slipped, so you know precisely what to look at.
Realism Score: does each row make sense?
Structural correctness does not catch everything. A row can satisfy every constraint and still be nonsense: a luxury purchase by a customer with a poverty-line income, a hospital visit with a discharge before admission. The Realism Score, out of 10, is the judgment half, an independent language-model pass that reads sampled rows and scores three things:
- Internal consistency: do the values within a row agree with each other?
- Domain realism: would this row be unremarkable in a real system of this kind?
- Value quality: are the individual values well-chosen, or lazy filler?
Structure asks “is it well-formed?” Realism asks “would a domain expert nod at it?” You need both, because each is blind to the other’s failures.
A second opinion, on purpose
The realism pass runs the sample past two models of different strength and compares them. When a weaker and a stronger judge disagree sharply, that gap is itself a signal: it flags data that is subtly hard to read as either genuinely nuanced or genuinely off, and surfaces it for a closer look rather than burying it in an average.
The cross-row verification section
The invariants that live between rows get their own dedicated pass, because a per-row check cannot see them. Every cross-row rule the plan declared (a reconciling ledger, a sum-to-target allocation, a legal state machine, a three-way match, a bracket, a bin-packing constraint) is re-verified against the generated output and reported with the share of groups that comply. A rule that was declared but not actually satisfied shows up here as a failure, not a green checkmark, so a plan can never quietly claim an invariant it did not deliver.
Why self-grading is the point. When a run carries a Structure Score, a Realism Score, and a per-rule cross-row breakdown, you are not taking a vendor’s word that the data is good. You are handing a reviewer (or an auditor) a report they can read and challenge. For regulated teams, “here is the verification that ran on the exact data you received” is a very different conversation than “trust us.”
How to read a low score
A low number is not a dead end; it is a pointer. Because the report attributes scores to specific tables, columns, and rules, a dip tells you where to intervene: usually by sharpening the schema description or the generation hints so the intent is unambiguous, then regenerating. The report is a feedback loop, not a verdict: it is how a run that scored 91 becomes one that scores 98.
That loop is the whole idea. Data you can ship on is not data that looks right: it is data that has been checked, scored, and attributed, every time, before it reaches you. If you have not seen it on your own schema yet, that is the fastest way to understand what the numbers are worth.