Reach for a fixture library and you get well-formatted values fast: real-looking names, valid emails, dates in range, prices with two decimal places. Each field is convincing. The trouble is that the library fills each field without looking at any other field, and integration tests exist precisely to check the relationships between fields and rows.
So the fixtures pass the eye test and fail the actual test. Worse, they fail intermittently, because a new random seed produces a different impossible row each run. Here is where it goes wrong, in the order you usually hit it.
1. Foreign keys that point nowhere
You generate 500 customers and 5,000 orders. The order generator sets customer_id to a random integer in a plausible range. Roughly none of them line up with a customer you actually created. On insert you get a wall of foreign-key violations; if your schema has no FK constraints, it is worse: the violations are silent, and every join in your test quietly drops rows.
The test that was supposed to check your checkout flow now fails on data setup, and the stack trace points at your insert code.
The self-referential version is nastier
An employees.manager_id that references the same table cannot be filled from a list that does not exist yet: the managers are being generated in the same pass. Naive generators either leave it null everywhere or point it at IDs that never materialize, and any test that walks the org tree hangs or throws.
2. Assertions on derived values
Your invoice test asserts invoice.total == sum(line_items.amount). Faker filled total with one random number and each line item with another. They will never match. You can special-case the fixture to compute the total by hand, and now you are hand-maintaining every derived relationship in your schema, which is the job you wanted the generator to do.
The same trap catches tax = subtotal × rate, end_date = start_date + term, and any column whose value is a function of another. Independent generation gets all of them wrong by construction.
3. Timelines that run backwards
Sort an entity’s events by timestamp and the story falls apart: a shipment delivered before it was picked_up, a subscription cancelled before it was created, a payment posted after the account was closed. Any test that replays a sequence (a state-machine test, an event-sourcing test, a “does the timeline make sense” assertion) trips over the first impossible transition. Because the statuses were chosen at random, about half of any multi-event history is illegal.
shipment_id status event_ts SHP-4471 delivered 08:14 SHP-4471 picked_up 11:02 ← after delivery SHP-4471 in_transit 09:37 this sequence cannot physically happen
4. Aggregations built on inconsistent rows
Group revenue by country and the numbers are meaningless if city, country, and currency were sampled independently: you have customers in Berlin, Brazil paying in yen. A dashboard test that checks “EU revenue > 0” might pass by luck and fail next seed. Nothing in the code changed; the fixtures did.
5. The flakiness tax
Every one of the above is intermittent. Because the bad rows appear at random, the test is green four runs out of five and red on the fifth. The team learns to re-run CI until it passes, which is the exact habit that lets real regressions through. Flaky fixtures do not just waste time; they erode trust in the whole suite.
Why the usual escapes do not hold
- Hand-written fixtures. Correct, but they do not scale past a few tables, and they rot the moment the schema changes.
- A copy of production. Consistent by definition, and a compliance problem the moment it leaves prod. PII in a test database is a breach waiting for an audit.
- More generator rules. You can bolt relationship rules onto a per-column generator one at a time, but it is whack-a-mole: every new invariant is new bespoke code, and you find the missing ones when a test fails in CI.
The actual fix: generate with row context, then verify
The failures above share one root cause: values chosen without reference to the rest of the row or the rest of the table. Fix that and they all disappear at once. This is what CrossRow is built to do:
- Foreign keys resolve, including self-referential ones like
manager_id, with realistic depth and null rates. - Derived columns compute exactly (totals, taxes, offset dates, string-built usernames) instead of landing in a plausible range.
- Statuses walk a legal state machine in timestamp order, so sorted timelines always make sense.
- Geography stays consistent per row, so grouped aggregates mean something.
- No production data is involved, so there is nothing to leak and nothing to redact.
And because the same run is independently verified and scored before you use it, you are not trusting that the invariants held; you are handed the check. Fixtures that pass the eye test and the real test, without a copy of production and without hand-maintaining every relationship yourself.