The text-to-SQL model that wrote perfect queries against nonsense

A team fine-tuning a text-to-SQL model built their training triples the obvious way: a question, the SQL that answers it, and the result of running that SQL against a generated database. The queries the model learned to write were clean. They parsed, they executed, they looked expert. And the results were garbage, because the database they trained against was incoherent. Foreign keys dangled, so joins dropped rows. Values were random, so a SUM over "revenue" meant nothing. The model had faithfully learned to produce SQL that works against data no real system would ever hold.

The database, it turned out, was the hard part all along. And it is the part CrossRow supplies.

The two things a training database needs

First, breadth. A model trained against one toy schema overfits to it and stumbles on the next. CrossRow ships 125 industry schemas, banking to healthcare to logistics, each a genuine multi-table database. Generating across many of them gives a general model the structural range it needs.

Second, and this is the one the team missed, coherence. Breadth is worthless if the joins return nonsense. Because CrossRow generates each database with its relationships intact, a query runs against data that behaves like a real system, and the answer is real:

# question
"Average order value for customers in Germany"

# sql the model learns to write
SELECT AVG(o.total) FROM orders o
JOIN customers c ON c.id = o.customer_id
WHERE c.country = 'Germany'

# the answer is trustworthy because
✓ every o.customer_id resolves to a real customer
✓ c.country agrees with c.city
✓ o.total equals the sum of its line items

The rebuild

They regenerated their databases with CrossRow and rebuilt the triples against data that actually held together. You can explore any generated database in the browser and run SQL against it directly, or pull the files from S3 into your own pipeline for triple generation at scale.

The lesson they took away

They had been chasing row count, assuming more data meant a better model. For text-to-SQL, correctness beats volume every time. A million rows of incoherent data teaches a model to write queries whose results are noise. A smaller set of coherent databases teaches it the actual mapping from language to correct SQL. And because every CrossRow run ends with a quality report, they could confirm the data held together before building a single triple from it.