Train on Synthetic, Test on Real: Our Commitment to Unimpeachable Quality

How can you be sure that synthetic data is actually any good?

It’s a fair question, and one that should be asked of any synthetic data provider. In a field as precise as law, "it looks okay" is not a valid quality check. A visual inspection can't tell you if a dataset has truly captured the complex statistical patterns, nuances, and structures of real-world legal documents. For a legal tech company building its flagship product, hope is not a strategy. You need empirical proof.

That’s why our promise of "unimpeachable quality" isn't just a marketing slogan; it's a commitment backed by a rigorous, scientific validation framework known as Train on Synthetic, Test on Real (TSTR).

What is TSTR? The Gold Standard of Validation

The TSTR methodology is the gold standard for verifying the utility of synthetic data. It’s a simple but powerful experiment designed to answer one crucial question:

Can a machine learning model, trained exclusively on our synthetic data, perform a real-world task on a set of real documents it has never seen before?

If the answer is yes, it provides definitive proof that the synthetic data has successfully replicated the essential, high-value patterns of the real data. It proves the data works.

Our TSTR Process: A Commitment to Proof

Our implementation of TSTR is a core part of our data generation pipeline. Here’s how we do it:

1. The Final Exam: Before any training begins, we take a portion of our curated, real-world legal documents and lock them away. This is our Real Test Set—the final exam that the AI models will have to pass.
2. The Synthetic-Trained Model: We train our first AI model (Model A) only on our high-fidelity synthetic data. This model never sees a single real document during its training.
3. The Real-Data Benchmark: We train a second, identical AI model (Model B) on real-world data. This model acts as our benchmark, representing the best-case performance scenario.
4. The Test: We take both models and evaluate their performance on the same task using the Real Test Set. The task is a practical, real-world challenge, such as accurately classifying different types of legal documents (e.g., identifying a document as a 'Motion', 'Contract', or 'Pleading').

Defining Success: The 92% Rule

The results are then compared. For our synthetic data to pass the quality bar, the synthetic-trained model (Model A) must achieve a performance score that is at least 92% of the real-data benchmark model (Model B).

This isn't an easy target to hit. It means our synthetic data must be so statistically representative of the real data that the AI model can barely tell the difference.

Why This Matters for You

This rigorous validation process is our promise to you. When you build your AI application on Axiom’s data, you are not just getting data that looks right—you are getting data that is proven to work in a real-world context. The TSTR framework de-risks your development, accelerates your timeline, and gives you the confidence to build products that are not just innovative, but also reliable and effective.

We handle the burden of proof so you can focus on building the future of legal tech.

Ready to build on a foundation of proven quality? Subscribe for launch updates and be the first to know when our datasets are available.