Quality and performance report¶
Quality and performance report is the downloadable audit of a finished project. Take it from the project detail, next to Train dataset, or from the download menu in the project list. It is only available once the project has finished.
The file is produced in the language the interface is in — English,
Spanish or Russian — and arrives as
<project name> - Summary.xlsx.
Looking for the short on-screen verdict instead? That is the Quality certificate.
It no longer reports training accuracy
The report used to show each model's score on its own training rows. That number rewarded memorisation: a Random Forest that memorises the rows it was trained on scores close to 100%, while a nearest-neighbour model that deliberately leaves the closest row out scores lower — even when it produces closer distributions, closer crosstabs and no near-copies. None of it looked at a single synthetic row.
The report now measures the delivered rows. The old number is still there, honestly labelled Fit on training data (%), next to a majority baseline so you can see what it is worth.
What the metrics mean¶
Everything below compares your original rows against the generated rows, question by question.
| Indicator | Reads as |
|---|---|
| Fidelity including missing (%) | How closely the generated answers reproduce the real distribution, counting blanks as an answer. 100 = identical. |
| Answered-only fidelity (%) | The same, ignoring blanks — how close the answers are, independently of who skipped the question. |
| Original missing (%) / Generated missing (%) | Share of blanks in each side. |
| Missing-rate gap (pp) | The difference between them, in percentage points. A large gap means the synthetic rows skip questions at a different rate than real respondents. |
| Category coverage (%) | Share of the real answer options that appear at least once in the generated rows. A rare option that never appears shows up here. |
| Answered crosstab fidelity (%) | The same fidelity measured inside each profile group (gender, age, country first, then other low-cardinality columns). This is the number that matters when the deliverable will be crossed by segment. |
| Majority baseline (%) | What you would score by always answering the most frequent option. Context for Fit on training data, not a quality target. |
| Cells an inferred logical repair would change (%) | Coherence: the share of generated cells a logical repair would still want to touch. Lower is better. |
| Near copies in similarity sample (%) | Share of generated rows that are 80% or more identical to their closest real respondent, estimated on a sample of up to 500 rows. This is the "is it just copying my respondents?" number. |
| Generated rows in duplicate groups / Excess exact duplicate rows | Repetition among the generated rows themselves. |
| Exact copies of originals | Generated rows identical to a real respondent — counted over all rows, not a sample. |
Higher is better for every fidelity and coverage figure; lower is better for missing-rate gap, coherence, near copies and duplicates.
Fidelity is not predictive accuracy
The report says so itself, in the Interpretation row: distribution fidelity is not accuracy on held-out respondents. It tells you the delivered dataset behaves like the real one in aggregate — not that any individual synthetic answer was "right".
The four sheets¶
Summary — one indicator per row, for the project as a whole: original and generated row counts, the mean fidelity figures, the novelty and duplicate counts, the identifier checks, coherence, and the total response-field time. It opens with three scope notes worth reading once:
- Data scope — the figures describe the stored data rendered to questionnaire columns; optional final post-processing is not included. A deduplicated download can be cleaner than the report suggests.
- Interpretation — the fidelity-is-not-accuracy caveat above.
- Rule scope — structural hypotheses are advisory; explicit rules take precedence.
The identifier block reports whether your configured ID column is safe: collisions between original and generated rows, excess duplicate identifiers on each side, and generated rows with no identifier at all. Preservation of the uploaded file is deliberately marked Not checked by this report.
Fields — the same indicators, per question: fidelity with and without blanks, missing rates and their gap, category coverage, crosstab fidelity, majority baseline, plus any observed numeric rule for that field, how many rows violate it, and its status.
Models — which model answered each question, how long it took, and Fit on training data (%).
Structural rules — each numeric relationship the platform inferred from your file (for example, parts that must add up to a total), with a status of Consistent, Violations, Partially supported or Unsupported inputs, and the counts behind it on both the original and the generated side. When nothing could be inferred the sheet says so explicitly — "No structural hypotheses available; this is not a validation pass" — which is not the same as passing.
Reading it in practice¶
- Sanity check a delivery: mean answered-only fidelity and crosstab fidelity high, coverage at or near 100, missing-rate gap small.
- Answer "are these real people duplicated?": near copies in the similarity sample, exact copies of originals, and generated rows in duplicate groups. If those are high, run Remove duplicates in post-processing and download again.
- Before crossing the data by segment: Answered crosstab fidelity, not the overall fidelity.
- Ignore in isolation: Fit on training data (%). Compare it with the majority baseline or not at all.
The report is built by a worker rather than in your browser, so it behaves like any other large download — see Waiting, queues and errors.