Formal Privacy Costs the Smallest Groups Most

On our own public release, a synthetic cohort generated under formal privacy reproduces the population's mix of diagnoses inside every subgroup, which makes it nearly right for the majority and wrong for each minority in proportion to how different that minority really is. We measured it. We split the 5,000-person SX-SAMPLE-SYNPUF-001 release and its 100,000-person public seed into 24 subgroups by sex, recorded race and birth decade, and compared each synthetic subgroup with its real counterpart, with a plain random sample of the seed as the control. The sample keeps 90 to 99 percent concordance in every subgroup. The release keeps about 80 percent in every subgroup, and that 80 percent is the distance to the population, not to the subgroup. This note publishes the numbers, the method and the consequence for what a certificate must say, because the AI Act now asks providers to show whether synthetic data can detect bias in exactly these groups.

What did we measure, and on what?

The release is the one described in Synthetic Medicare Claims Data. It holds 5,000 synthetic persons across five OMOP CDM v5.3-format tables, generated at ε = 1.0 and δ = 10⁻⁵ with the privacy unit set to a whole person, seeded from the 100,000-person OMOP conversion of the CMS DE-SynPUF file on the AWS Open Data registry. The seed is itself synthetic, which is why we can publish everything.

We defined 24 subgroups by crossing sex (two values), the race recorded in the seed (White, Black, and Other or unknown) and birth decade (before 1930, the 1930s, the 1940s, 1950 and later). The largest subgroup, women recorded as White and born in the 1930s, holds 16,403 seed persons. The smallest, women in the Other group born 1950 or later, holds 409. For each subgroup we compared three code tables, diagnoses, procedures and drugs, between the seed and the release.

Concordance is the measure the certificate already uses, one minus the Jensen-Shannon distance, computed on the 50 most frequent codes in the seed subgroup with every other code pooled into a remainder. We also counted, for each subgroup, how many of the codes that occur at least 20 times in the seed subgroup appear at all in the synthetic subgroup. As a control, we drew a random sample of 5,000 persons from the seed with a fixed random seed and ran the same comparisons, then repeated the draw 100 times with different seeds to check that no row below depends on a lucky draw. The control answers the question any sceptic should ask first, which is how much of the loss is simply having 5,000 people instead of 100,000. The scripts and outputs are in the method record that accompanies this note and can be run by anyone holding the release and the public seed.

What does the release actually contain?

The seed holds 10,217 distinct diagnosis codes, 8,462 procedure codes and 267,763 drug codes. The random sample of 5,000 persons holds 7,901, 5,085 and 133,801. The release holds 51 of each, the 50 most frequent codes in the population plus a pooled remainder. The certificate's fidelity snapshot states that code-level comparisons are computed on that support. What it does not state, and what this note adds, is what the support means for anyone looking at a subgroup. In the release, 44 percent of diagnosis rows, 36 percent of procedure rows and 76 percent of drug rows carry the remainder label rather than a code.

That is the first number a buyer should know, and it is not a subtle one. A study of a condition outside the top 50 cannot be done on this release at all, in any subgroup, because the code is not there. The contraindication for rare-disease cohorts printed on the certificate says this in words. The table below says it in numbers.

What happens inside each subgroup?

Three things stand out. First, the control behaves as sampling theory says it should. Concordance falls gently with subgroup size, from 0.985 in the largest group to about 0.88 in the smallest, and most codes survive. Second, the release does not fall with size at all. It sits between 0.79 and 0.81 for every one of the 24 subgroups. Third, the retained-code column for the release rises in the smallest groups only because those groups have fewer codes above the threshold to lose. In absolute terms almost nothing outside the population's top 50 survives anywhere. Across the 100 repeated draws, the control's 5th to 95th percentile band for diagnosis concordance runs from 0.974 to 0.987 in the largest subgroup and from 0.850 to 0.923 in the smallest, and its 5th percentile never falls below 0.84 in any subgroup, so the gap to the release's 0.79 to 0.81 is not an artefact of one sample.

The flat line is the finding, and it is two effects at once, which should be kept apart. The first is support truncation. Pooling every code outside the top 50 into a remainder removes rare codes from every subgroup alike and is what pulls even the whole-population concordance from 0.99 in the control to 0.81 in the release. The second is missing subgroup conditioning. Within the 50 codes that remain, each synthetic subgroup carries the population's distribution rather than its own, which is why the release is no closer to any subgroup than to the average. We then asked whether each synthetic subgroup resembles its own real subgroup more than it resembles the population as a whole. It does not. Across all 24 subgroups, concordance of the synthetic subgroup against its real counterpart ranges from 0.789 to 0.813, and against the whole seed population from 0.795 to 0.812. The two numbers are the same to within noise in every row. The sample control, by contrast, is 0.88 to 0.99 against its own subgroup. The release reproduces one mix of diagnoses and places it inside every subgroup.

Why that costs the smallest groups most is visible in the seed itself. On the same support, the majority subgroups differ from the population by a Jensen-Shannon distance of about 0.01. The smallest subgroups differ by 0.04 to 0.05, and in the drug table by up to 0.08. A release that gives every subgroup the population mix is therefore almost right for a White woman born in the 1930s and most wrong for the groups whose care looked least like the average. That is the mechanism Ganev, Oprisanu and De Cristofaro called the Matthew effect for private synthetic data, observed here on a clinical release rather than a benchmark.

Two smaller observations belong in the record. Subgroup sizes are preserved to within sampling error in most rows, but not all. The release has 609 women recorded as White and born in the 1940s where 501 were expected, and 44 Black men born 1950 or later where 63 were expected. And 78 of the 5,000 synthetic persons carry a birth year after 1983, which is the latest year in the seed. The age distribution has been smoothed past the edge of the real one, which is what the certificate's age-band concordance of 0.941 looks like when you open the file.

Why does formal privacy do this?

None of this is a surprise to the literature, and that is the point of publishing it. Bagdasaryan, Poursaeed and Shmatikov showed in 2019 that private model training reduces accuracy far more for under-represented classes than for the majority. Farrand and colleagues showed in 2020 that even small imbalance produces unequal accuracy under private training. Ganev, Oprisanu and De Cristofaro showed in 2022 that private synthetic data generators distort the size and classifier accuracy of minority subgroups, and that whether the distortion flattens or amplifies differences depends on the mechanism. Cheng and colleagues showed in 2021, on datasets including clinical ones, that classifiers trained on private synthetic data lose accuracy and fairness for minority subgroups as privacy tightens. The 2025 scoping review by Mohammadi and colleagues of 74 medical deep-learning studies found that the losses fall on demographic subgroups and that only a small minority of studies report any subgroup result.

The reason is structural rather than a flaw in any one method. A formal privacy guarantee bounds how much any one person can change the output. A rare code or an unusual combination of codes is, by definition, carried by few people, so it is exactly what the bound suppresses first. Dwork and Roth set this out in 2014 as the price of the guarantee, and no mechanism escapes it. What a mechanism can choose is where the price is paid. On a fixed budget, estimating separate code distributions inside 24 demographic intersections would multiply the noise in every cell, and the contents of this release show the opposite choice, one population-level distribution over a 50-code support. That keeps the table-level concordances high and costs the release its subgroup-specific differences. It is a defensible choice. It is not a free one.

Official statistics went through this argument first. Santos-Lozada, Howard and Verdery showed in 2020 that formally private census counts distort mortality-rate estimates most for small, rural and minority populations. Krieger and colleagues found in 2021 that the effect on health-inequity estimates was largest in tracts with small populations. Steed and colleagues showed in 2022 that the resulting shifts in school funding were real but often smaller than the uncertainty already present in the data. The health-data field is now where the statistical agencies were in 2020, with one difference. The agencies published their utility-loss accounting. Synthetic-data producers mostly publish a single utility score.

What does this mean for a certificate?

A single number cannot carry this. The certificate for this release reports a composite marginal concordance of 0.91, and that figure is true. It is also true that the release reproduces nothing specific to any of the 24 subgroups below the population's top 50 codes. Both facts have to be printed, and the second has to be printed as a floor, the subgroup size and code frequency below which the release was not shown to preserve anything, rather than as a caution.

The uncomfortable part is what the floor means. Printing a subgroup breakdown does not rescue the synthetic dataset for those subgroups. It rescues the downstream researcher from using the dataset where it is blind. When a subgroup sits below the floor, the honest recommendation on a certificate is not "use with caution". It is "do not use synthetic data for this subgroup, apply for permit access to the real records inside the secure processing environment". A certificate whose job is sometimes to send the buyer away from the product is the only kind worth issuing, and it is the document that Article 4a of the amended AI Act now asks a medical AI provider to hold before touching real special-category data. Mitchell and colleagues proposed disaggregated reporting for models in 2019. Markatou and colleagues proposed in 2025 that EHDS metadata expose dataset diversity. Neither covers a privacy-induced floor on a synthetic release, and that is the field we are in.

The fitness-for-purpose frame in Fit for Which Purpose? already placed rare-disease cohorts and safety signals in the rows where synthetic data is unsuitable. This note gives those rows a measurement, and it adds a row the earlier article did not have. Any question whose answer depends on how one subgroup differs from the population is, on this release, unanswerable, and that includes most questions about bias.

What we will do next

• Print the floor. The next revision of the certificate will state, per release, the subgroup size and code frequency below which fidelity was not demonstrated, in the contraindications section rather than in a footnote.
• Report per subgroup by default. The 24-subgroup table above, or its equivalent for the release's own protected attributes, will accompany every release alongside the composite score.
• Repeat this on a release with a wider support, and publish the cost on the majority of widening it, so that the trade is visible rather than asserted.
• Invite replication. The release, the public seed and the scripts are enough for any group to check these numbers without trusting us, and we would rather be corrected than agreed with.

Frequently asked questions