The AI Act's Synthetic-Data Clause Has Never Been Tested

Since 27 July 2026 a provider of a high-risk medical AI system may process special-category personal data to detect bias only when synthetic or anonymised data cannot do the job, and no published standard yet defines how to prove that it cannot. Regulation (EU) 2026/1744, the Digital Omnibus on AI, moved that condition out of Article 10(5) of the AI Act into a new Article 4a, extended it to deployers and to providers of other systems, and attached a duty to record the reasons. The published research on synthetic data says it fails most often exactly where bias testing looks, in the smallest groups. Read together, the clause and the evidence turn what most commentary treats as a permission into a measurement that medical AI providers will have to make, and write down, before their high-risk obligations apply.

What does Article 4a actually say?

The AI Act of 2024 required providers of high-risk systems to examine their training, validation and testing data for biases likely to affect health and safety, fundamental rights or non-discrimination, and to take measures to detect, prevent and mitigate them. Those duties sit in Article 10(2), points (f) and (g). Because bias detection often needs the protected attribute itself, the original Article 10(5) let providers process special-category data for that purpose, under conditions. The first condition was that detection "cannot be effectively fulfilled by processing other data, including synthetic or anonymised data".

The Omnibus, adopted on 8 July 2026 and in force since 27 July 2026, deletes Article 10(5) and inserts Article 4a. Paragraph 1 keeps the six conditions for providers of high-risk systems, and the first condition keeps its wording. Condition (f) now requires that the records of processing "include the reasons why the processing of special categories of personal data was strictly necessary to detect and correct biases, and why that objective could not be achieved by processing other data". Paragraph 2 extends the permission to providers and deployers of other AI systems and models, and to deployers of high-risk systems, where bias is likely to affect health, safety or fundamental rights. It adds that the paragraph "does not create any obligation to conduct such bias detection and correction". Article 10(1) now points to Article 4a(1) directly.

The same Regulation moved the dates. Obligations for systems listed in Annex III, which include emergency triage and eligibility for healthcare services, now apply from 2 December 2027. Obligations for systems that are high-risk because they are regulated products under Annex I, which is where most clinical AI sits through medical-device law, apply from 2 August 2028. Article 4a itself is already in force.

Why is "cannot be effectively fulfilled" a measurement?

A condition that must be recorded with reasons is a condition that must be tested. The record under condition (f) has to say why synthetic or anonymised data could not achieve bias detection. The only honest way to write that sentence is to have tried, or to point to evidence that trying would fail. Either way the provider needs three things. A definition of the subgroups bias detection will examine, including intersections such as sex within age band within ethnicity. A stated method for detecting bias, such as a performance gap between subgroups on a held-out test set. And either a structural bound on the synthetic release showing that the subgroup sits below its evaluated resolution, or a comparison run inside the data holder's environment showing that the method on synthetic data diverges from the real data. The first path can be walked before any special-category data are touched, which matters, because a provider cannot process real data to prove it needs to process real data. The clause also covers correction, and synthetic data fails harder there, since re-weighting or augmenting a rare intersectional subgroup is impossible when the generator has already smoothed its tail.

The clause assumes synthetic data is the gentler alternative and puts the burden on the provider to explain why it was not used. That burden is only as light as the evidence that synthetic data preserves what bias detection needs. The literature on that question is short, recent and consistent.

What does the evidence say?

Bias in medical AI is found in the tails. Seyyed-Kalantari and colleagues showed in 2021 that chest X-ray classifiers trained on three large public datasets under-diagnosed under-served groups, with the worst rates in intersectional subgroups such as Hispanic female patients. Daneshjou and colleagues showed in 2022 that dermatology models performed substantially worse on dark skin tones and on uncommon diseases. Obermeyer and colleagues found in 2019 that a care-management algorithm's racial bias was hidden when the score was judged against the healthcare costs it predicted, and visible only when clinical outcomes were examined alongside the score across groups. In each case detection depended on the joint distribution of inputs and clinical outcomes within a subgroup being preserved faithfully enough for a disparity to show.

Synthetic data generated under a formal privacy guarantee loses the tails first. Bagdasaryan, Poursaeed and Shmatikov showed in 2019 that private model training reduces accuracy far more for under-represented classes than for the majority. Ganev, Oprisanu and De Cristofaro showed in 2022 that private synthetic data generators distort both the size and the classifier accuracy of minority subgroups, and that whether a generator flattens or amplifies the differences depends on the mechanism chosen. Cheng and colleagues showed in 2021, on datasets including clinical data, that classifiers trained on private synthetic data lose accuracy and fairness for minority subgroups, and that the loss grows as privacy tightens. A 2025 scoping review by Mohammadi and colleagues of 74 medical deep-learning studies using differential privacy found that accuracy losses fell disproportionately on demographic subgroups, and that only a small minority of the studies reported any subgroup analysis at all.

Synthetic data without a formal guarantee can preserve subgroups better, as the benchmarking by Yan and colleagues in 2022 shows when generators are ranked on fidelity rather than privacy. But that data is then not "anonymised" in the sense Article 4a offers as the alternative. The European Data Protection Board's draft Guidelines 02/2026 state that synthetic data can still permit inferences about individuals and apply the singling-out, linkability and inference tests to it. Opinion 28/2024 requires that a generative model itself be assessed for what it leaks. The clause names two alternatives, synthetic and anonymised, but under current guidance they are one alternative judged twice, once for what it reveals and once for what it preserves.

The consequence is uncomfortable for anyone selling synthetic data as a shortcut. The better a synthetic dataset reproduces the minority subgroups that bias detection needs, the harder its anonymity claim becomes. The stronger its privacy guarantee, the more likely a provider can honestly record that condition (a) is met and proceed to real special-category data under the other five safeguards. Article 4a, read with the evidence, pushes providers toward real data under controls unless the synthetic data arrives with a subgroup-level account of what it does and does not preserve.

What would a provider have to show?

The record under condition (f) can be written as answers to five questions. Two of them can only be answered by whoever produced the synthetic data, which is why the clause reaches producers as much as providers.

Items 2 and 4 are properties of a release, not of a provider's project, and no current label format requires either. The TEHDAS2 guideline for health data access bodies asks for residual-risk assessment of synthetic outputs but is silent on subgroup reporting. The evaluation literature reviewed by Nafis and colleagues in 2025, covering 101 studies, found no consensus on which utility measures to report and poor reporting of dataset characteristics. A provider asking a supplier for item 2 today will usually receive a single utility score, which answers none of the five questions. The argument that evaluation must be stated per purpose is made in Fit for Which Purpose?; Article 4a makes bias detection a purpose with legal consequences.

Who else will ask the same question?

The European Medicines Agency's 2024 reflection paper on AI in the medicinal product lifecycle notes that synthetic data may be used to expand training data, subject to justification. The joint EMA and FDA principles of January 2026 ask for a stated context of use and data governance across the lifecycle. Neither says how a synthetic training set should be evaluated for subgroup coverage, so a provider's Article 4a record is likely to be the first document that does. From 26 March 2029, health data access bodies under the European Health Data Space will decide whether synthetic outputs may leave a secure processing environment, and a provider that wants to use such outputs for bias detection will be asking the access body the same five questions. The criteria an access body could apply are set out in What Makes a Synthetic Release "Adequate" Under the EHDS?

The question also has a representation dimension that the regulation does not name. Kaushal, Altman and Langlotz found in 2020 that 71 percent of US deep-learning studies with identifiable cohorts drew patients from three states. European training sets have not been audited the same way. A provider whose source data under-represents a group cannot detect bias against that group with real data either, and no synthetic derivative can add what the source never held. Article 4a asks whether synthetic data can substitute for real data. It does not ask whether the real data were ever adequate, and that is the question behind it.

What should providers do before December 2027?

• Define the subgroups now, including intersections, and record their counts in every training, validation and test set. This is item 1 and it belongs in the Article 10 file regardless of Article 4a.
• Ask every synthetic-data supplier for results per subgroup and for the privacy status of the release, in writing. A supplier that cannot answer has answered.
• Where the synthetic certificate already lists a subgroup as below its evaluated floor, file that contraindication as the reason. For subgroups above the floor, run the chosen bias metric on synthetic and on held-out real data inside a controlled environment and keep the comparison, whichever way it comes out. A record that synthetic data was sufficient is as valuable as one that it was not, because the next audit will ask.
• Write the condition (f) statement when the comparison is done, not when the file is requested.

Frequently asked questions