A digital health company develops an algorithm that analyzes retinal images to detect diabetic retinopathy. The algorithm demonstrates exceptional performance on training and validation datasets, with sensitivity and specificity that exceed those of most human ophthalmologists. The company concludes that these analytical performance metrics constitute sufficient clinical evidence for MDR conformity assessment. The notified body disagrees. What the company demonstrated was that the algorithm correctly identifies features in images; what regulators require is evidence that using the algorithm in clinical practice benefits patients compared to standard care. These are not the same question, and conflating them is among the most common clinical evidence failures in SaMD submissions.
1. What "Proportionate" Clinical Evidence Means for Software
The EU Medical Device Regulation requires manufacturers to demonstrate that their devices achieve their intended purpose and are safe when used as intended. For medical devices generally, this obligation translates into clinical evaluation requirements under Art. 61 and Annex XIV MDR: manufacturers must plan, conduct, and document clinical evaluation that critically assesses available clinical data to confirm safety and performance throughout the device's lifecycle.1Regulation (EU) 2017/745, Art. 61(1) and Annex XIV Part A. The regulation does not exempt software from these requirements; software classified as a medical device under Rule 11 MDR must demonstrate clinical evidence proportionate to its risk classification.
The MDCG has issued specific guidance on clinical evaluation of SaMD (MDCG 2020-1), recognizing that software presents distinct evidentiary challenges.2MDCG 2020-1, Guidance on Clinical Evaluation (MDR) / Performance Evaluation (IVDR) of Medical Device Software (March 2020). Traditional clinical evaluation methodology (identifying equivalent devices, reviewing published literature, analyzing clinical investigation data) may not transfer directly to software contexts. An algorithm processing medical images has no physical equivalent device; published literature on software performance may not address the specific clinical context; and clinical investigations involving software require careful design to isolate software contribution from other clinical variables.
Swiss requirements under the MepV substantially mirror MDR clinical evaluation obligations. Swissmedic applies the same proportionality principle: clinical evidence must match device risk classification and claimed clinical benefits. Under Rule 11 MDR, SaMD that provides diagnostic or therapeutic information is classified as at least Class IIa; the analysis of SaMD classification covers where that rule is hard to apply. Very little SaMD is Class I in practice: the residual Class I limb of Rule 11 reaches only software that neither provides information for diagnostic or therapeutic decisions nor monitors physiological processes, so most software with a medical purpose is drawn into the higher limbs. A Class IIa SaMD requires clinical evaluation demonstrating safety and performance; a Class IIb SaMD, where decisions taken on the information it provides may cause serious deterioration of health or a surgical intervention, demands a more developed evidence base; a Class III SaMD, where such decisions may cause death or irreversible deterioration, requires stronger evidence still. Manufacturers must work out where on this spectrum their software falls and what evidence package its classification requires.
The mutual recognition agreement between Switzerland and the EU lapsed for medical devices in May 2021, because its medical devices chapter was never updated to cover the MDR. Swiss-based SaMD manufacturers are therefore third-country manufacturers under the MDR. For EU market access they must designate an EU Authorized Representative and meet EU clinical evidence requirements, while also satisfying MepV requirements for the Swiss market. Swissmedic has published guidance on clinical evaluation requirements that aligns with MDCG methodology,3Art. 46(3) MepV, SR 812.213; Swissmedic, Information Sheet on Medical Device Software (BW630_30_007, V3.0, 10 April 2024). but keeping parallel clinical evidence documentation for two regulatory systems, each with its own submission formats and review timelines, is a burden that manufacturers serving only the EU market do not carry.
The gap between proving an algorithm works and proving it helps patients is where most SaMD evidence strategies fail. Notified bodies have learned to look for it.
2. Analytical vs. Clinical Validation
The distinction between analytical validation and clinical validation represents the conceptual foundation of SaMD clinical evidence strategy. Analytical validation confirms that the software performs its technical function correctly: that an image classification algorithm correctly identifies the features it claims to identify, that a risk calculator produces mathematically correct outputs, that a diagnostic support system applies its rules as designed. Clinical validation confirms that using the software in clinical practice achieves beneficial health outcomes for patients.
The IMDRF SaMD Framework provides a structure that many regulators reference when assessing clinical evidence adequacy.4IMDRF SaMD Working Group, Software as a Medical Device: Clinical Evaluation (IMDRF/SaMD WG/N41FINAL:2017). The framework distinguishes three evidentiary components: valid clinical association between the software output and the targeted clinical condition, analytical validation demonstrating correct technical performance, and clinical validation demonstrating that using the software achieves claimed benefits under conditions of intended use. All three components must be addressed, though the depth of evidence required for each varies with the software's risk categorization.
The IMDRF framework, however, was developed in 2017, before the post-2020 wave of machine-learning-based SaMD deployment, and its limitations have become apparent.5IMDRF N41FINAL:2017 (n 4); for the underlying risk categorization, IMDRF/SaMD WG/N12FINAL:2014. The three-component model assumes static software outputs: a fixed algorithm produces deterministic results from given inputs, enabling clean separation between analytical and clinical validation. Adaptive algorithms that learn from deployment data challenge this assumption, because the software's analytical performance is no longer fixed between validation and clinical use. Several regulators (notably Health Canada and the US FDA) have noted these limitations for continuously learning systems. Despite these constraints, the IMDRF model remains useful as a foundational structure for clinical evidence planning, provided manufacturers recognize where adaptive AI/ML behavior exceeds its assumptions.6IMDRF N41FINAL:2017 (n 4); MDCG 2020-1 (n 2), s 4.
Valid clinical association establishes the scientific foundation linking software output to clinical meaning. If software calculates a cardiovascular risk score, valid clinical association requires evidence that the score meaningfully predicts cardiovascular events, typically drawn from published medical literature, clinical practice guidelines, or foundational clinical research. This component often receives insufficient attention in SaMD submissions; manufacturers focus on demonstrating that their software correctly calculates the score without establishing why that score matters clinically.
Analytical validation demonstrates that the software correctly generates its intended outputs from given inputs. For diagnostic algorithms, this typically involves testing against annotated datasets with known ground truth: images where expert consensus establishes the correct diagnosis, patient records where confirmed outcomes are documented. Performance metrics (sensitivity, specificity, positive predictive value, area under the ROC curve) quantify analytical validation results. These metrics are necessary but not sufficient; they establish that the software works technically but do not establish that using it benefits patients.
Analytical validation does not exist in isolation; it connects to the broader software lifecycle standards that notified bodies assess within the technical documentation framework. IEC 62304 governs the software development lifecycle, establishing requirements for design verification and validation (V&V) that directly support analytical validation arguments.7IEC 62304:2006/AMD1:2015 (medical device software lifecycle); IEC 82304-1:2016 (health software products); ISO 14971:2019 (risk management). IEC 82304 addresses health software products more broadly, while ISO 14971 risk management outputs feed into clinical evidence adequacy by identifying which software functions require clinical validation and at what depth. Notified bodies assess clinical evidence within this broader technical documentation context, and the interaction between lifecycle documentation quality and clinical evidence scrutiny remains difficult to predict. Software lifecycle compliance does not substitute for clinical evidence, but the relationship between the two shapes how notified bodies evaluate the evidentiary package as a whole.
Clinical validation bridges the gap between analytical performance and patient benefit. An algorithm that correctly identifies a condition on imaging may not improve patient outcomes if clinicians ignore its output, if the condition would have been identified through standard care anyway, if identification leads to unnecessary interventions, or if real-world image quality differs from validation datasets. Clinical validation studies address these questions through prospective evaluation of software use in clinical settings, comparison against standard-of-care pathways, and measurement of clinically meaningful endpoints.
3. Can Real-World Evidence Replace Clinical Trials?
The role of real-world evidence in SaMD clinical evaluation has expanded significantly as regulators recognize that traditional clinical investigation designs may not capture software performance in actual use conditions. Real-world evidence (data derived from sources outside traditional clinical trials, including electronic health records, registries, insurance claims, and patient-generated data) can supplement or, in appropriate circumstances, substitute for prospective clinical investigation data.
The MDCG has addressed real-world evidence considerations for SaMD, acknowledging that software performance may vary between controlled validation environments and clinical deployment.8MDCG 2020-1 (n 2), s 4; real-world evidence considerations are discussed throughout the guidance document. An algorithm trained and validated on high-quality images from academic medical centers may perform differently when processing images from community hospitals with older equipment and variable imaging protocols. Real-world evidence can reveal performance gaps that controlled validation studies miss, but only if the evidence is collected systematically with appropriate quality controls.
The evidentiary value of real-world evidence depends heavily on data quality, completeness, and relevance to the intended use population. Registry data capturing software performance across diverse clinical settings may provide stronger evidence of generalizability than a controlled study at a single institution, but only if the registry captures outcomes with sufficient rigor and the data can be linked to software use with confidence. Whether registry-based real-world evidence (RWE) will be accepted depends on whether selection bias, confounding, missing data, and outcome ascertainment have been addressed at the methodological standard that clinical-investigation review applies, a threshold the regulation does not specify and that operates differently across notified bodies.
Post-market clinical follow-up under Annex XIV Part B MDR extends clinical evidence obligations beyond initial certification. Manufacturers must proactively collect and evaluate clinical data throughout the device lifecycle, confirming that clinical evidence remains adequate as the device is used in broader populations and as clinical practice evolves. For SaMD, this ongoing obligation has particular significance: software that learns from data may change its behavior over time, and clinical evidence supporting the initial version may not apply to evolved versions. The relationship between post-market clinical follow-up and AI/ML algorithm updates presents unresolved regulatory questions that manufacturers must address in their clinical evidence strategies.
Two further matters bear on real-world evidence: what regulators expect of its quality, and the HTA rules of Regulation (EU) 2021/2282. As of October 2025, the MDCG continues to develop guidance on real-world data quality standards for clinical evaluation, refining expectations around data provenance, completeness, and fitness for regulatory purpose. Separately, the EU HTA Regulation (2021/2282), applicable from 12 January 2025 for joint clinical assessments of certain medicinal products and reaching specified high-risk medical devices and Class D IVDs through periodic Commission selection rather than a fixed commencement date, introduces a parallel evidentiary framework where RWE standards for health technology assessment may diverge from those applied in regulatory approval.9Regulation (EU) 2021/2282 on health technology assessment [2021] OJ L458/1; JCAs from 12 January 2025 (oncology medicinal products and ATMPs); Class III implantable devices and Class IIb active devices that administer or remove a medicinal product, where an expert panel has given its opinion under Art. 54 MDR, and Class D IVDs, enter by periodic Commission selection under Art. 7(4) HTA Regulation, not a fixed date. SaMD manufacturers seeking both CE marking and reimbursement face a dual-evidence burden: clinical evidence sufficient for MDR conformity assessment may not satisfy HTA bodies' requirements for comparative effectiveness and budget impact data. Whether evidence generation can be aligned across both frameworks from the outset, or whether two separate packages are needed, depends on data governance choices that intersect with privacy requirements in ways the regulations do not coordinate.
Germany's DiGA pathway offers the most developed European model for SaMD reimbursement and illustrates the practical interplay between regulatory and health-economic evidence requirements. Under the DVG and its implementing regulation (DiGAV),10Digitale-Versorgung-Gesetz (DVG) vom 9. Dezember 2019 (BGBl I S 2562); Digitale Gesundheitsanwendungen-Verordnung (DiGAV) vom 8. April 2020 (BGBl I S 768). manufacturers can obtain provisional listing in the DiGA directory on preliminary evidence for up to twelve months to generate confirmatory evidence of positive healthcare effects through controlled studies, and the BfArM may extend that period by up to twelve more where the trial results make later proof more likely than not. The DiGA evidence requirements operate independently of MDR clinical evidence obligations (a device must satisfy both to achieve reimbursed market access), creating strategic complexity around evidence planning and study design. Belgium's framework for mobile health applications (applications mobiles de santé) and France's emerging pathway for digital medical devices (dispositifs médicaux numériques) signal that comparable reimbursement-linked evidence requirements will spread across European markets, raising parallel questions about evidence for reimbursement that neither the MDR nor the HTA Regulation settles.
4. Why No Study Design Clearly Fits SaMD
Clinical validation studies for SaMD require careful design to generate evidence that regulators will accept as demonstrating clinical benefit. The fundamental challenge lies in isolating the software's contribution to clinical outcomes from other variables: physician expertise, patient characteristics, healthcare system factors, and concurrent treatments all influence outcomes independently of whether the software is used. Study designs must control or account for these confounding factors while remaining practically feasible and ethically acceptable.
Randomized controlled trials represent the gold standard for clinical evidence but present practical challenges for SaMD. Randomizing patients to receive care with or without software assistance raises ethical questions when preliminary evidence suggests software benefit. Blinding is often impossible; clinicians know whether they are using the software. Crossover designs may be contaminated by learning effects as clinicians internalize software recommendations. Despite these challenges, randomized designs remain the most convincing evidence for regulatory purposes, particularly for higher-risk SaMD where clinical decisions have serious consequences.
Comparative effectiveness designs may provide acceptable evidence when randomization is impractical. Comparing outcomes between facilities that adopt the software and facilities that continue standard care can demonstrate clinical benefit if confounding factors are adequately controlled. Propensity score matching, difference-in-differences analysis, and synthetic control methods can strengthen causal inference from observational data, but these approaches require methodological sophistication and large sample sizes that many SaMD manufacturers cannot achieve.
As of 2025, regulators accept alternative study designs that reflect the practical realities of SaMD evaluation. Single-arm studies with external (historical) controls can generate acceptable evidence when randomized comparison is impractical, particularly where the comparator is no software assistance at all and clinical equipoise is limited. Basket and umbrella trial designs, originally developed for oncology therapeutics, are being adapted for SaMD contexts where a single algorithm addresses multiple clinical indications or where multiple algorithms target the same condition. The FDA's De Novo pathway has produced instructive precedents: the authorization of IDx-DR for autonomous diabetic retinopathy detection relied on a prospective single-arm study against a reading-center reference standard, establishing a regulatory template for AI-based diagnostic SaMD. These developments suggest that regulators accept a wider range of study designs than a strict reading of traditional evidence hierarchies would imply.
Endpoint selection profoundly influences study feasibility and regulatory acceptability. Clinical endpoints (mortality, disease progression, quality-adjusted life years) provide the most compelling evidence of patient benefit but require long follow-up periods and large sample sizes. Surrogate endpoints (diagnostic accuracy, time to diagnosis, treatment initiation rates) can be measured more quickly but require validation linking the surrogate to clinical outcomes. Regulators may accept surrogate endpoints for lower-risk SaMD but require clinical endpoints for higher-risk classifications, particularly when the clinical benefit claim involves improved patient outcomes rather than merely improved diagnostic performance.
MDCG 2020-1 ties study design to what the software's output does: a prospective study may be required where the software determines a patient's future state, such as a prognosis, or its output affects clinical outcomes or patient management decisions, and a retrospective analysis may be more appropriate otherwise. The guidance does not frame the question by whether a clinician checks the output, and software whose output a clinician checks before acting can still affect patient management decisions, so manufacturers' assumptions about clinician independence may not survive regulatory scrutiny or real-world observation of actual use patterns.11MDCG 2020-1 (n 2), s 4.4.1 (clinical investigations and clinical performance studies).
5. Why Evidence Planning Fails When It Follows Product Design
Clinical evidence strategy for SaMD interacts with product conception in ways that are difficult to retrofit once development is complete. The evidence required for regulatory approval shapes product design decisions: intended use claims, target populations, clinical workflows, and performance thresholds all affect what evidence manufacturers must generate. A company that designs a product and then asks what evidence regulators require may discover that feasible evidence strategies do not support the intended claims, forcing either claim modification or evidence generation that delays market entry by years.
Classification Interactions
Intended use claims, the evidence a manufacturer can feasibly produce, and what regulators expect depend on one another, so they cannot be settled one at a time. An intended use claim that appears straightforward may demand prospective clinical validation rather than analytical performance data alone, yet the feasibility of prospective validation depends on the available clinical literature establishing the foundational association the software exploits. Where that literature is thin, the manufacturer faces evidence generation at the foundational level before clinical validation can even be designed. Whether real-world evidence is accepted depends on the device's classification and the jurisdiction, and also on how the particular notified body reads MDCG guidance, so a general evidence plan cannot reliably predict the outcome of conformity assessment.
A lower class does not always bring lighter scrutiny: a notified body assessing a contested Class IIa argument may treat the evidence with more skepticism than it would an uncontested Class IIb submission, even though Class IIa carries the lower formal requirements. Classification and evidence burden therefore have to be assessed together, and how they bear on each other varies by notified body, product type, and the maturity of guidance in the relevant clinical domain.
AI/ML and Regulatory Convergence
The EU AI Act (Regulation (EU) 2024/1689) classifies most medical device AI systems as high-risk, imposing data governance, transparency, and human oversight requirements that interact with, but do not replace, MDR clinical evidence obligations.12Regulation (EU) 2024/1689 (AI Act), OJ L, 2024/1689, 12.7.2024, Art. 27, 43(3), 72. Art. 43(3) AI Act integrates the AI Act's high-risk requirements into the MDR conformity assessment procedure, so that they are assessed as part of that single procedure, potentially streamlining compliance for manufacturers facing dual regulatory frameworks. Whether this integrated assessment mechanism will prove sufficient in practice, or whether notified bodies will effectively require parallel evidentiary submissions addressing each regulation's distinct requirements, remains an open question as implementing guidance develops.
The AI Act also introduces obligations with no direct MDR equivalent, though their reach into SaMD is narrower than a first reading suggests. The fundamental rights impact assessment under Art. 27 AI Act applies to deployers that are bodies governed by public law or private entities providing public services, but its scope is restricted to the high-risk AI systems referred to in Art. 6(2) AI Act (the stand-alone categories listed in Annex III AI Act). Medical-device AI is high-risk under Art. 6(1) AI Act (the product-safety legislation listed in Annex I AI Act) and therefore generally falls outside the reach of Art. 27 AI Act; whether SaMD deployers in public-sector contexts nevertheless face parallel fundamental-rights review obligations under national or sectoral law is a separate question the AI Act does not foreclose. Post-market monitoring under Art. 72 AI Act imposes continuous performance surveillance requirements that parallel, but do not replace, MDR post-market surveillance obligations under Art. 83 and Annex III MDR. Whether one post-market monitoring system can satisfy both frameworks, or two separate systems are needed, depends on the deployment context, and as of October 2025 neither the MDCG 2020-1 guidance nor the AI Act implementing measures fully address the burden of keeping compliance documentation for both regimes at the same time.
Across the Atlantic, the FDA's predetermined change control plan (PCCP) framework, finalized in December 2024, offers an alternative regulatory model for managing AI/ML algorithm evolution.13FDA, Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions (December 2024). The PCCP allows manufacturers to pre-specify, within the original marketing submission, anticipated modifications to algorithm performance or retraining methodology that keep the device within its intended use, so that these updates need no new regulatory clearance each time. A comparable approach is taking shape internationally: as of October 2025, the IMDRF is developing guidance on predetermined change control plans for AI-enabled medical devices, which is expected to inform future EU guidance. For manufacturers targeting both markets, whether clinical evidence strategies can be aligned with both the PCCP framework and emerging EU change management requirements, or whether duplicative regulatory submissions become necessary, remains unsettled as EU guidance develops.
Pre-Market vs. Post-Market Evidence Strategy
Manufacturers may obtain certification with limited clinical evidence if they commit to collecting post-market data that confirms clinical benefit in real-world use. This approach accelerates market entry but creates ongoing obligations and risks: if post-market data reveals inadequate clinical performance, regulatory consequences may include market withdrawal. Whether to pursue accelerated market entry with post-market evidence commitments or to invest in more comprehensive pre-market evidence depends on competitive dynamics, capital availability, and risk tolerance that vary by manufacturer and market context.