A digital health company develops an algorithm that analyzes retinal images to detect diabetic retinopathy. The algorithm demonstrates exceptional performance on training and validation datasets, with sensitivity and specificity that exceed most human ophthalmologists. The company concludes that these analytical performance metrics constitute sufficient clinical evidence for MDR conformity assessment. The notified body disagrees. What the company demonstrated was that the algorithm correctly identifies features in images; what regulators require is evidence that using the algorithm in clinical practice benefits patients compared to standard care. These are not the same question, and conflating them is among the most common clinical evidence failures in SaMD submissions.
1. What "Proportionate" Clinical Evidence Means for Software
The EU Medical Device Regulation requires manufacturers to demonstrate that their devices achieve their intended purpose and are safe when used as intended. For medical devices generally, this obligation translates into clinical evaluation requirements under Art. 61 and Annex XIV MDR: manufacturers must plan, conduct, and document clinical evaluation that critically assesses available clinical data to confirm safety and performance throughout the device's lifecycle.1Regulation (EU) 2017/745, Art. 61(1) and Annex XIV Part A. The regulation does not exempt software from these requirements; software classified as a medical device under Rule 11 MDR must demonstrate clinical evidence proportionate to its risk classification.
The MDCG has issued specific guidance on clinical evaluation of SaMD (MDCG 2020-1), recognizing that software presents distinct evidentiary challenges.2MDCG 2020-1, Guidance on Clinical Evaluation (MDR) / Performance Evaluation (IVDR) of Medical Device Software (2020). Traditional clinical evaluation methodology (identifying equivalent devices, reviewing published literature, analyzing clinical investigation data) may not transfer directly to software contexts. An algorithm processing medical images has no physical equivalent device; published literature on software performance may not address the specific clinical context; and clinical investigations involving software require careful design to isolate software contribution from other clinical variables.
Swiss requirements under the MepV substantially mirror MDR clinical evaluation obligations. Swissmedic applies the same proportionality principle: clinical evidence must match device risk classification and claimed clinical benefits. Under Rule 11 MDR (whose classification complexities are examined in Insight 07), SaMD providing diagnostic or therapeutic information is classified as minimum Class IIa. Very little SaMD is Class I in practice: the residual Class I limb of Rule 11 reaches only software that performs no action on data beyond storage, archival, communication, or simple search, so most software with a medical purpose is drawn into the higher limbs. A Class IIa SaMD requires clinical evaluation demonstrating safety and performance; a Class IIb SaMD whose decisions may cause serious deterioration of health or surgical intervention demands a more developed evidence base; a Class III SaMD whose diagnostic or therapeutic decisions may cause death or irreversible deterioration requires substantially more robust evidence. The challenge for manufacturers lies in determining where on this spectrum their software falls and what evidence package the specific classification requires.
Swiss manufacturers face an additional structural consideration. Following the non-update of the medical devices chapter of the mutual recognition agreement between Switzerland and the EU in May 2021, Swiss-based SaMD manufacturers are third-country manufacturers under MDR, requiring designation of an EU Authorized Representative and compliance with EU clinical evidence requirements for EU market access, while simultaneously satisfying MepV requirements for the Swiss market. Swissmedic has published guidance on clinical evaluation requirements that aligns with MDCG methodology,3Art. 46(3) MepV, SR 812.213; Swissmedic, Information Sheet on Medical Device Software (BW630_30_007, V2.0, 26 May 2022). but the practical burden of maintaining parallel clinical evidence documentation for two regulatory systems, each with distinct submission formats and review timelines, adds complexity that purely EU-based manufacturers do not face.
The gap between proving an algorithm works and proving it helps patients is where most SaMD evidence strategies fail. Notified bodies have learned to look for it.
2. Analytical vs. Clinical Validation
The distinction between analytical validation and clinical validation represents the conceptual foundation of SaMD clinical evidence strategy. Analytical validation confirms that the software performs its technical function correctly: that an image classification algorithm correctly identifies the features it claims to identify, that a risk calculator produces mathematically correct outputs, that a diagnostic support system applies its rules as designed. Clinical validation confirms that using the software in clinical practice achieves beneficial health outcomes for patients.
The IMDRF SaMD Framework provides a structure that many regulators reference when assessing clinical evidence adequacy.4IMDRF SaMD Working Group, Software as a Medical Device: Clinical Evaluation (IMDRF/SaMD WG/N41FINAL:2017). The framework distinguishes three evidentiary components: valid clinical association between the software output and the targeted clinical condition, analytical validation demonstrating correct technical performance, and clinical validation demonstrating that using the software achieves claimed benefits under conditions of intended use. All three components must be addressed, though the depth of evidence required for each varies with the software's risk categorization.
The IMDRF framework, however, was developed in 2017, before the post-2020 wave of machine learning–based SaMD deployment, and its limitations have become apparent.5IMDRF N41FINAL:2017 (n 4); for the underlying risk categorization, IMDRF/SaMD WG/N12FINAL:2014. The three-component model assumes static software outputs: a fixed algorithm produces deterministic results from given inputs, enabling clean separation between analytical and clinical validation. Adaptive algorithms that learn from deployment data challenge this assumption, because the software's analytical performance is no longer fixed between validation and clinical use. Several regulators (notably Health Canada and the US FDA) have noted these limitations for continuously learning systems. Despite these constraints, the IMDRF model remains useful as a foundational structure for clinical evidence planning, provided manufacturers recognize where adaptive AI/ML behavior exceeds its assumptions.6IMDRF N41FINAL:2017 (n 4); MDCG 2020-1 (n 2), s 4.
Valid clinical association establishes the scientific foundation linking software output to clinical meaning. If software calculates a cardiovascular risk score, valid clinical association requires evidence that the score meaningfully predicts cardiovascular events, typically drawn from published medical literature, clinical practice guidelines, or foundational clinical research. This component often receives insufficient attention in SaMD submissions; manufacturers focus on demonstrating that their software correctly calculates the score without establishing why that score matters clinically.
Analytical validation demonstrates that the software correctly generates its intended outputs from given inputs. For diagnostic algorithms, this typically involves testing against annotated datasets with known ground truth: images where expert consensus establishes the correct diagnosis, patient records where confirmed outcomes are documented. Performance metrics (sensitivity, specificity, positive predictive value, area under the ROC curve) quantify analytical validation results. These metrics are necessary but not sufficient; they establish that the software works technically but do not establish that using it benefits patients.
Analytical validation does not exist in isolation; it connects to the broader software lifecycle standards that notified bodies assess within the technical documentation framework. IEC 62304 governs the software development lifecycle, establishing requirements for design verification and validation (V&V) that directly support analytical validation arguments.7IEC 62304:2006/AMD1:2015 (medical device software lifecycle); IEC 82304-1:2016 (health software products); ISO 14971:2019 (risk management). IEC 82304 addresses health software products more broadly, while ISO 14971 risk management outputs feed into clinical evidence adequacy by identifying which software functions require clinical validation and at what depth. Notified bodies assess clinical evidence within this broader technical documentation context, and the interaction between lifecycle documentation quality and clinical evidence scrutiny remains difficult to predict. Software lifecycle compliance does not substitute for clinical evidence, but the relationship between the two shapes how notified bodies evaluate the evidentiary package as a whole.
Clinical validation bridges the gap between analytical performance and patient benefit. An algorithm that correctly identifies a condition on imaging may not improve patient outcomes if clinicians ignore its output, if the condition would have been identified through standard care anyway, if identification leads to unnecessary interventions, or if real-world image quality differs from validation datasets. Clinical validation studies address these questions through prospective evaluation of software use in clinical settings, comparison against standard-of-care pathways, and measurement of clinically meaningful endpoints.
3. Can Real-World Evidence Replace Clinical Trials?
The role of real-world evidence in SaMD clinical evaluation has expanded significantly as regulators recognize that traditional clinical investigation designs may not capture software performance in actual use conditions. Real-world evidence (data derived from sources outside traditional clinical trials, including electronic health records, registries, insurance claims, and patient-generated data) can supplement or, in appropriate circumstances, substitute for prospective clinical investigation data.
The MDCG has addressed real-world evidence considerations for SaMD, acknowledging that software performance may vary between controlled validation environments and clinical deployment.8MDCG 2020-1 (n 2), s 4; real-world evidence considerations are discussed throughout the guidance document. An algorithm trained and validated on high-quality images from academic medical centers may perform differently when processing images from community hospitals with older equipment and variable imaging protocols. Real-world evidence can reveal performance gaps that controlled validation studies miss, but only if the evidence is collected systematically with appropriate quality controls.
The evidentiary value of real-world evidence depends heavily on data quality, completeness, and relevance to the intended use population. Registry data capturing software performance across diverse clinical settings may provide stronger evidence of generalizability than a controlled study at a single institution, but only if the registry captures outcomes with sufficient rigor and the data can be linked to software use with confidence. Whether registry-based RWE will be accepted depends on whether selection bias, confounding, missing data, and outcome ascertainment have been addressed at the methodological standard that clinical-investigation review applies, a threshold the regulation does not specify and that operates differently across notified bodies.
Post-market clinical follow-up under Annex XIV Part B MDR extends clinical evidence obligations beyond initial market authorization. Manufacturers must proactively collect and evaluate clinical data throughout the device lifecycle, confirming that clinical evidence remains adequate as the device is used in broader populations and as clinical practice evolves. For SaMD, this ongoing obligation has particular significance: software that learns from data may change its behavior over time, and clinical evidence supporting the initial version may not apply to evolved versions. The relationship between post-market clinical follow-up and AI/ML algorithm updates presents unresolved regulatory questions that manufacturers must address in their clinical evidence strategies (see Insight 12).
The evidentiary landscape is further complicated by regulatory expectations for RWE quality and the HTA dimension introduced by Regulation (EU) 2021/2282. As of October 2025, the MDCG continues to develop guidance on real-world data quality standards for clinical evaluation, refining expectations around data provenance, completeness, and fitness for regulatory purpose. Separately, the EU HTA Regulation (2021/2282), applicable from 12 January 2025 for joint clinical assessments of certain medicinal products and reaching specified high-risk medical devices and Class D IVDs through periodic Commission selection rather than a fixed commencement date, introduces a parallel evidentiary framework where RWE standards for health technology assessment may diverge from those applied in regulatory approval.9Regulation (EU) 2021/2282 on health technology assessment [2021] OJ L458/1; JCAs from 12 January 2025 (oncology medicinal products and ATMPs); Class III and implantable Class IIb medical devices and Class D IVDs enter by periodic Commission selection under Art. 7(4), not a fixed date. SaMD manufacturers seeking both market authorization and reimbursement face a dual-evidence burden: clinical evidence sufficient for MDR conformity assessment may not satisfy HTA bodies' requirements for comparative effectiveness and budget impact data. Whether evidence generation can be aligned across both frameworks from the outset, or whether two separate packages are needed, depends on data governance choices that intersect with privacy requirements in ways the regulations do not coordinate.
Germany's DiGA pathway offers the most developed European model for SaMD reimbursement and illustrates the practical interplay between regulatory and health-economic evidence requirements. Under the DVG and its implementing regulation (DiGAV),10Digitale-Versorgung-Gesetz (DVG) vom 9. Dezember 2019 (BGBl I S 2562); Digitale Gesundheitsanwendungen-Verordnung (DiGAV) vom 8. April 2020 (BGBl I S 768). manufacturers can obtain provisional listing in the DiGA directory on preliminary evidence, with a twelve-to-twenty-four-month window to generate confirmatory evidence of positive healthcare effects through controlled studies. The DiGA evidence requirements operate independently of MDR clinical evidence obligations (a device must satisfy both to achieve reimbursed market access), creating strategic complexity around evidence planning and study design. Belgium's applications mobiles de santé framework and France's emerging dispositifs médicaux numériques pathway signal that comparable reimbursement-linked evidence requirements will proliferate across European markets, raising parallel reimbursement-evidence questions that the MDR and HTAR do not coordinate.
4. Why No Study Design Clearly Fits SaMD
Clinical validation studies for SaMD require careful design to generate evidence that regulators will accept as demonstrating clinical benefit. The fundamental challenge lies in isolating the software's contribution to clinical outcomes from other variables: physician expertise, patient characteristics, healthcare system factors, and concurrent treatments all influence outcomes independently of whether the software is used. Study designs must control or account for these confounding factors while remaining practically feasible and ethically acceptable.
Randomized controlled trials represent the gold standard for clinical evidence but present practical challenges for SaMD. Randomizing patients to receive care with or without software assistance raises ethical questions when preliminary evidence suggests software benefit. Blinding is often impossible; clinicians know whether they are using the software. Crossover designs may be contaminated by learning effects as clinicians internalize software recommendations. Despite these challenges, randomized designs remain the most convincing evidence for regulatory purposes, particularly for higher-risk SaMD where clinical decisions have serious consequences.
Comparative effectiveness designs may provide acceptable evidence when randomization is impractical. Comparing outcomes between facilities that adopt the software and facilities that continue standard care can demonstrate clinical benefit if confounding factors are adequately controlled. Propensity score matching, difference-in-differences analysis, and synthetic control methods can strengthen causal inference from observational data, but these approaches require methodological sophistication and large sample sizes that many SaMD manufacturers cannot achieve.
As of 2025, regulators accept alternative study designs that reflect the practical realities of SaMD evaluation. Single-arm studies with external (historical) controls can generate acceptable evidence when randomized comparison is impractical, particularly where the comparator is no software assistance at all and clinical equipoise is limited. Basket and umbrella trial designs, originally developed for oncology therapeutics, are being adapted for SaMD contexts where a single algorithm addresses multiple clinical indications or where multiple algorithms target the same condition. The FDA's De Novo pathway has produced instructive precedents: the authorization of IDx-DR for autonomous diabetic retinopathy detection relied on a prospective single-arm study against a reading-center reference standard, establishing a regulatory template for AI-based diagnostic SaMD. These developments suggest a more nuanced regulatory landscape than a strict reading of traditional evidence hierarchies would imply.
Endpoint selection profoundly influences study feasibility and regulatory acceptability. Clinical endpoints (mortality, disease progression, quality-adjusted life years) provide the most compelling evidence of patient benefit but require long follow-up periods and large sample sizes. Surrogate endpoints (diagnostic accuracy, time to diagnosis, treatment initiation rates) can be measured more quickly but require validation linking the surrogate to clinical outcomes. Regulators may accept surrogate endpoints for lower-risk SaMD but require clinical endpoints for higher-risk classifications, particularly when the clinical benefit claim involves improved patient outcomes rather than merely improved diagnostic performance.
The role of the healthcare professional in SaMD use affects study design requirements. Software intended to provide information that clinicians independently verify before acting may require less robust clinical validation than software providing recommendations that clinicians are expected to follow. The MDCG 2020-1 guidance distinguishes these scenarios, but the boundary remains unclear in practice, and manufacturers' assumptions about clinician independence may not survive regulatory scrutiny or real-world observation of actual use patterns.11MDCG 2020-1 (n 2), s 4.4 (clinical performance considerations for MDSW).
5. Why Evidence Planning Fails When It Follows Product Design
Clinical evidence strategy for SaMD interacts with product conception in ways that are difficult to retrofit once development is complete. The evidence required for regulatory approval shapes product design decisions: intended use claims, target populations, clinical workflows, and performance thresholds all affect what evidence manufacturers must generate. A company that designs a product and then asks what evidence regulators require may discover that feasible evidence strategies do not support the intended claims, forcing either claim modification or evidence generation that delays market entry by years.
Classification Interactions
The interdependencies between intended use claims, evidentiary feasibility, and regulatory expectations create cascading constraints that resist sequential analysis. An intended use claim that appears straightforward may demand prospective clinical validation rather than analytical performance data alone, yet the feasibility of prospective validation depends on the available clinical literature establishing the foundational association the software exploits. Where that literature is thin, the manufacturer faces evidence generation at the foundational level before clinical validation can even be designed. These constraints compound further when real-world evidence enters the picture: its regulatory acceptability varies not only by classification and jurisdiction but by the specific notified body's interpretive approach to MDCG guidance, making general evidence planning unreliable as a predictor of conformity assessment outcomes.
Classification and clinical evidence interact in ways that defy straightforward optimization. The relationship between the classification a manufacturer asserts and the evidence scrutiny it receives is not linear: notified bodies assessing a contested Class IIa argument may apply greater evidentiary skepticism than they would to an uncontested Class IIb submission, even though the formal requirements differ. The result is that classification outcomes and evidence burdens cannot be analyzed independently, and the interaction between them varies by notified body, product type, and the maturity of guidance in the relevant clinical domain.
AI/ML and Regulatory Convergence
For SaMD incorporating artificial intelligence or machine learning, an additional regulatory layer compounds clinical evidence complexity. The EU AI Act (Regulation (EU) 2024/1689) classifies most medical device AI systems as high-risk, imposing data governance, transparency, and human oversight requirements that interact with, but do not replace, MDR clinical evidence obligations.12Regulation (EU) 2024/1689 (AI Act), OJ L, 2024/1689, 12.7.2024, Art. 27, 43(3), 72. Art. 43(3) AI Act integrates the AI Act's high-risk requirements into the MDR conformity assessment procedure, so that they are assessed as part of that single procedure, potentially streamlining compliance for manufacturers facing dual regulatory frameworks. Whether this integrated assessment mechanism will prove sufficient in practice, or whether notified bodies will effectively require parallel evidentiary submissions addressing each regulation's distinct requirements, remains an open question as implementing guidance develops (examined in detail in Insight 14).
The AI Act also introduces obligations with no direct MDR equivalent, though their reach into SaMD is narrower than a first reading suggests. The fundamental rights impact assessment under Art. 27 AI Act applies to deployers that are bodies governed by public law or private entities providing public services, but its scope is restricted to the high-risk AI systems referred to in Art. 6(2) (the Annex III stand-alone categories). Medical-device AI is high-risk under Art. 6(1) (Annex I product-safety legislation) and therefore generally falls outside Art. 27's reach; whether SaMD deployers in public-sector contexts nevertheless face parallel fundamental-rights review obligations under national or sectoral law is a separate question the AI Act does not foreclose. Post-market monitoring under Art. 72 AI Act imposes continuous performance surveillance requirements that parallel, but do not replace, MDR post-market surveillance obligations under Art. 83 and Annex III MDR. Whether a single post-market monitoring system can satisfy both frameworks or whether separate monitoring architectures are required varies by deployment context, and the resulting burden of maintaining compliance documentation for both regimes simultaneously creates complexity that neither the MDCG 2020-1 guidance nor the AI Act implementing measures fully address as of October 2025.
Across the Atlantic, the FDA's predetermined change control plan (PCCP) framework, finalized in December 2024, offers an alternative regulatory model for managing AI/ML algorithm evolution.13FDA, Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions (December 2024). The PCCP allows manufacturers to pre-specify anticipated modifications to algorithm performance, retraining methodology, or intended use within the original marketing submission, enabling pre-authorized updates without requiring new regulatory clearance for each change. The EU is moving toward a comparable approach, with guidance on predetermined change control plans for AI-enabled medical devices under development at the IMDRF level as of October 2025, expected to inform future EU guidance. For manufacturers targeting both markets, whether clinical evidence strategies can be aligned with both the PCCP framework and emerging EU change management requirements, or whether duplicative regulatory submissions become necessary, remains unsettled as EU guidance develops.
Pre-Market vs. Post-Market Evidence Strategy
The relationship between pre-market clinical evidence and post-market clinical follow-up deserves strategic attention. Manufacturers may achieve market authorization with limited clinical evidence if they commit to robust post-market data collection confirming clinical benefit in real-world use. This approach accelerates market entry but creates ongoing obligations and risks: if post-market data reveals inadequate clinical performance, regulatory consequences may include market withdrawal. Whether to pursue accelerated market entry with post-market evidence commitments or to invest in more comprehensive pre-market evidence depends on competitive dynamics, capital availability, and risk tolerance that vary by manufacturer and market context.