Use of RWE in the reimbursement assessment of medical devices

Nov 30, 2021

Amy Crompton (Systematic Review Analyst) and Tom Macmillan (Consultant - Systematic Review)

Hierarchy of evidence 

Different study designs are often ranked in a hierarchy of evidence based on their validity and robustness. Randomised controlled trials (RCTs) are typically considered the gold standard source of evidence, however systematic literature reviews and meta-analyses sometimes appear at the top of the hierarchy. 

Randomised controlled trials of medical devices 

Randomised controlled trials are often resource-intensive, expensive, and may not always be the most appropriate study design (1). An RCT can take several years to complete, at which point another medical device may have been proven more effective (2). Additionally, it may be unethical to assign patients to an untreated control group, and blinding is not always possible in the assessment of medical devices. The evidence for medical devices is therefore often limited to non-randomised studies only.  

Value and application of real-world evidence 

Real-world evidence (RWE) can provide valuable insights into the benefits and negative sequalae of healthcare interventions. Real-world clinical and economic data, patient-reported outcomes, and health-related quality of life data (3) can support both the clinical and economic evidence base for medical devices. 

“The popular belief that only RCTs produce trustworthy results and that observational studies are misleading does a disservice to patient care, clinical investigation, and the education of health care professionals” – John Concato, M.D., M.P.H. (4). 

The value of RWE is increasingly being recognised by health technology assessment (HTA) agencies. For example, HTA agencies in Australia, France and Scotland (5-7) recommend that RCT evidence should be utilised to support assessment for reimbursement, but also note that observational and non-randomised studies may be used to support this evidence. The National Institute for Health and Care Excellence (NICE) guidance for medical device reimbursement in England does not currently state a preference for a particular study design (8). However, the NICE methods and processes for health technology evaluation programmes are being reviewed (consultation ended on the 13th of October 2021) and are expected to place a greater emphasis on RWE (9). This will be discussed further in part two of this blog series.  

The currently preferred and permitted evidence types for four countries are summarised in Table 1 

Table 1. Preferred and permitted study designs for the reimbursement of medical devices 

Country (agency)  Preferred study design  Permitted study designs 
 Australia   (MSAC)   RCT   Pseudo-randomised trial, comparative cohort study (concurrent controls, historical or self-control), case series (pre-test/post-test outcomes) 

 Study designs per NHMRC hierarchy (10) 

 England (NICE)   Not reported   Directly observed clinical outcomes, non-clinical studies (e.g. in vitro), evidence  syntheses, modelling studies 
 France (HAS)  Double-blind RCT  Non-randomised or non-blinded trials, patient preference cohort, prospective comparative observational study, propensity score matched cohort 

Use of non-randomised study designs must be justified 

 Scotland  (SHTG)  RCT
(per IDEAL Framework) 
Lived experiences, RWD, systematic reviews 













†No formalised methods or process guideline was identified; study design requirements have been inferred from published assessments. 

Abbreviations: HAS, Haute Autorité de Santé; MSAC, Medical Services Advisory Committee; NHMRC, National Health and Medical Research Council; NICE, National Institute for Health and Care Excellence; RCT, randomised controlled trial; RWD, real-world data; SHTG, Scottish Health Technologies Group. 


Real-world evidence is increasingly being recognised as a valuable source of evidence for medical interventions and is accepted by a number HTA agencies. Observational cohort studies may have higher external validity than RCTs, increasing the likelihood that results can be extrapolated to the population of interest (1). However, the internal validity is lower, as bias and confounding variables are harder to eliminate (11). Therefore, RWE should be analysed and interpreted with caution (12) and may be most useful in support of RCT evidence. In the assessment of medical devices, RCTs may not always be the most suitable study design, while RWE forms a feasible alternative for evaluation. 

If you would like to learn more about systematic literature reviews, please contact Source Health Economics, an independent consultancy specialising in evidence generation, health economics, and communication. 


  1. Bondemark L, Ruf S. Randomized controlled trial: the gold standard or an unobtainable fallacy? European Journal of Orthodontics. 2015;37(5):457-61.
  2. Neugebauer EAM, Rath A, Antoine S-L, Eikermann M, Seidel D, Koenen C, et al. Specific barriers to the conduct of randomised clinical trials on medical devices. Trials. 2017;18(1):427.
  3. RWE navigator. Real-world evidence. Available at:
  4. Concato J, Shah N, Horwitz RI. Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs. New England Journal of Medicine. 2000;342(25):1887-92.
  5. Medical Services Advisory Committee (AU). Guidelines for preparing assessments for the Medical Services Advisory Committee. 2021.
  6. HAS. Évaluation des dispositifs médicaux – Principes d’évaluation de la CNEDiMTS relatifs aux dispositifs médicaux à usage individuel en vue de leur accès au remboursement. 2019(Available online:
  7. SHTG. What we do. (Available online:
  8. National Institute for Health and Care Excellence. Medical technologies evaluation programme methods guide 2017 [Available from:
  9. National Institute for Health and Care Excellence. Review of methods for health technology evaluation programmes: proposals for change 2021 [Available from:
  10. National Health and Medical Research Council (AU). NHMRC levels of evidence and grades for recommendations for developers of guidelines. 2009.
  11. Carlson MD, Morrison RS. Study design, precision, and validity in observational studies. J Palliat Med. 2009;12(1):77-82.
  12. Ioannidis JPA, Haidich A-B, Pappa M, Pantazis N, Kokori SI, Tektonidou MG, et al. Comparison of Evidence of Treatment Effects in Randomized and Nonrandomized Studies. JAMA. 2001;286(7):821-30.


More Insights

Modelling requirements for HTA in Europe’s big 5 economies

Modelling requirements for HTA in Europe’s big 5 economies

Written by Oliver Burn, Consultant Health Economist, and Margaret Petrou, Consultant Health Economist Adapting a global economic model for Health Technology Assessment (HTA) submissions in several countries can be challenging, as requirements ... Read more

Advances in artificial intelligence: ChatGPT and the implications for medical writing

Advances in artificial intelligence: ChatGPT and the implications for medical writing

Written by Dom Partridge, Medical Writer As artificial intelligence (AI) technology continues to advance, natural language processing (NLP) models like ChatGPT have gained a lot of attention for their ability to generate human-like text. ... Read more

Obtaining a NICE recommendation for your medical device: a view from the external assessors

Obtaining a NICE recommendation for your medical device: a view from the external assessors

Written by Mark Pennington, Consultant Health Economist A National Institute for Health and Care Excellence (NICE) recommendation can open the door to widespread adoption for a new medical device. But does getting a NICE recommendation require ... Read more

10 steps for a successful global economic model

10 steps for a successful global economic model

Written by Beth Hancock, Senior Consultant Health Economist Pharmaceutical companies often develop global cost-effectiveness and budget impact models, with the aim of adapting them for specific countries. In principle, this is an efficient and ... Read more