The NICE HST process (3/3) – challenges associated with appraising treatments for ultra-rare indications

Nov 2, 2019

Written by Chris Hellmund (Senior Medical Writer) and Neil Webb (Head of Systematic Review)

A poster on this topic was presented at ISPOR Europe 2019. This blog post provides an overview of our findings.The abstract is available here.

In previous blog posts, we explored the origins of NICE’s Highly Specialised Technologies (HST) process and compared it with the Single Technology Assessment (STA) process.

In this blog, we review the first 10 HST appraisals and identify key issues raised by the Evidence Review Groups (ERGs).

The indications evaluated under the HST process are often associated with unavoidable limitations, such as small trial sample sizes, and limited quality of life and natural history data. However, identification of avoidable issues would enable us to develop recommendations for manufacturers completing future submissions.

Evidence challenges – SLR

Key issues in the reporting of SLRs were generally avoidable. In some submissions, search strategies were not comprehensive, for example because no proper attempt had been made to search for comparators, and not all terms and synonyms for the intervention had been included. Some submissions lacked transparency in reporting, and others failed to present the flow of studies through the SLR in a consistent and logical manner.

Recommendations – systematic literature reviews

SLR strategies should be comprehensive, and SLRs should be reported in a fully transparent manner, allowing the methods to be reproduced.

Evidence challenges – clinical evidence

Some criticisms of the clinical evidence presented may be considered unavoidable in the context of HSTs. For example, some trials were open label or lacked comparator arms. For novel technologies with innovative modes of action it may be neither possible nor ethical to use a placebo, and comparators may not exist. Additional unavoidable issues included lack of clarity about clinical relevance thresholds – if novel endpoints are used, established thresholds may not exist.

Potentially avoidable issues identified included statistical analyses being conducted differently for different trial outcomes, and inappropriate comparison of trial results with historical control data. Naïve comparisons with historical data were criticised due to the evolution of standard of care over time, and differences in inclusion criteria between studies.

Recommendations – clinical evidence

  1. Appropriate statistical methods should be used when comparing single-arm trial data against historical control data.
  2. Novel endpoints should be fully justified, and well-established secondary endpoints should be used in conjunction with these.
Evidence challenges – economic evidence

In 3/10 submissions the approach to generating utility values raised concerns. In one submission these related to the use of clinical experts rather than patients (as per the NICE reference case) for deriving utility values. In another, the ERG had concerns about the specification of a statistical model which was fitted to EQ-5D-5L data in the pivotal clinical trial. The model permitted impossible values (i.e. utility values above 1), and the ERG believed that it didn’t properly take into account the distribution of the underlying data. In the same submission, the ERG also had concerns about the assumption that patients in the best supportive care arm experienced considerable reductions in health related quality of life.

Given the potential for more frequent use of the 1.5% discount rate in HST submissions, it was of interest to assess how frequently ERGs accepted this approach. In line with the NICE Guide to the Methods of Technology Appraisal, interim HST guidance states that discount rates of 1.5% may be used if:

  • treatment restores people to full or near full health when they would otherwise die or have a very severely impaired life, and this is sustained over a very long period (normally at least 30 years); and
  • the introduction of the technology does not commit the NHS to significant irrecoverable costs [1].

Most submissions (6/10) used this 1.5% discount rate in the company base-case, with the remainder either using a 3.5% rate (3/10) or differential discounting of 3.5% for costs and 1.5% for outcomes (1/10).

The ERG agreed with the use of the 3.5% rate. However, in 4/6 cases where a 1.5% rate was used, the ERG felt the conditions for use of the lower discount rate had not been met and/or the ERG’s interpretation of NICE guidance was that the 1.5% rate should be presented as a scenario only, highlighting ambiguity in the interim HST guidance.

The use of differential discounting was rejected by the ERG in the one submission where it was used; the ERG stated that it is not considered appropriate, and both discount rates were set to 3.5%.

Figure 1: Approach to discounting across appraisals

Recommendations – economic evidence

  1. The use of 1.5% discount rates should be carefully justified, and a scenario should be included assuming discount rates of 3.5%.
  2. De novo utility studies should align as closely as possible with NICE’s reference case if appropriate utility values are not available from trials or the literature.
Conclusions

By identifying key issues raised by ERGs and developing recommendations to mitigate these risks in future submissions, we hope that these findings can be of use to manufacturers making future HST submissions to NICE.


More Insights

NICE refines the HST routing criteria: What you need to know

NICE refines the HST routing criteria: What you need to know

Written by Brian O'Toole   The National Institute for Health and Care Excellence (NICE) Highly Specialised Technologies (HST) programme is designed to evaluate innovative treatments for rare and complex conditions. On 19th March ... Read more

Systematic review case study: scoping research to identify comparators

Systematic review case study: scoping research to identify comparators

Written by Sneha Bhadti and David Pritchett   Establishing a list of relevant comparators at the protocol development stage is a vital step in the systematic literature review (SLR) process. Scoping research during this stage can help ... Read more

National Numeracy Day 2025: Why money and maths matter in health economics

National Numeracy Day 2025: Why money and maths matter in health economics

Written by Abby Paine and Dom Partridge This is our third year celebrating National Numeracy Day, and this year’s theme of money is especially relevant for those of us working in health economics and market access. In previous years, ... Read more

Maximising SLRs for HEOR – why PICO matters

Maximising SLRs for HEOR – why PICO matters

Written by Susan O’Connell and Sara Bartlome   Introduction Systematic literature reviews (SLRs) seek to answer a specific research question by translating the question into search terms and developing a literature search strategy to ... Read more