Securing quality RWE in AIT

RWE is an important, complementary source of evidence to randomised clinical trials (RCTs). Within AIT, relevant questions which can be best explored using real world study (RWS) designs are: whether results from RCTs are generalisable to broader patient populations treated in clinical practice, whether benefits from AIT are sustained over longer-time horizons, and whether there are other benefits to patients, which are difficult to measure in RCTs. Despite the acknowledgement that RWSs are relevant, only a limited number have been conducted to date within AIT.

Content in table adapted from Fritzsching et al (JACI 2022)

But how can studies be conducted to provide good quality and reliable evidence? The recently published article ‘Real-world evidence: Methods for assessing long-term health and effectiveness of allergy immunotherapy’ in the Journal of Allergy and Clinical Immunology (JACI) attempts to help its readers to identify the caveats relating to RWSs, and understand how the quality of RWS data can be improved by mirroring certain methodologic steps that are used in RCTs, and based on existing guidelines within RWE.

What data should be used to conduct RWSs?

A central first step is to identify the most appropriate data source for an RWS. While RCTs always use prospectively collected data, RWSs can use both retrospective data sources (e.g., registries, prescription databases, insurance claims databases), or can collect data prospectively (e.g., prospective observational cohort studies, pragmatic trials, non-interventional trials). Within AIT, retrospective data sources can be of value, as they allow for the investigation of broader patient populations and longer time horizons than are typically collected in prospective studies. However, when using retrospective data, there is a need to pay special attention to the quality of the data and to consider this in the study design.

The 5 key steps to ensuring high quality when generating RWE

Conducting a high-quality and scientifically reliable RWS requires rigorous methodology, mirroring that for RCTs. The table below shows the similarities in methodologies for both RCTs and RWSs.

Content in table adapted from Fritzsching et al (JACI 2022).

The 5 key steps below elaborate further on how a good quality RWS can be conducted:

Step 1: Pre-specify the study and register it publicly

RWSs are sometimes criticised as ‘fishing’ for outcomes that are of interest, since the data is available in the databases. To avoid this accusation, pre-specification and transparency around the study design, and analyses prior to execution, are important first steps. These can be achieved by pre-specifying the study design and statistical analysis plan in a protocol and publishing it, and/or pre-registering the study, via a public registry (clinicaltrials.gov or similar), just as for RCTs.

Step 2: Apply methods that make groups more comparable and help avoid confounding

One of the key differences between RWSs and RCTs is that RWSs are not based on randomisation. This means that there is a much greater potential of bias and confounding than in RCTs. A common cause of bias is confounding by indication and/or disease severity. For example, in AIT studies, patients with allergic rhinitis who are treated with AIT are likely to have more severe allergic rhinitis versus patients who are not treated with AIT. The more different the compared groups are at baseline, the higher the risk of confounding. However, there are methods which can minimise the risk of confounding by mimicking what randomisation does in RCTs: for example, matching designs, such as propensity score-matching, or instrumental variable techniques.

Step 3: Predefine outcomes which are measured in a valid way and reported transparently

In RWSs that use retrospective data, it is often necessary to use proxies to measure effectiveness or disease severity. For example, in the case of AIT studies, prescription medications for allergic rhinitis (AR) and asthma, as well as confirmed diagnoses, can be used as proxies for disease severity in the absence of symptom scores. In cases where proxies are used, it is important to define outcomes clearly and ensure they are valid.

Step 4: Discuss results adequately and put them into perspective in relation to existing research

The results from RWSs should be discussed in light of existing evidence, and should ideally replicate what has been seen in RCTs with similar objectives and outcomes. It is also important to describe any limitations, including potential biases and confounding factors, following a discussion on how these may influence the results.

Step 5: Be transparent about conflicts of interest

Any conflict of interest must be clearly stated. There are ways to mitigate conflicts of interest, for example, through the involvement of third parties in the design, conduct, and analysis of the study.

The REACT (REAl world effeCtiveness in allergy immunotherapy) study was designed using the above key steps. Learn more about the REACT study here.