Securing quality RWE in AIT – alk

There is a need for more, and better-quality, real world evidence in allergy immunotherapy – but how can data quality be improved?

Recently, a number of professional and scientific societies, including the European Academy of Allergy and Clinical Immunology (EAACI), have raised a call for action to generate more and better-quality real world evidence (RWE) in the field of allergy immunotherapy (AIT). But when is it most appropriate to generate evidence using real world data, how can studies be conducted to generate reliable results, and how can we identify which studies have been conducted in a rigorous, scientific way?

RWE is an important, complementary source of evidence to randomised clinical trials (RCTs). Within AIT, relevant questions which can be best explored using real world study (RWS) designs are: whether results from RCTs are generalisable to broader patient populations treated in clinical practice, whether benefits from AIT are sustained over longer-time horizons, and whether there are other benefits to patients, which are difficult to measure in RCTs. Despite the acknowledgement that RWSs are relevant, only a limited number have been conducted to date within AIT.

Content in table adapted from Fritzsching et al (JACI 2022)


But how can studies be conducted to provide good quality and reliable evidence? The recently published article ‘Real-world evidence: Methods for assessing long-term health and effectiveness of allergy immunotherapy’ in the Journal of Allergy and Clinical Immunology (JACI) attempts to help its readers to identify the caveats relating to RWSs, and understand how the quality of RWS data can be improved by mirroring certain methodologic steps that are used in RCTs, and based on existing guidelines within RWE.

What data should be used to conduct RWSs?

A central first step is to identify the most appropriate data source for an RWS. While RCTs always use prospectively collected data, RWSs can use both retrospective data sources (e.g., registries, prescription databases, insurance claims databases), or can collect data prospectively (e.g., prospective observational cohort studies, pragmatic trials, non-interventional trials). Within AIT, retrospective data sources can be of value, as they allow for the investigation of broader patient populations and longer time horizons than are typically collected in prospective studies. However, when using retrospective data, there is a need to pay special attention to the quality of the data and to consider this in the study design.

The 5 key steps to ensuring high quality when generating RWE

Conducting a high-quality and scientifically reliable RWS requires rigorous methodology, mirroring that for RCTs. The table below shows the similarities in methodologies for both RCTs and RWSs.

Content in table adapted from Fritzsching et al (JACI 2022).

The 5 key steps below elaborate further on how a good quality RWS can be conducted:

Step 1: Pre-specify the study and register it publicly

RWSs are sometimes criticised as ‘fishing’ for outcomes that are of interest, since the data is available in the databases. To avoid this accusation, pre-specification and transparency around the study design, and analyses prior to execution, are important first steps. These can be achieved by pre-specifying the study design and statistical analysis plan in a protocol and publishing it, and/or pre-registering the study, via a public registry ( or similar), just as for RCTs.


Step 2: Apply methods that make groups more comparable and help avoid confounding

One of the key differences between RWSs and RCTs is that RWSs are not based on randomisation. This means that there is a much greater potential of bias and confounding than in RCTs. A common cause of bias is confounding by indication and/or disease severity. For example, in AIT studies, patients with allergic rhinitis who are treated with AIT are likely to have more severe allergic rhinitis versus patients who are not treated with AIT. The more different the compared groups are at baseline, the higher the risk of confounding. However, there are methods which can minimise the risk of confounding by mimicking what randomisation does in RCTs: for example, matching designs, such as propensity score-matching, or instrumental variable techniques.


Step 3: Predefine outcomes which are measured in a valid way and reported transparently

In RWSs that use retrospective data, it is often necessary to use proxies to measure effectiveness or disease severity. For example, in the case of AIT studies, prescription medications for allergic rhinitis (AR) and asthma, as well as confirmed diagnoses, can be used as proxies for disease severity in the absence of symptom scores. In cases where proxies are used, it is important to define outcomes clearly and ensure they are valid.


Step 4: Discuss results adequately and put them into perspective in relation to existing research

The results from RWSs should be discussed in light of existing evidence, and should ideally replicate what has been seen in RCTs with similar objectives and outcomes. It is also important to describe any limitations, including potential biases and confounding factors, following a discussion on how these may influence the results.


Step 5: Be transparent about conflicts of interest

Any conflict of interest must be clearly stated. There are ways to mitigate conflicts of interest, for example, through the involvement of third parties in the design, conduct, and analysis of the study.


The REACT (REAl world effeCtiveness in allergy immunotherapy) study was designed using the above key steps. Learn more about the REACT study here.

FDA: Submitting documents using real-world data and real-world evidence to FDA for drugs and biologics guidance for industry May 2019. Available at: Date accessed: February 24, 2022
Paoletti al.: Allergen immunotherapy: the growing role of observational and randomized trial “Real-World Evidence”. Allergy. 2021; 76: 2663-2672
Roche N. et al.: The importance of real-life research in respiratory medicine: manifesto of the Respiratory Effectiveness Group: endorsed by the International Primary Care Respiratory Group and the World Allergy Organization. Eur Respir J. 2019; 5419011511
Fritzsching et al.: Real-world evidence: Methods for assessing long-term health and effectiveness of allergy immunotherapy. JACI 2022 vol 149, 3:881-883.
Berger et al. Good Research Practices for Comparative Effectiveness Research: Defining, Reporting and Interpreting Nonrandomized Studies of Treatment Effects Using Secondary Data Sources: The ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report—Part I. Value in Health, 2009, 12 (8): 1044–1052
von Elm E. et al: The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. PLoS Med. 2007; 4: e296