Popular Searches:

    The Real-World Evidence Glossary

    Last updated: April 11, 2022

    Composite mortality variable
    Data element that determines vital status and date of death (DOD) in Flatiron Health databases by aggregating several mortality surveillance sources (all cause mortality): electronic health record (EHR) structured information, EHR unstructured information, commercial obituary data and the Social Security Death Index (SSDI). As of 2022, this variable has been benchmarked against the National Death Index (NDI) across 18 different diseases in oncology.
    Key Resources:

    Real-world data and the case of the missing deaths

    Contemporaneous External Control
    When there is alignment of patient cohort entry time with the time period of the trial to construct the external control comparator cohort.

    Comparator cohort

    Analysis aimed to determine causal inference from comparing cohorts generated from large observational databases to recreate a randomized experiment—the target experiment or target clinical trial—that would answer a question of interest. Current proposed framework assumes specific analytic steps aligned with target trial procedures.
    External control cohort
    Patient cohort not enrolled in a clinical trial and used for analyses whose results, at some level, are intended as reference or to provide context to the results of analyses performed on a study arm accrued to a clinical trial.

    External comparator cohort, Real-world external comparator

    Key Resources:

    Guidance for Industry: E 10 Choice of Control Group and Related Issues in Clinical Trials

    External validity
    Determination of whether or not the observations and conclusions of a study (by causal inferences or other) are applicable or generalizable to different measures, patients, settings, and times.
    The process, carried out via machine-learning (ML)-based methods, of pulling out information documented in unstructured sources within the EHR, with the purpose of translating that information into a data model
    Face Validity
    Based on a subjective assessment, determination of whether or not a study or test measures what it is supposed to measure and appears reasonable. First step (also weakest) to seek agreement/alignment with clinical/regulatory stakeholders on discovery or research and development (R&D) work. Note: Face validity is not interchangeable with: external validity, regulatory-grade validation. These terms have other appropriate uses, but they are not equivalent.
    Hybrid clinical trial
    Clinical study where a randomized trial has its control arm supplemented/augmented by patients sourced externally, for example EHR-derived RWD patients, that closely follows the prospective eligibility specifications.

    Hybrid-controlled clinical trial

    Integrated evidence
    Evidence resulting from the combined analysis of multiple sources of data with the intent of being more robust than any of its component data sources.
    Key Resources:

    Beyond real-world evidence: Integrated evidence offers a new perspective

    Internal validity
    Determination of the quality in the conduct of a study and whether or not the intended interpretation of a study (be causal inference or other) is justified based on the specific circumstances of the study (such as presence of confounders, biases, missing information or misclassifications). For instance, studies that use highly representative cohorts could potentially have high external validity, or generalizability. However, if the data have a high degree of incompleteness, or the methods used are not appropriate to the research goal, they may end up being inconclusive due to lack of internal validity.
    Natural history cohort
    Patient cohort receiving standard of care and/or emergent care whose outcomes are studied for the purpose of characterizing the average/expected course of a given disease under off-protocol/standard-of-care management.
    Real-world adverse event
    Any unfavorable and unintended sign, symptom, or disease temporally associated with a treatment/intervention that is documented during routine care without judgment as to causality.


    Real-world best response
    The best response (real-world complete response > real-world partial response > real-world stable disease > real-world progressive disease, from best to worst) during the index line of interest. Note: Some studies may require confirmation of the best response on a subsequent evaluation before categorizing a patient’s best response.


    Real-world Complete Response Rate
    Real-world response category corresponding with proportion of patients on a given treatment that have at least one documented complete response (CR) assessment determination during the course of treatment by line of therapy (LOT). Note: Some studies may require confirmation of complete response on a subsequent evaluation before categorizing a patient as having a complete response,


    Real-world data
    Data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources. (Examples include: electronic health records (EHRs); medical claims and billing data; data from product and disease registries; patient-generated data, including from in-home-use settings; and data gathered from other sources that can inform on health status, such as mobile devices.)


    Key Resources:

    Framework for FDA's Real-World Evidence Program

    Real-world disease control rate
    Proportion of patients on a given treatment that have at least one real-world partial response (rwPR), real-world complete response (rwCR), or real-world stable disease (rwSD) assessment determination during the course of treatment by LOT. Note: Some studies may require confirmation of disease control (as defined above) on a subsequent evaluation before categorizing a patient as having disease control.


    Real-world duration of response
    The time from the date of the first real-world complete response (rwCR) or real-world partial response (rwPR) after a therapy starts to the date of the first subsequent real-world progressive disease (rwPD) (or death, depending on the research question), within the LOT. Note: Patients for whom there are no rwPD determinations (or no death, if death is included as an event) within a LOT are censored at the last known response assessment determination that is rwCR, rwPR, or rwSD within the LOT. There may be adjustments to this general analytic guidance in disease settings with standards of care that include maintenance therapy; always consult the analytic plan specifics for granular censoring rules in a given study.


    Real-world endpoints
    Precisely defined variables intended to reflect outcomes of interest that are statistically analyzed to address a particular research question. (Examples include: real-world overall survival (rwOS); real-world progression-free survival (rwPFS); and rwRR (real-world response rate))

    Real-world outcomes

    Key Resources:

    BEST (Biomarkers, EndpointS, and other Tools) Resource

    Real-world evidence
    The clinical evidence about the usage and potential benefits or risks of a medical product derived from analysis of RWD.


    Key Resources:

    Framework for FDA's Real-World Evidence Program

    Real-world indeterminate response
    Real-world response category corresponding to patients with explicit documentation of the inability to make a response assessment determination.
    Real-World Overall Survival
    Time from index date (e.g., advanced diagnosis or the start of a LOT) to date of death as determined using the composite mortality variable. Patients without a date of death are censored at the last confirmed activity date.


    Real-world Partial Response
    Real-world response category corresponding to patients with documented decrease in disease burden, though disease is still present. Partial reduction in tumor burden in some or all areas without any areas of increasing disease.


    Real-world Progression-Free Survival
    Defined as the time from index date (e.g., start date for the desired LOT) to either the date of progression or death (as per the composite mortality variable) using the rwP variable.* Patients without progression or a death event are censored at the last clinic note date.


    Real-world Progressive Disease
    Real-world response category corresponding to patients with documented increase in disease and/or presence of any new lesions. This is a slightly different definition than progressive disease abstracted using rwP forms.


    Real-world Pseudoprogression
    Real-world response category corresponding to patients with documentation of pseudoprogression or related terminology (e.g., tumor flare) with regard to the response scan in the setting of an immunotherapy.
    Real-world response
    Clinician assessment of change in disease burden following radiographic imaging during a LOT (not necessarily following any standard rule). This approach is based on RWD documented for clinical purposes, and response assessment categories are abstracted based on the clinician’s qualitative description of response to therapy. Note: the salient difference between this concept and rwP is that real-world response assessments are tied to radiographic imaging evaluations, while that is not a requirement for rwP. Note: rwR and RECIST-based response are separate instruments that measure different concepts and therefore, Flatiron Health strongly recommends not conducting any direct statistical comparisons (e.g. differences in rates, odds ratio, p-values) between rwR-based and RECIST-based endpoints in relevant use cases such as real-world control arms.


    Real-world response rate
    rwRR is defined as the proportion of patients with a real-world complete or a real-world partial response, based on the treating clinician’s assessment of change in disease burden following radiology assessment (ie, the rwR variable categories). Use: strongly preferred term for analyses based on the rwR variable, due to the major conceptual differences between the rwR variable and response defined by RECIST. It is a matter of integrity to ensure that there is never an implication of equivalence between these variables and endpoints.


    Real-world Stable Disease
    Real-world response category corresponding to patients with no documented change in overall disease burden, relative to preceding assessment. rwSD is also used to capture mixed response (some lesions increased, some lesions decreased).


    Real-world time-to-next treatment
    Defined as the time from index date to initiation of the LOT subsequent to the index therapy, or date of death (DOD) for patients who do not have a subsequent therapy. Note: Patients without a subsequent therapy or DOD are censored at their last confirmed EHR activity date. Note: For censoring and treatment eligibility rules, refer to the specifications of individual studies.


    Real-world time-to-progression
    Defined as the time from an index date (e.g., start date for the desired LOT) to the date of real-world progression where death is not considered an event.


    Real-world time-to-treatment- discontinuation
    Defined as the time from the index date to discontinuation for any reason. Note: For censoring rules, refer to the specifications of individual studies.


    Study-specific Cohort
    Cohort of patients to be analyzed in a given specific study, and selected according to eligibility criteria established for that purpose.