Using EHR-derived real-world data for HTA decision-making and market access

September 23, 2020

 

is becoming a critical component of Health Technology Appraisals (HTAs) worldwide, particularly in oncology. However, questions remain about how it can be used effectively to support market access. In this video, Dr. Scott Ramsey discussed the current evidence gaps in HTAs and how can address those gaps and Dr. Dominik Heinzmann walks attendees through a practical use case of RWE for HTAs.

Jump to transcript

Questions from the audience

How do HTA bodies view real world data and submissions?

Scott Ramsey: Yeah. Happy to take a first shot at that. And I can speak mostly from my perspective as a decision maker on US pharmacy and therapeutics committees, which review these evidence dossiers, but I do have a number of colleagues who work overseas and have relayed to me their experiences. It's a hard question to answer because it varies so much depending on the particular submission and the particular product that's being considered and where it is in the life cycle. I would say the use, as a rule of thumb, the use of real-world data and the acceptance of real-world data is inversely related to the quality, if you will, or completeness of the data that are available at the time of submission.

So for example, with one arm trials, trials with nonstandard comparators, or even the more recent basket trials, which genomically defined treatment regimens where patients have handfuls of patients in particular cancer subtypes are used then the decision makers in HTA are often very frustrated by that clinical data and they're hungry for new data sources to help understand how those products are working in the real world.

So in those cases, data like Flatiron type data, particularly the linked Flatiron data with the FMI clinical genomics database, can really provide a fair amount of influence to help those decision makers. I think Dominik gave a great example of how extrapolated data can be used to inform a decision that NICE made because they were not satisfied with the data that were available at the time that that product was launched.

But if you have a trial on the other hand, or if you have a product that has phase three trials with extended data points with randomization, those are going to be looked at more carefully, and be given much more strength than real-world data. So, as I said, it's hard to generalize on when their most... real-world data is most accepted, but it is generally related to the quality of evidence at submission.

How do HTA bodies view real-world data and submissions?

Dominik Heinzmann: I think in general they are very welcome. If it's about supplemental data, which helps to close certain evidence gaps. I see there's a big support to use them. They require a very transparent communication about elements like data quality, exact sources of biases you may have with the data source. So you have addressed it. So once you have transparency, I feel real-world data is really welcomed by health authorities that supplement to evidence together with some clinical data definitely, and most of the preference is for some sort of randomized data.

When it comes to supplement, some of the clinical data we've seen before heard from Scott in certain instance where we have a single arm trial, and situation gets a little bit more complex. So it really depends on the exact situation where you can use for example, real-world data to generate an external control arm to conceptualize single arm data.

There are different considerations in which case it may suit well but in certain situations it may be a little bit challenging to accept this as some final evidence to make a decision for reimbursement. So in general, I think there's really a lot of learning and a lot of welcome culture for real-world data, as long as the communication is very transparent. However on a deeper and knowledge about really the sources of biases, how you address them, all this more methodological aspect, I think there's still a lot ongoing. And what I see is really great collaborations between all the stakeholders in this ecosystem to get the ways to really make real-world data as part of the submission dossier at the end really help the patients to get access to effective and safe treatment.

Given that HTA is a decentralized process, and there's different HTA bodies in each country, how do these countries or markets differ in their assessment of drugs, and then also their acceptance of real-world data?

Dominik Heinzmann: I think to start really historically it's probably not very different to what we've seen years ago in the regulatory landscape if it goes about new treatment modalities. You see a certain initially heterogeneous assessment all with the really same principals to assess properly the molecule or the health technology to have it really safe, effective for the population on the regulatory side.

I think similar was that HTA that you see currently still a little bit of heterogeneity, and in hope, the weight the evidence generated by real-world data. But I think there's also a greater for ongoing different associations really, and bringing the different stakeholders together, have very productive discussions. OSC is seeing a convergence of all these concepts, but we get closer and closer to really quantify what is meaningful in real world data, this includes amount, quality, or other attributes as well.

So I think it comes together. Currently we need to live with this slight and be creative among the assessments, but the thing at the end of the day, I think everybody, the underlying principles in this assessments are very similar. So even if you see some heterogeneity at the end, fundamentals are very similar between the HTA, all this and payer assessments.

What is the acceptability of US EHR data in ex-US HTA submissions? And the question also asks if there's anything specific to Asian markets that should be considered.

Scott Ramsey: Well yes, regarding acceptability, I think there is growing acceptability of US data. And I have to admit my only experience with that is with HTA bodies. I don't have knowledge of what's happened in Asian markets. But I know for example, that Flatiron data has been used in multiple submissions to NICE, has been brought to IQWiG in Germany, and other countries in Europe, and has been influential in making decisions. And the reason I think is twofold. One is the thing I mentioned in my talk that there's usually a lag when the European regulators look at real-world data. So they can go to the US which has often adopted the products earlier and more aggressively to really understand how the products are performing.

And then the other is, most European countries are much smaller than the US, and they don't have the sufficient numbers for some of these products that are used in rarer tumor types. And then the US simply by its size and numbers can be very useful. The question that always comes up though is, is the US patient and the US healthcare system representative of the experience in Europe or Asia or elsewhere.

And the short answer of course is that it's not, but in cases where you have no data in your country, it's a bit of an art, but you have to decide in which situations what's happening in the US can be translated to the different country. So things like overall survival can be useful, safety signals from real-world data can be very useful. So I think there will be a use in all countries of US data by its size, its availability, to answer questions that just won't be there. But it varies from country to country.

With the complexities and heterogeneous payers that we have in the US, do you think that the US will ever have an HTA body?

Scott Ramsey:Well the US actually has multiple HTA groups. They are all private, a great example is ICER. But there's a number of other groups that perform health technology assessments. And there actually are standards for submitting HTA dossiers, particularly by the AMCP Academy of Managed Care Pharmacy, that almost all health plans, commercial health plans in the US adhere to. So HTA is used and viewed by all health plans.

Now, is it used the way it is used in Europe and Asia? No, we haven't gotten to the point, particularly where cost effectiveness is used to influence decisions that other countries have gotten to. But times will change, there's ever-growing pressure to control costs in healthcare. And the US has tried almost every other solution besides cost effectiveness, and someday it may figure out that they have to use cost effectiveness to manage their spend trend. I can't predict when that's going to happen, but I do think you're going to see more use of things that focus on value, and real-world data is going to be a part of that.

Can you share some thoughts on how to deal with selection bias when using real-world data?

Dominik Heinzmann:Of course, I think that's an important consideration. Even selection bias is some heterogeneous, depending really what caused this selection bias, and depending on that you have different methodologies to deal with. So I think fundamentally the most important is really to identify potential sources of bias. How this could interfere with your interpretation and then apply the right methodology and luckily I think, methodology for this field to deal with such sort of biases has been tremendously progressed over the last many, many years.

So I think there are good tools for methodologies approaches available to deal with, but the fundamental challenge remains that you really are clear, you know the biases, even the source, and then you can on average appropriately correct to get really an appropriate inference out of your data analysis.

What do you see as the role of real-world evidence in HTA decisions in five years from now?

Scott Ramsey:Well I can quote Yogi Berra for my five-second summary, which is it's tough to make predictions, especially about the future. But I think more seriously, I think that we are going to see a growing role for HTA or for EHR derived real-world data in HTA decisions in the future, because we were going to have to use this data as the volume of products grows and the pressure on health systems to control the spend grows. I just don't see any other way, but using these real-world data sources to help address fundamental problems that just aren't going be addressable through the regulatory evidence structure.

Dominik Heinzmann:I think it will be really a central part of the assessments, because of increasing data volume we see with real-world data, and also total enhancement on the data quality, which makes really inference out of those data sources even more reliable. And also advances in the methodologies to deal with the different bias we heard before in the questions about selection bias. So all this really ensures probably this will become a very integral part of the assessment dossiers.

And in addition I think, we will also see novel questions which you can currently not address by clinical trials addressed as for example treatment sequences. When we think about patient journeys, how you really assess which sequence of the treatments you actually administer so that the patients really derives the most benefit. So I think we will see much more innovation in the type of questions you're able to address as well in a few years.

Transcript

Danielle Bargo: All right. It looks like we have a really good amount of people on the phone today. So, thank you everyone for joining our webinar today. We're looking forward to spending the next hour with everyone. My name is Danielle Bargo, and I'm the head of HTA Program Development at Flatiron Health. My team is focused on advancing the use of real-world evidence to support HTA decision making. Today I'm joined by Dr. Scott Ramsey from the Fred Hutchinson Cancer Center, and Dr. Dominik Heinzmann from Roche. The goal of our webinar today is to discuss the recurrent evidence gaps that need to be addressed when preparing to submit HTA submissions, and specifically how real-world data derived from electronic health records can be used to address those gaps.

We know that today more than ever, researchers are turning to real-world data in order to gain valuable insights throughout the oncology therapeutic lifecycle. This means that real-world data's being leveraged throughout the product lifecycle from early discovery and clinical development, to supporting regulatory approval, all the way through to commercialization and market access. So today we will be focusing on using real-world evidence to support market access with a focus on Flatiron data being a source of clinical evidence for health technology appraisals.

A few housekeeping tips before we get started, given that this is a webinar. I would like to draw your attention to the Q&A option towards the bottom middle of your screen with the black bar. That is available throughout the webinar. So at any point during today's presentation, please feel free to submit questions using the Q&A. We'll do our best to answer as many questions as possible towards the end of our webinar, but we will follow up after the webinar with any unanswered questions. Finally, before we jump in, please do excuse any guest appearances from our pets, children, or loved ones, given that we are all working from home.

So before I hand things over to Dr. Scott Ramsey, we're going to start with a quick poll question, which should be launching on your screen momentarily. Your answers to this question will help guide our work and make sure that we're supporting you as best as possible. So our first question is, which real-world data sources have you used for HTA submissions in the past? It could be none, you could have used chart reviews, EHR derived data, claims data, or registries. We'll give everyone 30 seconds to answer.

Okay. We're going to end the poll and share the results. So it does look like around 50% of the attendees today have used EHR-derived data, which is really exciting, closely followed by claims data, which isn't surprising given the health care resource utilization and the cost data available there, followed by registries which is also quite a high number, and chart reviews. But still around 20% of our attendees today haven't used real-world data for HTA submissions. So hopefully everyone will be able to learn some valuable insights today from our webinar and learn more specifically around how EHR derived real-world data can be used for HTA submissions.

So, I'd now like to introduce Dr. Scott Ramsey, director of the Hutchinson Institute for Cancer Outcomes Research at the Fred Hutchinson Cancer Research Center in Seattle, Washington. Dr. Ramsey will be reviewing how real-world data is typically used in HTA submissions and address recurrent evidence gaps. He will then dive in to opportunities and challenges associated with using EHR derived real-world evidence for HTAs. Over to you Scott.

Scott Ramsey: Thanks a lot, Danielle, and thanks to everyone who's joined today for the program. So what I'm going to do today is give a high level overview of health technology assessment, drawing on my experience as a researcher in this field and also my experience in US pharmacy and therapeutics committees where I've been involved directly in using health technology assessments for decision makers in US commercial health plans, health insurance plans. But I am going to talk a little bit about the international focus of health technology assessment, and then focus the comments specifically on the potential benefits and some limitations of using electronic health record derived real-world data for these HTA submission packages.

So, let's start off on the next slide with just a definition. I think it's good to ground everybody. For those of you who know this field, you can find a number of definitions of HTA, but I found this one to be perhaps the most useful for the widest number of stakeholders. So HTA is by design a multidisciplinary process that uses scientifically robust methods that are well described in multiple guidelines to assess the value of using health technology at different points in its lifecycle, although I'd have to say most of the time HTA is applied very early, usually post product approval. HTA is comparative and systematic and importantly very transparent. The best HTAs get input from multiple stakeholders, payers, providers, patients, and others. Because of this, it can be used to inform health policy and decision making to promote efficient, sustainable, and equitable health care systems.

So, contrast that a little bit with the regulatory approach, which is on the next slide. So regulators have a much narrower mandate, basically focusing on safety and efficacy. In contrast, as I mentioned, HTA supports health care decision making from a variety of perspectives that include insurance coverage, access, reimbursement, and patient care. So that's a broader mandate than safety and efficacy, and as a result really increases the complexity of health technology assessment to include factors such as what's the net clinical benefit compared to available treatments, or what is the appropriate price to provide good value for a particular drug relative to comparators.

So on the next slide, I just wanted to take you a little bit into the mind of HTA decision makers. Those of us who have been in this role, we're trying to avoid making mistakes. We would like to avoid denying access to therapies that significantly enhance or prolong life in real-world practice. Similarly, we'd like to avoid granting access to therapies that don't enhance or prolong life significantly or even do worse, such as have unintended harms in real-world practice. And of course we don't want to pay more than is necessary for a therapy. Given the fact that countries are operating under real and explicit budget constraints for health care, regulators tend to bias towards the null, in other words, assume that bullet two is the more pressing issue.

Next slide. So, just a note on paying for technologies, most HTA bodies have cost effectiveness as part of their remit. It's used much more forcefully overseas than it is in the United States, but it is part of HTA submissions worldwide. NICE here has done a nice job of characterizing their perspective, which is, marketing authorization can be granted when there's enough evidence that a drug is effective and safe, but HTA bodies have to be satisfied that the drug is clinically effective and offers good value. And notice that this statement captures two concepts. The first is that how the drug works in practice can be different from how it works in trials. And second is that health systems act under budget constraint, and must account for the opportunity cost of adopting a new technology. In other words, if you adopt a new technology under a fixed budget, somewhere else something has to give.

Next slide. So, to make the task of HTA even more complicated is the evolving body of evidence. So we start with clinical trials which are fixed and designed in nature to provide support for regulatory approval, and that of course is used to inform a product label. Then once the product label is out and the drug is available for marketing, clinical practice guidelines come into play and offer clinicians some guidance on how to use that. But that's definitely not the end of the story.

As the drug is used in real-world practice, one learns quite a bit about the product that wasn't really known in registration trials, which by nature are in artificial environments and smaller numbers. What's learned about safety, efficacy, and use in clinical practice is then used to inform clinical practice guidelines, and sometimes can even impact the product label. But all of this dynamic process has to be taken into account by HTA bodies, and that's why the HTA decision making can be so complicated.

So, HTA groups understand that the evidence generated for regulatory submissions have limitations for understanding its use in the real world. For example their trial populations are artificial and highly refined relative to patients who receive products in the real world. There's often lack of comparators or comparators in the trials that are not representative of practice in real world. There's also often a lack of evidence on longterm outcomes, for example, survival. So real-world data is going to play an important role in addressing those issues.

I'm just giving you here a few examples of HTA evidence gaps that decision makers have faced when new products are introduced. So for example, problems with the target population. What is the portion of people receiving subsequent therapy after treatment with pemetrexed? For comparative effectiveness, when alectinib was approved, HTA groups want to know how it compares with ceritinib in terms of benefits and safety, particularly in subgroups of patients with CNS metastases. But of course there's no head to head trial offering a direct comparison, so we have to use indirect techniques to make that comparison.

Speaking of longterm survival, the proportion of patients who were treated with standard care would be expected to be alive at 10 years, few if any clinical trials, particularly in oncology, have that time horizon to understand survival, and frequently surrogate endpoints such as progression-free survival are the norm. So we have to turn to real-world data sources to understand survival and to project survival over longer time horizons.

So, as I mentioned, HTA groups turn to real-world data to address some of these evidence gaps, and it looks like many of you are familiar with the types of real-world evidence, and they're listed here. But I'm going to focus today, as I mentioned, specifically on electronic health records, and their unique role in real-world evidence relative to the others, the claims, disease registries, and so on.

Next slide please. Now what are the particular advantages of electronic health record derived real-world data? I see them in these three domains. First of all, the electronic health record is the closest record we have of clinical decision making. So, it provides an ability to infer what treatment decisions have been made, why treatment decisions have been made by clinicians. It also has extensive clinical depth and breadth that is rarely matched in other databases. So you have deep clinical information, laboratory values, radiology reports, biomarkers, clinical notes, and often genomic data that really represents a realtime record of the patient's clinical situation and the decisions are being made.

Finally, because this record is used for clinical decision making, it can be very recent. So, disease registries, claims, databases, can be months or even years behind the current time, whereas electronic health records, if they're able to be accessed relatively contemporaneously to when the data was recorded, can be much more recent and can provide a more proximal record of what's happening in clinical care.

So, I'm going to talk about four key applications when using EHR derived real-world evidence for value assessment. I'm going to go through these one by one and talk about the whys and hows of these different applications and go through them in some depth. So let's get straight to the first one.

For disease background and clinical context, EHR data elements reflect clinical decision making, and that's great, but they also, because they cover large populations, they can have scale to understand decisions at the population level. So how can this real-world data be used? Well, for the clinical problems that I mentioned a couple slides earlier, we're able to match populations based on clinical characteristics and clinical practice guidelines, to match what was seen and what is recommended, the populations in the real world to what's being recommended.

After doing that, we can use it to assess treatment patterns, getting down to levels of granularity that aren't possible in other data sources, such as dosing, looking specifically at lines of therapy, and so on. So that can provide a lot of benefits relative to other databases. The large sample size allows us to evaluate rare cancers, and this can be a particular advantage for some of the newer trials that are coming out that are focused on genomic subgroups.

Also, real-world data has outcomes and endpoints that are relevant to clinical decision making and to payers at the same time, for example, data of first locoregional and distant recurrence, instances of metastasis, or even reasons why drugs were discontinued. So all of this context can provide a much richer understanding of how drugs perform in the real world relative to some of the other real-world data sources that we commonly use.

As I mentioned, HTA bodies are really interested in looking at longterm outcomes, particularly overall survival, which is critical for cost effectiveness analysis when we're looking at quality-adjusted survival, but is also important, because it's an endpoint that both patients and clinicians feel is the most important, certainly in cancer.

So how can the real-world data be used here? Well, if the overall survival data is not mature at the time that a trial ends, and that trial is used for registration data, then that OS data can be used after launch to understand how the product performs. Also if survival is available from trials, that still leaves the question of does that survival match what happens in the real world. And real-world data can be used to externally validate survival estimates as well.

Now for HTA groups that are overseas, there's often a substantial lag between when the product's approved in the US, and when the product gets through the regulatory and HTA approvals in a different country. That provides those countries with an opportunity to use US data to understand real world survival beyond the trial time horizons. So, that lag in Europe, Australia, Canada can often be used to their advantage, because they can look to the US to see what's happened to patients who have used that product for a number of months or years.

Now to get there, the EHR derived real-world data has to have high quality validated mortality data. I would say that's a feature of most, though not all EHR-derived real-world data sources. It certainly is for Flatiron. EHR-derived real-world data has a number of key variables that can inform the survival analysis. Remember, we're talking about retrospective data, and with retrospective data we're going to have to account for factors that normally would be accounted for in a clinical trial, such as stages, diagnosis, time since diagnosis, and time since last therapy. So the more granular data you have, the more you can adjust for that in making comparisons, and that can provide a much more robust analysis.

Speaking of comparisons, real-world data, one of the biggest uses of real-world data are for indirect treatment comparisons. So this would be comparisons of two therapies where there isn't a direct comparison in the context of a clinical trial, but there is comparative use of that product for the same patients in the real world. So we have to do indirect comparisons using real-world data to make inferences about the relative benefits and harms.

So in this case, the real-world data can be used to better understand the effectiveness versus those registration studies, and we all know the registration studies have important limitations, such as single arm trials where we don't have a comparator or trials that use nonstandard comparators relative to what is most commonly used in clinical practice.

And as I mentioned, rare disease and subgroups, the clinical trials can have very small numbers, and it could be hard to make inferences about outcomes. Think of the so-called basket trials where patients are treated based on a genomic subtype across multiple cancers, or even trials such as CAR T therapies where the total number of patients treated are very small. So real-world data can play a very important role in these applications.

As I mentioned, there are a lot of benefits to using the EHR-derived real-world data in these instances. The clinical depth allows for that adjustment of clinical factors that could lead to potential confounders. We can also look closely at treatment sequences, and really understand where the product is being used in the patient treatment pathway. And as I said in some situations there will be a lag between HTA decision and what's available in the United States in terms of followup, so that longer followup gives us a window to look at the impact of the product over time horizons that are beyond that of the clinical trial.

Finally, I think EHR derived real-world data can be very useful for reassessment of drugs that have been on the market for a period of time. So here the question we're seeking to ask is, how do the drugs perform in real-world practice and should we reassess how we pay for them? This evidence is of particular use for payers, but it also is useful for clinicians who are constantly reassessing when and how they should use particular drugs as newer drugs come on to the market.

So HTA bodies particularly in Europe have been demanding increasing evidence for novel products treating small patient populations, because those come with additional uncertainties that just simply aren't powered for the endpoints that are important to HTA groups, as I mentioned overall survival being one of the major ones. Again, these basket trials where you might see one or two patients in particular tumor types, whereas there might be 50 across multiple tumor types for a particular genomic signature, is a great example of small patient populations. So, yes the drug was approved for particular patients who have a particular mutation. Does that mean it works equally well in a lung cancer patient or a colorectal cancer patient? It's often hard to know.

Second, real-world comparative effectiveness studies can be used to evaluate issues affecting performance in practice, and these could be used to inform clinical practice guidelines. We all know that physicians don't use drugs exactly as they are in labels, and there are a variety of reasons for that. And patients who get these drugs don't always meet the eligibility criteria in guidelines. So seeing what happens in real-world practice can inform the guidelines, but also can enable payers to set up performance-based risk-sharing agreements such that if the product isn't performing as we thought it might in a clinical trial, there can be adjustments to the reimbursement to make the value equation stronger.

So the particular benefits of the EHR-derived real-world data can be used in these genomically defined cohorts. The latest, or one of the latest examples is the NTRK gene fusion mutation and the newer drugs for that. There are very few patients that actually have this mutation, and even fewer that respond to the drugs. So looking at clinical trials really gives a limited window of how they're performing in the real world. And also, again, more contemporaneous picture of oncology treatment. EHR because of its recency, allows us to get an early look of drugs that are adopted quickly by practitioners so that we can at least get an early signal of whether they're performing as we expected based on the clinical trial data.

So in summary, I think there are a lot of potential benefits for using EHR-derived real-world data that are worth considering by HTA groups, and by those of you in industry who are preparing evidence dossiers for those HTA groups. In summary, health technology assessment goes beyond regulatory evidence. It considers clinical practice, perspective of multiple stakeholders, and value for money, and the EHR data can be a great complement to claims and registries to really understand from a broader perspective how the products are performing. The particular advantage that I see for EHR derived real-world data is this high degree of clinical specificity, which allows us to match populations in real world much more closely to guidelines or product labels to fill in those critical evidence gaps that are always there at the time of regulatory approval.

So we're seeing more and more early uses of EHR derived real-world data, and I categorize them into several high level buckets. One is estimating and validating longterm endpoints such as overall survival. There's growing use supporting indirect comparisons, and emerging use reevaluating outcomes following introduction in clinical use. So I'll stop there and turn it back over to Danielle who can take it from here.

Danielle Bargo: Perfect. Thank you Scott. That was a great summary of HTA considerations and the challenges that we're facing when we're seeking reimbursement. So thank you for that. So before we hand over to Dominik, we're going to do another poll question, which should be launching on your screen momentarily. And the second poll question we'd like to have everyone answer today is which application of EHR derived real-world evidence is most feasible to you in value assessments: natural history, longterm survival extrapolations, indirect treatment comparisons for single arm trials, and reassessment? This is building off of what we just heard from Dr. Scott Ramsey. I'll give you 30 seconds to answer.

Okay, just a few more seconds. Okay. I think we can go ahead and end the poll and share the results. All right. So it looks like 45% of people said that using EHR-derived real-world evidence could be a potential source of evidence for indirect treatment comparisons, specifically for single arms trials, which is great to hear, given a lot of the thinking around single arm trials and how you actually achieve market access for those products. That's great to here that EHR-derived real-world evidence could be a source of data for that. All right. So now I'd like to hand it over to Dr. Dominik Heinzmann who is joining us from Switzerland today. He's the global head of personalized health care data science oncology for solid tumors at Roche. Over to you, Dominik.

Dominik Heinzmann: Thank you very much, Danielle. Next slide please. So, basically today I want to give you an example about the use of real-world data and in particular what Scott mentioned before in the extrapolation for OS. These are my disclosures.

Next slide please. And definitely the whole work is really great teamwork and a very multidisciplinary approach, like Scott mentioned, and also big thanks to Flatiron for a very efficient collaboration and making time for this project. So at Roche we have really the ambition to advance real-world data across the life cycle of our health technologies, really starting with research development up to regulatory interactions, access as well as clinical practice.

Today the case example or the use case I would share it's within the pocket of the access. Definitely as you see, access has a couple of different aspects you can address. It's nicely explained by Scott. And the one I will tackle is really about the extrapolation for OS or how you can use real-world data to support the assumptions in your modeling for the extrapolation part.

Next slide please. So to set the scene, the case example is really about atezolizumab or Tecentriq, and it was as part of the discussions in the UK with the NICE department. So basically Tecentriq or atezolizumab is a monoclonal antibody designed to bind to the protein PD-L1. Through that it blocks interactions with certain receptors including PD-L1 and like that it enables the activation of T-cells. Tecentriq is approved globally in multiple cancer indications, and the specific situation in the UK is as follows. So if you have a single payer system and it's the National Health Service, which covers about 90%, 93% of the population.

And then an integral part of the assessment of health technology is really in NICE. So NICE is the National Institute for Health and Care Excellence and actually performs this cost effectiveness of the health technologies. That's good also although it's really an iterative process. So you submit an initial package, they have questions and you bring on different experts on board, you provide additional evidence. So it's really cycling across the clinical evidence, but as well as the economic evidence to really come up with a decision which then puts your marking into one of the categories fully funded, funded with restrictions. For example if treatment duration somehow gets restricted or not recommended and then the NICE recommendation goes back to NHS, which finally makes the decision for reimbursement.

Next slide. Okay, let's start this journey with the example in the UK. So it was really back in time, 2017, 2018, the NICE review. The context was really the pivotal trial for atezolizumab the so called OAK study, which was a phase three study with roughly around 800 treated patients as primary population, and demonstrated atezolizumab is superior to docetaxel independent of PD-L1 status. So in the entire ITT population, leading to the initial target population of an all-comers.

However, the situation at that time was the other cancer immune therapies are actually funded for more narrow indications, mainly the PD-L1 enriched population. So what does it mean for patients? So if you have disrestricted access, first of all, the obvious things like PD-L1 negative patients, don't get access to the drug, but on the other side, it's also when you do this biomarker assessment, it can be that you don't have actually a result.

For example, if you don't have available biopsies in which case you would also not be eligible to be treated if your drug is only restricted for the PD-L1 enriched population. So based on this trial and some of the phase one data Roche submitted a dossier with initial extrapolation models. So technically a projection of the longterm survival, because in a pivotal trial, you normally don't have really very long follow up of a particular overall survival endpoint.

This is conceptualized on the right side where you can see the Roche extrapolation, so basically that's the green part for the docetaxel arm and the yellow part for the atezolizumab arm, and you have the projection or the mapping done by ERG, which is an expert group within NICE who consults NICE on the clinical evidence, and as well as the economic evidence. So basically their docetaxel extrapolation was slightly lower than the green one, and this had a tremendous impact on the resulting extrapolation for the atezolizumab.

So why NICE challenged, so although Roche provided a lot of real-world data, like UK registry data, US SEER data, and other data sources, and there were a couple of limitations with all these data sources, as an example, it was not exactly the right population to reference the data or to benchmark the data, or the data lacked certain robustness data quality, or it was just considered not contemporary enough to really allow a meaningful benchmarking, and with the assumptions in the extrapolation model.

So the solution which I explain on the next two slides a little bit more in detail, about really to use Flatiron-matched data, so we match the population, so that we have contemporary data with a validated OS endpoint to support the assumptions going into the Roche extrapolation model. Next slide please. So the methods we applied back then was really as follows. So for the experimental arm, given that cancer immune therapies can... like previous cancer immune therapies, like pembrolizumab for example, they're only approved three years prior.

So there was no way to give you a really a reliable, a longterm five-year milestone survival estimate for overall survival. However, for the control arm using docetaxel, the Flatiron data really allowed to have a reliable estimate of the five-year milestone survival, and hence this estimate could be used to verify your modeling assumptions for the extrapolation.

As you can see the results below you see what came out of that was the green part. So if we go the five-year milestone survival of about 4% in the docetaxel arm, which compared favorably to the 2% in the Roche assumptions for the extrapolation model, but on the other side which contradicted to some extent the ERG estimate of 1% in the docetaxel arm. And basically this was really based on the ERG at that time, and didn't believe there is really long term survivors on the docetaxel arm, however using the Flatiron data, we could really show that there is actually even not a large percentage, but there are those very long term survivals in the docetaxel arm.

And basically the matching part was really restarted with the full advanced, non small cell lung carcinoma EDM, which at the time was roughly around 37,000, and then we restricted to match the cohort to patients initiating docetaxel treatment in particular for the second line treatment. And we restricted the timeframe to exactly match the time horizon of the OAK trial. So we ended up with around 800 patients which gave us this estimate of 4% I just explained before.

Next slide. So basically a comprehensive data package was, including also Flatiron data lead to the approval of the first all-comer funding for cancer immune therapy. So it just means regardless of PD-L1 status or histology, and like what I mentioned before and Scott also highlighted it was really an iterative process. So we had a couple of meetings with NICE. And in the second meeting you see on the left side, that was really what they explicitly challenged the Roche extrapolation approach, because what I mentioned before, their assumed like long term survival rate was lower than the one that proposed by Roche for the extrapolation model in the control arm.

However then in a subsequent meeting, and they're actually Flatiron data was in the meantime submitted. The NICE committee concluded that using the Roche extrapolation approach was appropriate for decision making. So in conclusion, we could see there's really great potential for real-world data in HTA interactions to really help to further understand effectiveness safety, treatment modalities, associated with the different treatment options. And I think it really helps to supplement and then enrich an evidence package generated by clinical trial data but supplemented by real-world data to cover different aspects for allowing really a meaningful assessment of the health technology. Thank you. And with that, I hand back to you Danielle.

Danielle Bargo: Perfect. Great. Thank you Dominik for sharing that example of how Flatiron data has actually been used to validate longterm survival extrapolations, especially for an assessment by NICE. So before we move on to the Q&A portion, we're going to do one last poll, which should be launching on your screen now. And our last poll question for today is, when evaluating a real-world data source for HTA submissions, which of the following qualities are most important. If everyone can choose three qualities that are important, that would be great. And we'll give you 30 seconds to answer.

Okay. We'll give you a few more seconds. Okay, we can go ahead and end the poll and share the results. All right. So it seems like an overwhelming amount of people said that availability of real-world end points is an important quality when looking at real world data sources. So that's great to know, closely followed by clinical depth and availability of clinical variables, and rounding out number three is longitudinality. So actually having longitudinal data available.

So yeah, very good points. And hopefully everyone feels that they know a bit more about Flatiron data and how it addresses all three of those issues. So we're now going to move on to the Q&A portion of today's webinar. So all of us will actually take off our videos. So it looks like we have a number of great questions coming through, so we can go ahead and start answering those. So our first question that's come through is, how do HTA bodies view real world data and submissions? Scott, do you want to start us off? The question is how do HTA bodies view real-world data and submissions?

Scott Ramsey: Yeah. Happy to take a first shot at that. And I can speak mostly from my perspective as a decision maker on US pharmacy and therapeutics committees, which review these evidence dossiers, but I do have a number of colleagues who work overseas and have relayed to me their experiences. It's a hard question to answer because it varies so much depending on the particular submission and the particular product that's being considered and where it is in the life cycle. I would say the use, as a rule of thumb, the use of real-world data and the acceptance of real-world data is inversely related to the quality, if you will, or completeness of the data that are available at the time of submission.

So for example, with one arm trials, trials with nonstandard comparators, or even the more recent basket trials, which genomically defined treatment regimens where patients have handfuls of patients in particular cancer subtypes are used then the decision makers in HTA are often very frustrated by that clinical data and they're hungry for new data sources to help understand how those products are working in the real world.

So in those cases, data like Flatiron type data, particularly the linked Flatiron data with the FMI clinical genomics database, can really provide a fair amount of influence to help those decision makers. I think Dominik gave a great example of how extrapolated data can be used to inform a decision that NICE made because they were not satisfied with the data that were available at the time that that product was launched.

But if you have a trial on the other hand, or if you have a product that has phase three trials with extended data points with randomization, those are going to be looked at more carefully, and be given much more strength than real-world data. So, as I said, it's hard to generalize on when their most... real-world data is most accepted, but it is generally related to the quality of evidence at submission.

Danielle Bargo: Thanks Scott. And Dominik. Just to follow up the same question, in your opinion, how do HTA bodies view real-world data and submissions?

Dominik Heinzmann: I think in general they are very welcome. If it's about supplemental data, which helps to close certain evidence gaps. I see there's a big support to use them. They require a very transparent communication about elements like data quality, exact sources of biases you may have with the data source. So you have addressed it. So once you have transparency, I feel real-world data is really welcomed by health authorities that supplement to evidence together with some clinical data definitely, and most of the preference is for some sort of randomized data.

When it comes to supplement, some of the clinical datas we've seen before heard from Scott in certain instance where we have a single arm trial, and situation gets a little bit more complex. So it really depends on the exact situation where you can use for example, real-world data to generate an external control arm to conceptualize single arm data.

There are different considerations in which case it may suit well but in certain situations it may be a little bit challenging to accept this as some final evidence to make a decision for reimbursement. So in general, I think there's really a lot of learning and a lot of welcome culture for real-world data, as long as the communication is very transparent. However on a deeper and knowledge about really the sources of biases, how you address them, all this more methodological aspect, I think there's still a lot ongoing. And what I see is really great collaborations between all the stakeholders in this ecosystem to get the ways to really make real-world data as part of the submission dossier at the end really help the patients to get access to effective and safe treatment.

Danielle Bargo: Great points Dominik. And then there's a followup question that I think both of you would be great at answering is, so given that HTA is a decentralized process, and there's different HTA bodies in each country, how do these countries or markets differ in their assessment of drugs, and then also their acceptance of real-world data? So Dominik, let's start with you on this one. How do HTAs differ in their assessment of drugs and acceptance of real-world data?

Dominik Heinzmann: I think to start really historically it's probably not very different to what we've seen years ago in the regulatory landscape if it goes about new treatment modalities. You see a certain initially heterogeneous assessment all with the really same principals to assess properly the molecule or the health technology to have it really safe, effective for the population on the regulatory side.

I think similar was that HTA that you see currently still a little bit of heterogeneity, and in hope, the weight the evidence generated by real-world data. But I think there's also a greater for ongoing different associations really, and bringing the different stakeholders together, have very productive discussions. OSC is seeing a convergence of all these concepts, but we get closer and closer to really quantify what is meaningful in real world data, this includes amount, quality, or other attributes as well.

So I think it comes together. Currently we need to live with this slight and be creative among the assessments, but the thing at the end of the day, I think everybody, the underlying principles in this assessments are very similar. So even if you see some heterogeneity at the end, fundamentals are very similar between the HTA, all this and payer assessments.

Danielle Bargo: Thanks, Dominic. And we have a lot of questions here. So we're going to move on to the next one. So Scott, this is for you. And the question is, what is the acceptability of US EHR data in ex-US HTA submissions? And the question also asks if there's anything specific to Asian markets that should be considered.

Scott Ramsey: Well yes, regarding acceptability, I think there is growing acceptability of US data. And I have to admit my only experience with that is with HTA bodies. I don't have knowledge of what's happened in Asian markets. But I know for example, that Flatiron data has been used in multiple submissions to NICE, has been brought to IQWiG in Germany, and other countries in Europe, and has been influential in making decisions. And the reason I think is twofold. One is the thing I mentioned in my talk that there's usually a lag when the European regulators look at real-world data. So they can go to the US which has often adopted the products earlier and more aggressively to really understand how the products are performing.

And then the other is, most European countries are much smaller than the US, and they don't have the sufficient numbers for some of these products that are used in rarer tumor types. And then the US simply by its size and numbers can be very useful. The question that always comes up though is, is the US patient and the US healthcare system representative of the experience in Europe or Asia or elsewhere.

And the short answer of course is that it's not, but in cases where you have no data in your country, it's a bit of an art, but you have to decide in which situations what's happening in the US can be translated to the different country. So things like overall survival can be useful, safety signals from real-world data can be very useful. So I think there will be a use in all countries of US data by its size, its availability, to answer questions that just won't be there. But it varies from country to country.

Danielle Bargo: And Scott, I think a really good follow up question for you is with the complexities and heterogeneous payers that we have in the US, do you think that the US will ever have an HTA body?

Scott Ramsey: Well the US actually has multiple HTA groups. They are all private, a great example is ICER. But there's a number of other groups that perform health technology assessments. And there actually are standards for submitting HTA dossiers, particularly by the AMCP Academy of Managed Care Pharmacy, that almost all health plans, commercial health plans in the US adhere to. So HTA is used and viewed by all health plans.

Now, is it used the way it is used in Europe and Asia? No, we haven't gotten to the point, particularly where cost effectiveness is used to influence decisions that other countries have gotten to. But times will change, there's ever-growing pressure to control costs in healthcare. And the US has tried almost every other solution besides cost effectiveness, and someday it may figure out that they have to use cost effectiveness to manage their spend trend. I can't predict when that's going to happen, but I do think you're going to see more use of things that focus on value, and real-world data is going to be a part of that.

Danielle Bargo: Great point Scott. And Dominik, a question for you is, can you share some thoughts on how to deal with selection bias when using real-world data?

Dominik Heinzmann: Of course, I think that's an important consideration. Even selection bias is some heterogeneous, depending really what caused this selection bias, and depending on that you have different methodologies to deal with. So I think fundamentally the most important is really to identify potential sources of bias. How this could interfere with your interpretation and then apply the right methodology and luckily I think, methodology for this field to deal with such sort of biases has been tremendously progressed over the last many, many years.

Dominik Heinzmann: So I think there are good tools for methodologies approaches available to deal with, but the fundamental challenge remains that you really are clear, you know the biases, even the source, and then you can on average appropriately correct to get really an appropriate inference out of your data analysis.

Danielle Bargo: Thanks Dominik. And we have one last question, given the timing of where we are. So the question would be, what do you see as the role of real-world evidence in HTA decisions in five years from now? So Scott, if you can give us a 20-second summary and then we'll hand over to Dominik.

Scott Ramsey: Well I can quote Yogi Berra for my five-second summary, which is it's tough to make predictions, especially about the future. But I think more seriously, I think that we are going to see a growing role for HTA or for EHR derived real-world data in HTA decisions in the future, because we were going to have to use this data as the volume of products grows and the pressure on health systems to control the spend grows. I just don't see any other way, but using these real-world data sources to help address fundamental problems that just aren't going be addressable through the regulatory evidence structure.

Danielle Bargo: And Dominik, what are your thoughts? What's the role of real-world evidence for HTA in five years from now?

Dominik Heinzmann: I think it will be really a central part of the assessments, because of increasing data volume we see with real-world data, and also total enhancement on the data quality, which makes really inference out of those data sources even more reliable. And also advances in the methodologies to deal with the different bias we heard before in the questions about selection bias. So all this really ensures probably this will become a very integral part of the assessment dossiers.

And in addition, I think we will also see novel questions which you can currently not address by clinical trials addressed as for example treatment sequences. When we think about patient journeys, how you really assess which sequence of the treatments you actually administer so that the patients really derives the most benefit. So I think we will see much more innovation in the type of questions you're able to address as well in a few years.

Danielle Bargo: Perfect. Thank you for those closing thoughts. So with that, we'll conclude the Q&A portion of the webinar. So let me just first say thank you to everyone, our panelists, our attendees, for taking time out of your day and discussing with us how real-world data can be used in HTA submissions and to support market access globally. If you have any questions about the content presented, please don't hesitate to reach out to rwe@flatiron.com. Upon closing of out of this webinar, you will be prompted to take a short survey to help us improve future webinars. So your response is very much appreciated, but most importantly have a great rest of your day, stay healthy and stay safe. Thank you everyone.