Popular Searches:

    How COVID-19 is changing cancer care, health policy and research—for good.

    In our first episode of “The Real World, In Practice”, a podcast by Flatiron Health, we examine COVID-19 through the lens of oncology, real-world data and health policy. Our guests include Nicole Mahoney, senior director of regulatory policy, and our chief medical officers for our Provider and Research teams, Bobby Green and Mike Vasconcelles.

    Articles mentioned:

    Jump to transcript



    Bobby Green, MD

    Chief Medical Officer

    Clinical Oncology
    Linkedin Twitter

    Bobby Green, MD, is Flatiron’s chief medical officer and a practicing medical oncologist. At Flatiron, Bobby oversees the team responsible for clinical development of OncoEMR®, the industry’s leading electronic medical record software for medical oncology, and Flatiron’s other provider-facing software products. The team is also responsible for all clinical content development, and is supporting the development of Flatiron’s novel clinical trials offerings.

    Prior to joining Flatiron Health, Bobby was a managing partner at Palm Beach Cancer Institute and helped lead its merger with Florida Cancer Specialists (FCS) in 2013. Bobby continues to see patients in West Palm Beach Florida at FCS on a part time basis.

    He served two years as Chief Medical Officer of Cancer Clinics of Excellence where he led initiatives in molecularly based clinical trials, integrating genomic testing into community practices, alternative payment models, and data analytics. Bobby currently serves on the board of the Community Oncology Alliance, and is a former executive board member at FCS.

    Read more


    Michael Vasconcelles, MD

    Chief Medical Officer

    Research & Regulatory

    Michael Vasconcelles, MD, is chief medical officer at Flatiron Health, where he is responsible for defining and executing the company’s strategic vision for real-world evidence, providing thought leadership and guidance around novel clinical research methods, and partnering with clients and stakeholders across academia, life sciences and government agencies to further the use of real-world data. In this role, Mike works cross-functionally to drive the overall strategy and execution of Flatiron’s research business.

    Prior to joining Flatiron, Mike served as chief medical officer at Unum Therapeutics, a Cambridge, Massachusetts, cell and gene therapy company developing autologous engineered T cell products for the treatment of cancer.

    Mike currently serves as a clinical instructor in medicine at Harvard Medical School, where he has taught since 1996, and is an associate physician at the Dana-Farber Cancer Institute and the Brigham & Women’s Hospital.

    Read more


    Nicole Mahoney, PhD

    Senior Director

    Regulatory Policy

    Nicole Mahoney, PhD, is the Senior Director of Regulatory Policy at Flatiron Health, helping advance the use of real-world evidence for drug development and regulatory decision making. Nicole serves as an advisor to the Duke Margolis Center for Health Policy's Real-world Evidence Collaborative, and is an Associate Editor of Therapeutic Innovation and Regulatory Science.

    Prior to her current role, she was a Director of Global Regulatory Policy at Merck, advising on the use of real-world evidence for regulatory purposes, successfully advocating for incentives and new regulatory pathways to spur antibiotic development, and providing strategic input to cross-functional product teams regarding product-specific regulatory matters.

    Nicole served as the senior officer for the Pew Charitable Trusts' antibiotics and innovation project and as a U.S. Food and Drug Administration Commissioner’s Fellow. She earned her Ph.D. in biochemistry at the Albert Einstein College of Medicine and was a postdoctoral fellow at the University of California, San Francisco.

    Read more


    Alex Gorstan


    Research Marketing

    Alex Gorstan is a marketing director at Flatiron Health and the host of this episode of The Real World, In Practice podcast. He partners with experts across the organization to drive the measured and meaningful application of real-world evidence in oncology. Before joining Flatiron, Alex worked in Product Management and Software Engineering at companies including Opower, Johns Hopkins Medical Institute and CareFirst Bluecross Blueshield

    Read more

    In this episode:

    • 01:29

      Why this pandemic has become a seminal moment for real world evidence.

    • 09:18

      How has COVID-19 impacted the collection of data for researchers?

    • 13:16

      What changes are policy makers exploring?

    • 15:50

      Should we take a closer look at reframing clinical trials?

    • 22:51

      And finally, how is social distancing shaping the new normal for cancer patient care?


    00:00 Nicole

    Nicole Mahoney And I think the Food and Drug Administration and other parts of the government actually realized right away that they were sitting on essentially a treasure trove of information, because this was an entirely new disease and one that required an immediate response.

    00:20 Alex

    Alex Gorstan This is The Real World, in Practice. A podcast by Flatiron Health. I'm Alex Gorstan. Today, we sit down with Nicole Mahoney, senior director of regulatory policy at Flatiron and our chief medical officers for our provider and research teams. That's Bobby Green and Mike Vasconcelles. At the center of their discussion is the realization that COVID-19 is having a major impact on the practice of oncology. This is leading to downstream impacts on researchers who rely on real-world data. And as policymakers scrambled to leverage data against the growing pandemic, major health policy challenges are coming into focus.

    Now in the discussion today, we're going to hear a lot about real-world data. And if you're not already familiar with the term, real-world data refers to information gathered routinely outside of clinical trials. And in our case, we're really talking about detailed information gathered at the point of care. Information that's proving increasingly valuable in the fight against cancer.

    Okay. Let's dive in. Here's Bobby kicking things off. And just for background, in addition to being the chief medical officer for our provider team, Bobby is also a practicing community oncologist. Okay here's Bobby.

    01:29 Bobby

    Bobby Green There is a sense in a strange way that COVID has almost been this sort of seminal moment for real-world evidence. And so maybe just to start, what can real-world evidence do to really help elucidate insights about COVID-19?

    01:46 Alex

    Alex Gorstan As I mentioned before, Nicole is our senior director of regulatory health policy at Flatiron. She came to us from Merck where she was global regulatory policy director.

    01:54 Nicole

    Nicole Mahoney So when we think about what real-world evidence can do, I think it's important to say first and foremost, it can and is filling critical gaps in what we understand about this disease. So I think of things through the regulatory policy lens and early, very early on in the pandemic, FDA worked with a group of stakeholders, pharmaceutical companies, real-world data providers, and others, academics to think about what questions can we ask. And there was really an acute need at the time to understand the natural history of the disease. So that's been a real help to have real-world evidence address those things. So that means, who's getting COVID? Who requires ventilation? What are the outcomes for these patients and what treatments are they being exposed to? And are any of those treatments working? Do they appear to be working? And so the information can also be used in a whole variety of ways, including the management and allocation of resources.

    02:50 Bobby

    Bobby Green Gotcha.

    02:52 Alex

    Alex Gorstan This is Mike speaking here. He's our chief medical officer on the research side of the business. Also a practicing oncologist, Mike spent lot of his career leading the development of oncology therapeutics across a number of well known life science companies.

    03:05 Mike

    Mike Vasconcelles So Nicole, what's your view of just the immediacy of the access to the data? In terms of those questions that are being asked? I mean, to me that's really one of the potential real opportunities that a real-world data set can provide.

    03:20 Nicole

    Nicole Mahoney That's absolutely correct. And I think the Food and Drug Administration and other parts of the government actually realized right away that they were sitting on essentially a treasure trove of information. Because this was an entirely new disease and one that required an immediate response. And I think that real-world evidence can fill the gaps because it can be collected in essentially real time and shared out in real time. Or at least it has the potential to do that.

    03:49 Mike

    Mike Vasconcelles Right.

    03:50 Bobby

    Bobby Green I was just going to say Nicole, the flip side to that is, on the one hand we have electronic health records. The data is easier to pull out than it would have been 10 years ago or 15 years ago. But at the same time, the sort of world we envisioned with interoperability where large hospital systems all have EHRs that talk to each other and so you could easily analyze this data at scale, without a ton of work on the backend. We haven't really gotten to that point yet right?

    04:22 Alex

    Alex Gorstan Okay. Here's a good spot to make sure we're all on the same page here. Interoperability. So this is basically the idea that different health information systems should be able to work together and share information seamlessly. It's easy to imagine a world where primary care physicians, specialists, and even mobile apps on a patient's phone share the same picture of the care being delivered. And it seems like that's the way things should work. But as you heard from Bobby, we're not there yet. But still there were major opportunities right in front of us early on in the pandemic. Opportunities for real-world data to influence patient care. The off label use of drugs like Hydroxychloroquine, and Dexamethasone being great examples. Real-world data could enable physicians to make decisions faster, differently, or more confidently, but like most things in healthcare there are both benefits and limitations to consider. Let's hear Nicole speak to some of those nuances.

    05:14 Nicole

    Nicole Mahoney While there is a lot of promise in real-world evidence, I think this has laid bare a lot of the limitations to the way our current healthcare data ecosystem is actually structured. Even for things like public health and getting reporting out from different organizations. There are a whole set of technological difficulties in pulling information from different healthcare providers or healthcare centers. But there are, at the same time policy issues. Issues that make it difficult or unclear on how and when information can be shared, for example. So I think you're absolutely right. There are multiple layers to the issues that you've raised.

    05:58 Alex

    Alex Gorstan One thing that may be a shock to no one is that the healthcare sector is slow to adopt new technology. Now you might be thinking, if there ever was a time to accelerate health policy objectives like interoperability, now would be the time. Is that possible given the industry's track record?

    06:15 Bobby

    Bobby Green My 2 cents, I think there are certain things that have been accelerated by this. The perfect example, being telehealth. A lot of the restrictions and billing around telehealth was made easier. Which was an initiative by CMS, which was incredibly important, which I think we all hope will continue. And I think helped in care delivery across the countries, especially in the cancer space. I will take the cynics view that I'm not sure what can actually truly speed up interoperability beyond the relatively slow pace that's been going on. Just because I think it's a huge and complicated problem to tackle, which requires more than CMS changing billing codes for a certain thing. Just my perspective, but I'm skeptical that it will. I hope I'm wrong, but I'm skeptical that it will.

    07:06 Mike

    Mike Vasconcelles And Bobby building on the rapid adoption of caring for patients in unique ways, with a telehealth approach, immediately had to figure out how to incorporate those approaches to care in your practice. And I'm just curious, do you see that waning as the acute need to provide access in that way starts to dissipate over the coming... hopefully in the coming months or years?

    07:34 Bobby

    Bobby Green There was definitely a novelty component to it. And I think... I mean Mike you'll remember probably the first day you got your beeper in medical school, you were really excited. And then within a week you wanted to throw it out the window because you're like "This isn't as cool as I thought." I think there was a component to telehealth that was like that. The first day you're in clinic doing telehealth, you're like, "Wow, I can do a video chat with patients. This works really well." I think that there have been technical and logistical things that have made it a little harder. I think some of the telehealth companies just weren't ready for the huge flood of people who were going to use it.

    So the platform, at least that I've used sometimes gets a little bit clunky, hard to connect, patients have difficulty connecting. So it's certainly not a panacea, but that being said, there's clearly been a critical role during COVID. And to specifically answer your question, I think it's going to persist. There are certainly lots of things that I'm sure you've seen in clinic and that we all see as clinicians of examples when patients travel to the office for care. And at the end of the day, they probably didn't need to come in. There are certainly opportunities when people need to come in for treatment, that's different. When people critically need labs that you need to be able to look at that day, that's different. There's some times an assessment that you can only do with a physical exam and seeing someone in person. But there's also a lot of stuff that you can do remotely. And I think still deliver really good care. And I hope that persists, because in general it makes life easier for patients. And also, especially if you're in rural areas, areas where it's difficult to get transportation, it certainly becomes even more critically important.

    09:18 Alex

    Alex Gorstan So, it's safe to say we've seen some modifications to everyday treatment practice. Now, when one of the tried and true use cases of real-world data is to better understand treatment patterns under normal real-world circumstances, what are we to make of these data when normal is out the window? Here's Mike.

    09:35 Mike

    Mike Vasconcelles I think it's fair to say, at this point in time, we're really just working really to understand the impact of the way in which changes in care delivery are affecting the way in which we can properly ask research questions, and it goes really beyond sort of a shift or an addition to telehealth visits. We know, for example, kind of to your point, that access to the clinic in a variety of ways, whether it has to do with monitoring of care or delivery of medicines to treat cancer, all of that has had an impact because of the pandemic. That rolls through the research questions that we can ask.

    10:15 Bobby

    Bobby Green Gotcha.

    10:20 Nicole

    Nicole Mahoney When I think about answering important questions, I think about drug development. I think that it's, for certain, the COVID pandemic has accelerated a conversation about real-world evidence and accelerated the thinking around what can be done with it, what appropriate uses are, what type of questions can be answered. We're even getting into a granular level because data organizations are coming to the table, working through experiments, trying to answer questions collectively. We're really accelerating the conversation. The jury is still out, as Bobby was mentioning, on whether or not we'll come to solutions during this. It can be applied when things kind of go back to more of a normal state. So what works for a case where there's an acute need for information might not work under more normal conditions.

    11:10 Alex

    Alex Gorstan Depending on where you live in the country, COVID-19 may be impacting life in different ways. When we think back to mid-March, the country effectively slowed to a standstill. Everything from haircuts to elective procedures were just canceled. How does that affect routine procedures? The yearly mammograms, for instance, that just didn't happen?

    11:28 Bobby

    Bobby Green One of the sort of interesting, outstanding questions, in some ways we've done a natural experiment, right? We've said, just as an example, what happens if you stop doing mammograms on people for a couple of months, because elective testing shuts down? And there was a period of time when mammograms weren't getting done, there's been a period of time where colonoscopies haven't gotten done. And I sort of wonder what are the opportunities there for real-world evidence to answer questions about one, what the negative impact is. And there's also a flip side, right? There's probably things that we routinely do in care that we actually don't need to do and are there things that we stopped doing that aren't going to result in harm?

    12:09 Mike

    Mike Vasconcelles There is a pretty pervasive view that the impact will be negative, but did you suggest that there may be silver linings that could be gleaned or at least better understood from this sort of natural experiment?

    12:22 Bobby

    Bobby Green Yeah, I mean, so let's say, as an example, we saw that not doing mammograms on women between the ages of 40 and 50 actually had no impact, but the only impact we saw was on women over the age of 50. There's a lot of screening, preventative routine health care that we sort of take for granted that we need to do, but you could argue that the evidence on impact at least on mortality is questionable. I mean, Mike as you know, we have lots of people follow up with us in clinic at certain intervals without any data to support it, just because a bunch of experts say that we should, or this is how we trained 25 years ago. I had my colon cancer patients come in every X number of months and I did this on them. And who knows what the real impact is because they're just aren't data.

    13:16 Nicole

    Nicole Mahoney These types of questions definitely aren't lost on policymakers because we've seen over the past month and a half, two months, we've seen FDA come up with a new list of questions about exactly what you're describing. What are the ramifications for different patterns of treatments or missed screening techniques, this is what policymakers are really interested in. On the flip side, our life sciences partners are asking, "What can we maybe eliminate from the information that we collect for research purposes? What is extraneous information that may not be necessary to answer critical clinical questions?" And the interesting thing is this whole pandemic is really accelerating conversations around that too. We've seen drug development change essentially overnight, and in response, the policy makers are putting out guidance in a fast and furious way, continuously updating it almost in real time as they recognize that they can exhibit some flexibility on certain things. So it'll be very fascinating to see how this plays out over the next couple of years.

    14:22 Mike

    Mike Vasconcelles So Nicole, just for our listeners, I mean, on the latter point, that's more so directed in the clinical research realm, correct? Clinical trials and kind of classic development questions.

    14:33 Nicole

    Nicole Mahoney Exactly, and I think there's a way to bridge what's happening in clinical care to the way questions are answered and the ramifications for drug development. And I think that's what policy makers are trying to make the bridge between what is going on in the real world. And then what are the impacts for drug development?

    14:49 Mike

    Mike Vasconcelles Well, I think that's a really important point that we talk a lot about the generalizability and the representativeness of real-world data in the context of research questions and also the kind of comparability of those cohorts to say clinical trial cohorts.

    15:07 Alex

    Alex Gorstan Generalizability and representativeness are two terms we should pause on for a second. Generalizability refers to the idea that things that we learn by studying a smaller group are also true for the population at large. And representativeness, that speaks to the characteristics of the smaller group, that is, is the makeup of the smaller group, things like race, age, socioeconomic status, is the mix of the traits that you see in a smaller group, similar to the mix of traits that you see in the real world? For the best kinds of studies, you've selected a sample that's representative of the real world so that you can reach conclusions that are generalizable to the broader population.

    15:50 Mike

    Mike Vasconcelles I think, however, any of us who've been involved in clinical research intuitively know and understand that those populations that we enroll into clinical trials are not representative of the patients we care for day over day. And so the extent to which sort of the pragmatic realities of dealing with all of the issues that the pandemic has presented with respect to sort of the prosecution of clinical trials may, at the end, suggest that we are over-engineering the way in which we think about patients enrolled in those studies, we're collecting too much information, we're too prescriptive with respect to eligibility criteria. And in fact, if we designed clinical studies that were more pragmatic to the patients that we care for, that we'll end up with those cohorts over time that are more reasonably comparable if we look at a real-world experience compared to a clinical trials experience. So there is a lot that we're in the midst of learning that will sort of play out in the time that comes ahead of us.

    16:56 Alex

    Alex Gorstan This is a really interesting idea. Has the quest for scientific rigor led us to a place where we're collecting too much information or being too prescriptive about trial protocols and is this pandemic leading us to some realizations about that?

    17:09 Bobby

    Bobby Green And it feels like, I mean, Mike, you know this space much better than I do, but just like in routine clinical care, there are things that we do that are important, but there are also things that we do just because we've done them before, and we don't know how they're important. In sort of the routine of clinical trials, there are things that we do because it's important for patient safety, but then there may be all these other things that we do just because we've always done them and do we really need to? An example that sort of comes to mind is, in the midst of the pandemic, the FDA relaxed some of the requirements on, if you're on a clinical trial, you don't necessarily have to go to the site where you enrolled on the trial to get an assessment, you could go to a clinic closer to home to get lab work done or something like that because they didn't want people traveling. Couldn't that have happened sort of without the pandemic in a lot of circumstances, right?

    17:57 Nicole

    Nicole Mahoney That's what the hope is that some of these lessons learns will give the experience and confidence to change the way we approach clinical trials for drug development.

    18:06 Bobby

    Bobby Green Nicole, you posted an article internally, I think just this morning, talking about how sort of in the midst of everything that's going on, and work from home, and clinical trial accrual, which in some cases, unfortunately, has dropped off, that we can talk about, the FDA has continued to review oncology drugs and push through at breakneck speed on getting drugs approved. Just curious on your perspective there on what's been happening at the FDA with oncology drug approvals.

    18:37 Nicole

    Nicole Mahoney I think it's been remarkable. I don't think they've missed a beat, and I think they've pioneered a few different things, such as a virtual ODAC meeting, an advisory committee meeting. And I think that's a testament to their commitment to making sure that everyone is working towards getting patient access to effective therapies, when possible. We've certainly seen that in our collaboration with the FDA. We have a research collaboration with them. Not only are we keeping up on that collaboration and the joint work that we do, but we're talking about answering a couple of new projects. And so it's been fascinating. I think that they're running a tight ship. It's been very efficient. And as you saw on the article that I posted, the oncology folks are really being singled out, along with the antivirals group, as really over-performing and not missing any deadlines during this time period.

    19:33 Mike

    Mike Vasconcelles Yeah. I just want to echo that observation. I mean, it's nice to see the metrics coming forward, but sort of anecdotally, over the last few months, I've been candidly, repeatedly impressed on both fronts that you alluded to, Nicole.

    19:48 Bobby

    Bobby Green Yeah. I think in that space, and then you can sort of look in clinical care, both in the cancer community and the medical community in general, people sort of stepping up and realizing people still need care throughout this, whether it's people in intensive care units on ventilators with COVID or just cancer patients getting cared for in a community or academic practices across the country.

    I think the one area where there's been, my perspective, I'm curious your guys, a lot of criticism has been specifically around the ability and probably worse in the U.S. than other places, the ability to quickly run randomized clinical trials to answer important questions about COVID. Rob Califf, the former FDA commissioner has been out a lot saying it's just been a really lost opportunity that we can't, in these very large hospital systems where we have patients, run randomized trials. As you guys know, there were so many observational trials of hydroxychloroquine, thousands of patients treated off label, not that many patients treated on clinical trials. The Decadron study, which showed some benefit to dexamethazone was not run in the U.S. You know, I think Rob Califf would argue that we should have an infrastructure that allows us to do sort of lightweight, sort of pragmatic like trials in a crisis like this, and we just don't have it. Curious your take on that.

    21:11 Mike

    Mike Vasconcelles It's another opportunity in a way, right? I think probably the criticism has some validity, but then the question is how do you seize on that or build on that, to take advantage of ecosystems either in large academic medical centers or through other sort of consortiums that are built around research questions, hopefully not for the next pandemic, but for other sort of emerging sort of critical issues that inevitably will confront us in all kinds of ways.

    21:43 Nicole

    Nicole Mahoney Yeah. I think it's been very interesting. I listened to a podcast this morning where someone was talking about vaccine development and said, we're in the phase where the troops are storming the beach with all these different solutions. And it seems like that's what's happening in the U.S. So we have tons and tons of clinical trials getting off the ground. That's a good thing, you want multiple shots on goal. On the other hand, we probably don't have the central organizing force in the way that maybe some other countries have to kind of deploy all the resources in a more systematic way. So opportunity's there, but it also highlights some limitations, that we discussed before about our infrastructure for conducting studies.

    22:25 Bobby

    Bobby Green Yeah. It feels like it is an opportunity. I mean, there was just, you guys may have seen a STAT article by a Matthew Herper I think yesterday or the day before, and Dr. Califf was quoted. I just wrote it down here saying, "If the goal was to optimize the likelihood of figuring out the best treatment options, the system is off course." So I think we do have some work to do there.

    22:51 Alex

    Alex Gorstan Okay. Shifting gears a little bit. There's no doubt that clinics have been affected by COVID. We released data in the cancer letter back in May that brought this to light. Now we're six months into the pandemic. What's the new normal like for a clinician?

    23:05 Bobby

    Bobby Green I would say there are still challenges. It's not fun walking around an office all day wearing a mask, but you do it because that's the way to do it. I would say, and part of this is, from data that we've seen across our network and part of it is my own anecdotal experience still practicing, to a certain extent things have gotten remarkably back to normal. We've seen early on in March. No one really knew what to do, but practices seem to be pushing forward and coping pretty well.

    One of the things that's sort of been a nice surprise is we've not seen lots of clinicians get sick. We've not seen, to the best of my knowledge, lots of staff in the offices get sick or patients get infected from coming into the office. And again, I think clinics across the country are doing a good job of universal mask wearing, not, generally, keeping guests out. Yeah. I mean, I think, both in community and academics and hospital systems, we've seen things get back to some semblance of normal, which is a good thing, so.

    24:14 Mike

    Mike Vasconcelles How are you balancing delivery of care with anything structural around social distancing? I mean, are you trying to manage that proactively and in any sort of way with respect to scheduling or volume of people in the clinic at any one time, or are you just really relying on masks and delivering care as you normally would six months ago.

    24:38 Bobby

    Bobby Green So it's both. It's trying to keep people out of the waiting room and limiting visits, so the waiting room, which in pre-pandemic times can often be crowded. It's not crowded anymore. There's plenty of space in there. I mean, as you know, oncology patients tend to come in with lots of people with them and limiting that has, I think been impactful. Making sure that there's enough spacing in the infusion room. I think a lot of infusion rooms are set up that way to begin with, others are not, but practices have made attempts to increase the social distancing there.

    And there's a nontrivial percentage of patients who are still getting telehealth, which is dropping the volume of people coming into the office. I will say there are parts of a telehealth visit these days that feel more intimate than sitting in an exam room where both you and your patient are wearing masks and trying to maintain social distance as well. In fact, I've even had a couple patients where I've thought the physical exam isn't critical, where I've said, come in, get your labs, and then we can have a nice conversation over telehealth because it's honestly a much easier conversation. It's amazing how as you're talking to patients, when you're both wearing masks, like not having the visual cues of knowing if someone's smiling or not smiling is just some basic things like that are really, really hard, even in person.

    26:04 Nicole

    Nicole Mahoney I'm wondering how, as the only non-clinician here, how are practices using real-world data to inform their setups or what they do, or are they doing that?

    26:14 Bobby

    Bobby Green I would say, the areas where people have used real-world data has mostly been right now, Nicole, from an operational standpoint. And in aside from to the extent that real-world evidence has contributed to understanding how we treat COVID patients, I think there's been observational data that has shown that treating cancer patients during the pandemic has not been adverse, that you can safely get patients chemotherapy, and bad things aren't happening because of that. So I think that sort of informed the care.

    Other than that, I think it's mostly been around operational insights to practices. Like one of the things we've done is provided volume data, both to our individual practices and benchmarking so they can understand is the volume drop that they're seeing different than everywhere else, would help them operationally prepare. Mike, I don't know if you've seen anything different across academic centers or with anyone that you've interacted with.

    27:09 Mike

    Mike Vasconcelles No, I think it's similar, but I do wonder if we've seen real benefit derive from some of the network based sharing of data in the context of this situation. This goes to the future state, we're talking about the future state as it relates to an individual patient's care. But does your practice management organization, Bobby, are they yet clamoring for that network-wide perspective going forward on any kind of regular basis to help them manage their practices? Or is it-

    27:41 Bobby

    Bobby Green So I think one of the first and most notable things that happened at the beginning of the pandemic was people were craving understanding what their colleagues across the country was doing because there was panic and this was completely uncharted territory. So I think there was really this need to want to network and understand what was going around elsewhere, not just in your own practice. And I think that ultimately this only accelerates the need to do that, not just in the midst of a pandemic, but in general, I think there's value there.

    28:14 Mike

    Mike Vasconcelles So you do think it'll translate into some something that could be longitudinally beneficial to practices?

    28:20 Bobby

    Bobby Green I do. I think so. And I think it's... I mean the community oncology in general does do a pretty good job of networking and keeping in touch and sharing best practices. And I think that this serves to prove the value of that. And to the extent that organizations like ours and others can make it easier to share information and get insights from that. I think that's critically important.

    28:49 Mike

    Mike Vasconcelles Yeah.

    28:49 Bobby

    Bobby Green And listen, if there's a world where it was very easy to share, we're continuing to treat patients with XYZ and people are doing fine, we're continuing to do this and people were doing fine, that information ultimately is, as you say, Mike, looking at a future state would be super valuable, I think.

    29:13 Mike

    Mike Vasconcelles I'm just curious as we think about the pandemic and delivery of health care, Nicole, what keeps you up at night?

    29:21 Nicole

    Nicole Mahoney So that's a great question. And I think what keeps me up at night regarding real-world evidence in terms of the information we're producing is the issue about trust and credibility. And I think that keeps other organizations up at night too. And I say that because there's this huge need for information and for getting information quickly. And the flip side of putting out research papers to answer questions very quickly is that maybe they're not thoroughly vetted. And I think we're at a critical juncture where if we want to increase the credibility and trust in real-world data, we have to make sure that as community we're publishing reliable results. And I think what keeps me up at night is we may not have best practices down that everyone agrees with, but the good news is I see that coming.

    So since the pandemic started, the Reagan-Udall Foundation, on behalf of FD1A, has put together what they call a COVID evidence accelerator. And it's a huge multi-stakeholder group of data organizations, drug companies, academics, all talking every single week about how we can use real-world data to answer critical questions in the time of COVID. And just last week, there was a whole discussion about best practices to increase trust in real-world data. And I think that we'll have to follow through on those, but that is what keeps me up.

    30:49 Mike

    Mike Vasconcelles Bobby, what keeps you up?

    30:51 Bobby

    Bobby Green Other than that bars are open and people are still going to bars, and it will play out and we will eventually learn what the impact was. But I do worry that things that happened specifically around delay in certain screening or people delaying in seeking care in March and April, not just for cancer related things, but for all things that we're going to find out that, that had a big impact on mortality and outcomes that we are going to recognize down the road. And it's just going to become part of the impact in lives loss due to COVID. I mean, you've seen the data. The number of appendectomies went down, the number of heart attacks went down, the number of strokes went down. Well, they didn't go down, people just didn't get them taken care of. And ultimately what's going to be the impact of that. How about you, Mike?

    31:46 Mike

    Mike Vasconcelles I think what I'd add to that is just the challenge that it looks like we're going to continue to face, in the US in comparison to other parts of the world, that I think really speaks to some really interesting cultural and maybe structural issues, we alluded to some of this earlier with respect to the nimbleness with which we were able to respond cohesively to critical clinical research questions, maybe impeded by some of the structures that exist here. And I think we're seeing that play out in terms of the way in which we're managing through the pandemic in a way the individualism and entrepreneurial- ism, that's the bedrock of our society and our culture think probably having a more complex effect as it relates to cohesive response to a very pervasive issue. And we're seeing that play out in all kinds of ways. And we're going to struggle because of it over the next bit of time, but we'll get through it. I have no doubt about that. We'll get through it.

    32:53 Bobby

    Bobby Green Well said. Thanks, Mike.

    33:05 Alex

    Alex Gorstan Thanks for listening. To learn more at the intersection of real-world data, oncology and health policy, visit rwe.flatiron.com.