Season 8 Episode 5: Health Policy and AI
Season 8 Episode 5 [download this episode (MP3, 34.7 MB)]
In this episode Elizabeth chats with Kumanan Wilson, Chief Scientific Officer at Bruyere Health Research Institute, internal medicine physician at the Ottawa Hospital, and co-founder of Canimmunize, at unpack AI and health. They discuss tools like AI scribes. And personalized decision aids could ease clinicians' workloads and improve care. Kumanan shares his research on the value of Canadian health data, why data sovereignty matters amid US laws like the Cloud Act, and how middle-power collaboration could help. He argues policymakers should focus on capturing the economic opportunity of Canadian innovation.
Additional Resources:
Check out Kumanan’s other work on Google Scholar!
To learn more about AI scribes being used in health care settings, read AI Scribes Move Quickly Into Canadian Care Settings.
Interested in the study about how open models were better than the health specific ones, read General-purpose large language models outperform specialized clinical AI tools on medical benchmarks in Nature Medicine.
Read Amazon Web Services if you are interested in cloud commuting services.
Interested in Michael Geist work, check out his website.
To learn more about EMRs check out Electronic Medical Records (EMRs) Explained by Oracle Health, formerly Cerner.
Read Microsoft tells French lawmakers it can't protect user data from US demands to learn how this case requires Canada to protect our digital sovereignty as it can be subjected to US laws.
To learn more about the United States efforts to halt foreign adversaries' AI capabilities, read Anthropic disables top-tier AI models after US order limiting foreign access.
Check out Mark Carney;s transcribed Davos speech, if you are Interested in how middle power collaboration is strategically used to get around USs restrictions of AI and their access to our health data.
Find Kumanan on LinkedIn!
Contributors:
Host: Elizabeth Dubois
Research Lead: Julianne Bernier Robert
Transcript: Areej El-Sharif
Audio Editing: Sarah Pledge-Dickson
Translation: Julianne Bernier Robert
Publishing and Promotion: Aaron Brisson
Producer: Sarah Pledge-Dickson
Episode Transcript: Season 8 Episode 5: Health Policy and AI
Read the transcript below or download a copy in the language of your choice:
Elizabeth Dubois: [00:00:04] Welcome to Wonks and War Rooms, where political communication theory meets on the ground strategy. I'm your host, Elizabeth Dubois. I'm an associate professor and university research chair in Politics, Communication and Technology at the University of Ottawa. My pronouns are she/her. This season, we're talking about social and policy issues surrounding uses and governance of artificial intelligence. And today we're talking to Dr. Kumanan Wilson about AI and health. Kumanan, can you introduce yourself please?
Kumanan Wilson: [00:00:29] Sure. I have several hats. I'm the Chief Executive Officer [and] Chief Scientific Officer at Bruyère Health Research Institute. I'm also an academic and clinical research chair at the Department of Medicine in Digital Health Innovation. I'm a practicing internal medicine physician at the Ottawa Hospital, and I'm the co-founder of a digital health company called Canimmunize.
Elizabeth Dubois: [00:00:52] Thank you so much. I'm really excited to have you on the episode today. Obviously, health and AI is a very, very big topic. And with the many hats you do wear, I think you're going to have some really interesting perspectives here. I thought we'd maybe get started by getting us all on the same footing for a moment. When I say health and AI, it could be a whole bunch of different things. What do you think of when I say health and AI?
Kumanan Wilson: [00:01:17] I mean, first, I'm excited by the developments. They've been talking about this for a while and I was getting a little skeptical. [...] I thought [whether] it was just hype or is it real? And then I think when ChatGPT came on board, people started to say, “Oh, wow, this is real.” You know, in the health context is how do we use our data to improve care and use it in a real time way. So we don't do a randomized trial and wait five years, but where we can actually use data as it emerges to continually improve our health system. And I think some very practical examples are at the clinical level, the scribe functionality, where it can actually listen to the conversations and provide the clinic notes. And that's been incredibly helpful for primary care providers in particular, and reducing time to write notes and allow them to actually engage with their patients more directly. But moving beyond that, I think it's going to be very useful for clinical decision aids. It's going to help us make diagnoses. It's going to help guide management. I think it'll be a second opinion in many respects when we provide care.
Elizabeth Dubois: [00:02:25] That's really interesting. And in thinking about the kind of vast array of different uses is interesting all the way from a scribe to helping with diagnostic. That's quite a big range. Where does something like patients using AI tools come in to your thinking? So you mentioned ChatGPT off the bat. There's been a whole lot of, “Oh yeah, people are using ChatGPT to figure out health issues and getting it really wrong.” Where does that part come in?
Kumanan Wilson: [00:02:53] Well, I admit, I mean, just like everybody else on the planet. I use it for personal health issues. So for example, if you have a musculoskeletal problem, I've actually found it incredibly helpful. Is it accurate? I don't know, but [...] what I noticed is it's very tailored to you as an individual. And I think that's the paradigm shift we're seeing. So randomized controlled trials are the gold standard. But you know, I call it the tyranny of the majority. If the trial says it's positive, we assume it works for everyone. I know we can do subgroup analysis. If it says a negative, we assume it doesn't work for everyone. And we know that's not the case. There's some people things may work for, some people they may not. And we need to have a lot of questioning and maybe further more analysis of the data. I come at this from a vaccination perspective, or we know vaccination is incredibly effective and beneficial and safe, but there are subgroups where we do see harm. We often don't pick them up in the clinical trial because they're so rare, but at a population level, they become relevant. And we actually we saw during Covid, young men got myocarditis. Young women got the thrombotic condition. This is where I think AI starts to become really useful. Like, are there specifics about you that might change how the treatment works for you? And we know with longevity and everything, this is how everybody is thinking. So I think that's where I see real exciting opportunity. Now we have to evaluate it and make sure it works. Of course.
Elizabeth Dubois: [00:04:16] Yeah. And so there's this potential for personalization, both in terms of the kind of personalization that we think of with these conversational agents of they're going to speak in a vernacular and a language that works with you because they've interacted with you before, but even kind of a step back at the level of how do we tailor diagnostic and then treatment for people, because you don't necessarily need clinical trials on every rare condition for a particular vaccination or treatment. Is that what I'm hearing?
Kumanan Wilson: [00:04:47] Yeah. And we may not be able to answer these questions at that individual level [which] requires a lot of computing power, and as a doctor, you only have so much computing power in your head and also you fatigue and these things don't fatigue and these things don't get frustrated. So I think there's all of that and the ability to remember your conversations and come back to them. Humans are great, but a little help is welcome.
Elizabeth Dubois: [00:05:13] Yeah, that makes a lot of sense. I have a friend who's a family doc, and I remember her talking to me about using one of these scribe tools being like, “For the life of me, no matter what, I can never remember if a patient comes in with left or right knee pain. And like I knew it at the beginning of the appointment and they said it multiple times and just by the end of the appointment, I just can never remember.” It's not because this family doctor doesn't know left from right. It's because our brains are not very good at all of the little details 100% of the time when you're trying to pull it all in together.
Kumanan Wilson: [00:05:46] And should we be, you know, should we be freeing our brains up to do the higher level thinking, the more integrative thinking and not focusing on grunt work, so to speak. Let the AI do the grunt work. I mean, I think the AI I can do higher level thinking as well, but you can begin by just doing the stuff that unfortunately, has taken over a lot of our lives and doesn't allow us to do why we decided to become doctors in the first place.
Elizabeth Dubois: [00:06:10] Right, right. And so I know that one of the concerns with using these kinds of tools is the capacity for hallucination and the capacity to just get things wrong. But I also know that often health AI tools are designed on more specific data sets. Can you talk a little bit more about the use of health data to develop these health tools, and whether or not that helps prevent some of the risks we know, particularly generative AI produces?
Kumanan Wilson: [00:06:41] Yeah, again, we're in early stages and I like to say humans hallucinate as well. We have our own cognitive biases. And there's been a lot of work done on that. The interesting study, I think, was in Nature [Medicine] that looked at the health specific AIs and then looked at the frontier models, the open models, and they actually found the open models were better than the health specific ones. Now there's people [who] have pushed back on that. There's a lot of money in this area. I think that's philosophically interesting, and it sort of points to the fact that just by pulling on all the data, are there associations and things we just don't know are relevant, that are important? And sort of like the senior clinician just seems to know the answer because they just have absorbed so much information over their years, they may not even be able to explain how they came up with the answer. You know, I remember early on in my training, the senior – fantastic senior clinician was great, but he couldn't necessarily explain how he would come up with his diagnoses because it was just obvious to him because he pulled information from so many different places. You know, I think maybe the broader models may actually be better than the narrow model. I think that's something we're gonna have to look at very carefully.
Elizabeth Dubois: [00:07:52] That's super interesting. I didn't know about that, but we'll try and find that Nature [Medicine] piece and link it in the show notes too. I want to talk now a little more about some of the stuff that you've been talking about. You've done some some writing on this, both academic publishing and more publicly. And it's about the value of our data, particularly Canadian healthcare data. And I've seen you write, you know, “AI is making our healthcare data potentially even more economically valuable,” which then presents potential security risks and risks for surveillance. And there's a whole bunch of geopolitical stuff that starts to come in before we get too much into the geopolitical conflicts or anything like that. Can you just talk a little bit about what the value of Canadian health data is?
Kumanan Wilson: [00:08:36] Sure. Actually, I'm going to jump into the geopolitical.
Elizabeth Dubois: [00:08:38] All right.
Kumanan Wilson: [00:08:39] None of this would be a concern, you know pre the recent US elections. But now it has to be taken as a major concern. I mean, the good news story from a Canadian perspective are two fundamental Canadian values are going to benefit us from a health perspective and economic perspective even without that being the intent. And that is a single payer system and an immigration policy. So we have population based data that looks like the world, and that's unique. Not many places have that and that's incredibly valuable, particularly when you're building out these models. You know, Denmark's got great data. They don't have as many people as us, and they're not nearly as diverse as us. So that makes our data valuable. And it positions us to lead in this area. And we need to look at places to lead in this geopolitical environment. We need to build domestic economies. And those tax revenues go back into our health system. But that data is valuable to other countries. And the US has been incredibly effective at innovation and entrepreneurship and capturing markets. And our data could be valuable to them economically. You know, it may also be valuable to them from [a] surveillance perspective. And that's what the USA Patriot Act and maybe the CLOUD Act have highlighted as potential risks. And do we want our data to be used to screen people at borders as well? I mean, there are two different risks. I don't want them to take our data and make money off of it, but that actually may not be terrible if it improves care here. I definitely don't want them to use it for surveillance.
Elizabeth Dubois: [00:10:06] Right. Yeah. And it's not just health data. There's all kinds of different data that's collected and used and could be used for surveillance, even if initially it's been collected and used for something else. There's this risk that connects with privacy concerns, I think, even beyond health data. But I would say as an individual, my health data, getting used for those kinds of things is among the scariest kinds of my personal data.
Kumanan Wilson: [00:10:33] I would argue the scariest is nobody uses your data. So as much as I hate the idea of US companies using our data to build algorithms, I'd rather them do it then nobody does it. I don't want us to import algorithms built on other population data into Canada. That just won't work here. Because we're different people. We want to have data, algorithms and solutions built on our population. So what I mean, in my many years doing this, either both in the business side and the academic side, the biggest problem in Canada is we don't use our data like, you know, we built Canimmunize. And one of the concerns I remember doing an early interview with CBC said, “Well, what if people, you know, what's securing the data? What [if] do people [got] your immunization data? And the answer everybody told me “that we want someone to have our immunization data.” Like the concern isn't that someone's not going to, it's going to have your data when you don't want them to, is that nobody has your data when you need them to have it. So I think actually that's the bigger issue. But obviously privacy, security are very important concerns. I think we don't want [it] used for surveillance or insurance purposes, but we also want them to be used to create solutions. And there is an important study in the New England Journal that looked at the stroke registry, [where] you could opt out. The people that opted in weren't representative, they were richer, they were more homogenous, [and] I think they were whiter. And the risk of that policy is then [...] if you're going to apply something based on that to a broader population, it may not work on, you know, at risk or marginalized populations. So we need to use our data to help improve our care, and we need to use all of it.
Elizabeth Dubois: [00:12:11] Right. That makes a lot of sense. I hadn't really thought of it from that perspective, but I definitely do want somebody who is trustworthy. I say somebody, I mean, it's actually like lots of people. And there are almost certainly going to be commercial interests involved in that. Which then brings me to the next question, which is around data sovereignty. We keep hearing about data sovereignty, but I haven't heard it particularly well defined. I'm wondering if you could just give the short definition that you use when you're thinking about what data sovereignty is.
Kumanan Wilson: [00:12:45] It's that the data that's developed in this country is subject to the laws of this country. It's pretty simple where this really emerged, and I'm putting on my company hat, or I wasn't a company at the time. When we built Canimmunize, we were, I think, one of the first health tech solutions in Canada that used the AWS Amazon Web Services server that was located in Canada. They built one and [...] they actually wrote a case study. They were great to work with, great company. But the idea was that because the USA Patriot Act that if it was housed in Canada, it would not be subject to the act. And that made no sense to me because it's still a US company that owns your data. And then after the recent, you know, issues that have happened with the US, I called up my legal colleague, Michael Geist, and he said, “Yes, that makes no sense” and he had written a paper about it around the time. And now the CLOUD Act has made that even more of an issue that the US can force its companies to hand data over. And Microsoft testified in France that if data held on French soil, if the US government requests it, they will have to transfer it. Not great testimony for them, but it's a truthful testimony that has spurred Europe, Canada, a lot of places to say, well, how do we make sure our data is protected and secure and subject to our laws?
Elizabeth Dubois: [00:14:04] Right. So protected, secure, subject to our laws, which we used to think kind of meant as long as it's housed in Canada, that's enough. But now it's more than just the data residing in Canada. It's also the companies need to be Canadian. What does that look like?
Kumanan Wilson: [00:14:21] We've suggested a few options. So I think encryption is necessary. I thought that would be good enough encryption by design. But apparently if you're manipulating data in the cloud, it has to be encrypted and snapshots can get taken. So that's why that alone doesn't solve it, I think. Data localization helps. So that's an important part of it. My colleague Michael suggested that we put in a blocking statute into [Personal Information Protection and Electronic Documents Act] Pipeda, which is the privacy legislation for industry, and that would be analogous to what Switzerland does. They get a lot of requests for their banking data, and it creates a conflict of law, or at least become aware that people are going after your data and you can negotiate. Those are interim measures. Ultimately, we want to have sovereign Canadian cloud servers. That's easier said than done. The cloud giants are cloud giants for a reason. They have functionality and capacity that takes a while to build. We need to move in that direction. Potentially, we could license their technology to a Canadian company. That could be an option to at least start leapfrog the process a bit. [...] there are some Canadian cloud companies that could be supported to grow. I think that's ultimately what I'd love to see happen. There are other solutions like Air gap solutions, which is like actually going back in time to like the 1990s when we didn't have Wi-Fi. And your system is actually disconnected so that you have control. And even if they ask for it, the US government asks for it, they can't take it. So that's also a potential temporary measure.
Elizabeth Dubois: [00:15:49] Yeah, there's a lot of steps. And as we've said already, a lot of this is evolving and shifting. And so it's going to come up against a bunch of other things. You know, we have an episode in this series about AI and the environment and thinking about, okay, we need more cloud power in Canada, which means we need more servers in Canada, which means there's more environmental impacts. And so it will be interesting how this all develops, thinking about all the various ways that having the infrastructure for AI is going to impact.
Kumanan Wilson: [00:16:21] And, you know, and a new consideration, there always seems to be a new consideration is what we saw in the US government do recently with Anthropic, where they cut off access to foreign entities. Do we want to make ourselves that vulnerable in our health system? I mean, I think this speaks to a much, much broader issue that we have three major EMR servers, and they're all American, and we have three major cloud servers, and they're American.
Elizabeth Dubois: [00:16:46] What's EMR.
Kumanan Wilson: [00:16:48] Electronic medical records. So we at Epic [Systems], MEDITECH, and [Oracle Health, formally] Cerner are the three major ones that support most of the hospitals. Then we have three major cloud servers Microsoft, AWS, and Google. We wouldn't have considered this, but is it possible that the US government threatens to cut off some of these functionalities to put pressure on us? You never know. And do we want to even consider that remote possibility and protect ourselves and have some redundancies or safeguards in place just in case? I think we, unfortunately, are at a point in time that we need to take that seriously.
Elizabeth Dubois: [00:17:25] Yeah, that makes sense. One of the other things that I've read in your work was this idea of Canada and other middle power countries banding together, collaborating to ensure that we can, you know, minimize some of the risks that we're talking about, but still capitalize on the opportunities that this health data presents. Can you talk a little more about what that looks like?
Kumanan Wilson: [00:17:46] Absolutely. So I think it's to the point, do we want to be vulnerable to sort of what I've discussed, where the major powers. I mean, it’s China and the US are really leading in this area right now, have that kind of control and over your systems to picking up on the Davos speech. Middle power collaboration is sort of our way around it and data the logical place to go. And I think [...] some place that Canada could lead, given our expertise in health data and more data is better, and we can collaborate with other middle powers and share data, support them in developing a data infrastructure, learn from them, create algorithms based on the shared data, create AI based on the shared data. That could be a way of providing a further degree of sovereignty and security for our populations.
Elizabeth Dubois: [00:18:36] How similarly, or I guess differently structured, are our health data compared to these other potential collaborators? Like how difficult will it be to actually do that collaboration?
Kumanan Wilson: [00:18:48] Well, we have that problem within Canada alone. You know, when we built Canimmunize, we weren't building one app. We were building multiple apps because every province had its own schedule, its own way of coding data, its own vaccine information. It was basically a multinational app. So even within Canada, we have this problem within a province. We have this problem, like in Ontario, the primary care electronic medical record providers don't necessarily connect. We have to solve our own issues first and then work globally and I think if we can solve it here in Canada, I think the rest will be relatively easy. We'll have the roadmap and we're a federal nation and a very, actually very confederal nation in many ways. So that creates unique issues. But I think that a lot of expertise has been developed as we've grappled with these issues.
Elizabeth Dubois: [00:19:38] Yeah, yeah, that makes sense. And as you go through that process of trying to join big data sets, you figure out all of the problems you didn't even know existed, at least outside of health. In political communication research, where I am, that is always the case. It does make me think then, of another relatively unique context that we're faced with when we're thinking about the data of people here in Canada, and that's Indigenous data sovereignty and the Indigenous governance principles that many argue really should be integrated into how we develop our approaches to not just health AI, but AI more broadly. How do you see that fitting into this vision of, okay, Canada collaborating with these other middle powers, trying to create these opportunities that also kind of rebuff risks.
Kumanan Wilson: [00:20:26] Actual writing a grant on that right now. And the idea is obviously we need to protect specific groups. And Indigenous populations are one. But perhaps more importantly, we could learn from them. They've been thinking about this stuff a lot longer than we have worlds all of a sudden woken up to this risk. And they've been thinking about this for a while and developing principles that can protect their data, make sure it's used fairly, and we can perhaps take some of those principles and use them more broadly in Canada, either for other at risk population or for the country in general, because we're an at risk population right now. So I think that's important. I also think it's important that we don't forget, and I come back to this, that their data needs to be used to improve care. And we don't just say, well, we're not going to do it because it's a hassle. We need to make use of people's data, you know, vulnerable populations data, because that will improve care for them.
Elizabeth Dubois: [00:21:26] Yeah. I'll take this opportunity to mention that later in the season, we're going to have an episode with Jonathan Dore, who is the CEO of the First Nations Information Governance Center, on to talk more about that. So I'd like to turn now to questions about how the Canadian government so far has been dealing with health AI. One of the big things that we've heard from the Canadian government is, you know, it's all about having trust among the population in these AI tools and in their development. I guess first question is like, is that the right angle for us to take? Is that the thing we should be focusing on, the level of trust that Canadians have? If so, how is that actually built and maintained? If that's not the right angle, what do you think we should be thinking about?
Kumanan Wilson: [00:22:11] Yeah. They say the US leads the world in innovation and Europe leads the world in regulation. And we should strive to be in between. But if I were on one side right now, it's the innovation side. Canada is amazing at discovering. I would say we're the most entrepreneurial country in the world. AI came from Canadian inventors. Moderna was co-founded by a Canadian. GLP1 was discovered by a Canadian. Like just the list is endless. Anthropic was co-founded by Canadian. You know, Grok. I mean, if you want to claim Elon Musk, the Canadian. He's a Canadian. Open AI was co-founded by a Canadian. It's just like the list is endless. We just don't capture the economic opportunity. And that's no longer a nice-to-have that's a must-have. We now know how important it is. We need to develop our own economy. We need to capture the value of our own talent and our own discovery. So I would say the number one issue right now, we need to capture this economic opportunity. That's how we get sovereignty. That's how we get better health outcomes. That's how we get tax revenue that will fund our health care system.
Elizabeth Dubois: [00:23:08] Thank you. Yeah, that's a very convincing argument. We are coming up to time. So I wanted to end off with the final question. And that is from your perspective working in health and AI I and health policy. What is it that you think most policy makers are missing or maybe need to be reminded of?
Kumanan Wilson: [00:23:29] I think it would come back to – we need to figure out how we capture the economic opportunity here. How do we support Canadian controlled private corporations? That's how we're going to solve the sovereignty issue. Trust is obviously critical. We can hit a roadblock one skipping the road. And everybody may, you know, bring this to a halt. So that's a necessary part of the economic strategy. But I think that's exciting. I think Canada can lead the world in this. We can lead the world with Canadian values, building strong economy domestically, you know, leading a middle power lines globally and bringing it with a Canadian frame of mind. And I think people are looking for that right now. They're looking for a country with our values to help grow economies and trade with.
Elizabeth Dubois: [00:24:14] Perfect. Thank you. This has been a really interesting conversation. I really appreciate you taking the time. So thanks again.
Kumanan Wilson: [00:24:20] Thanks for having me on.
Elizabeth Dubois: [00:24:21] Thanks for listening. That was our episode on AI and health. I hope you enjoyed it. As always, we've got a whole bunch of links to different resources in the show notes, and you can go over to polcommtech.com for annotated transcripts in English and French that link to a bunch more resources and further reading. This season of Wonks and War Rooms is supported in part by the University of Ottawa's Office of Vice President, Research and Innovation, my University Research Chair in politics, Communication and Technology, the Faculty of Arts and the center for law, Technology and Society. I also want to acknowledge that in each of our episodes this season, I'm recording from the traditional and unceded territory of the Algonquin people, and I want to pay respects to the Algonquin people, acknowledging their long standing relationship with this unceded territory.
Comments