A 4-month-old child comes to the clinic with fever and nasal congestion. Vyonne Njeri thinks it’s just a cold. Then a yellow box appears on your computer telling you to check your heart, because your heart rate is elevated. Njeri is a registered clinical officer in Nairobi, Kenya; she sees patients alone like a nurse
A 4-month-old child comes to the clinic with fever and nasal congestion. Vyonne Njeri thinks it’s just a cold. Then a yellow box appears on your computer telling you to check your heart, because your heart rate is elevated. Njeri is a registered clinical officer in Nairobi, Kenya; she sees patients alone like a nurse practitioner. When you listen with a stethoscope, you hear a whistling sound, a sign that the baby might have a congenital heart defect.
“That’s something I would have missed any other day,” Njeri says of the visit a few months ago. “That kid would have just gone home.” She credits an AI tool that double-checks her work for helping her. Njeri referred him to a specialist who confirmed the diagnosis and prescribed medication; The child may need to have surgery.
The tool, called AI Consult, was tested in a randomized trial of nearly 10,000 patient encounters at 16 Kenyan primary clinics operated by Penda Health, and the results were published this summer in Nature medicine. Half of the clinical officers wrote their notes on a computer with OpenAI’s GPT-4o reviewing their electronic notes in the background.
GPT-4o is a large language model, or LLM, the type of AI that powers chatbots like ChatGPT. They have been trained largely on the Internet and can generate text in response to instructions.
The other half used a computer to take notes without the artificial intelligence tool.
This is how the AI program works. It provides three indications: green, yellow and red, like a traffic light. The AI looks for possible problems in a patient’s care or gaps in notes. Green means everything is fine. Yellow means “click me”: The AI found a small problem in the notes and offers some guidance. Red appears if the AI finds a critical concern based on the notes and alerts the doctor, who may need to act promptly.
For Njeri, these interventions offered welcome feedback: “You need to check it out” or “you’re on the right track.”
An independent panel of six Kenyan family doctors graded the notes. They felt that doctors for whom AI Consult served as what Njeri calls a “second pair of eyes” produced better diagnoses and treatment plans.
And it costs 4 cents per patient.
The study captures both the potential of AI to improve healthcare and the challenge of demonstrating it.
But while healthcare workers found this AI screening useful, the question of benefit to the patient remained unanswered. The study found no evidence of improved outcomes.
There was a 23% decrease in treatment failures, such as death or unresolved symptoms, but the decrease was not statistically significant because the number of those cases was low to begin with.
Therapeutic failures “are too rare in primary care,” he says Dr. Bilal Mateenco-author of the study and director of AI at PATH, the global health nonprofit that sponsored the trial. A trial would need about 139,000 people to detect a significant difference, he says.
The trial was funded by the Gates Foundation, which provides financial support to NPR for its global health team; NPR is solely responsible for all content.
However, in the continued exploration of the role AI can play in healthcare, particularly in lower-resource countries, there is praise for the study. Dr. Jonathan ChenAssociate professor of medicine and biomedical data science at Stanford University, who was not involved in the study, says, “This is an important study as it goes beyond simply running mock tests.”[s] with AI systems.” The study is one of the first trials to compare the role of this type of AI as an aid to primary care.
Mateen says that prompting health care providers with what he calls “information as intervention” will have a “significant incremental effect.”
Njeri shares that sentiment: “It helps us remember protocols and new guidelines,” he says. In a country where health resources are not comparable to those of wealthier nations, she says her team can care for five or six patients an hour with a wide range of conditions, often without specialists to support them. “Having this tool really adds value,” he says.
As for AI recommendations, Njeri says he finds them useful about half the time; The other half of the recommendations are not as helpful, but they are rarely wrong.
“But I still feel like it’s nice to have, to give you that second thumbs up or thumbs down, like having a superior telling you, ‘you could do better there’ or ‘you’re doing well,'” he says. The expert panel rated most of the AI recommendations as safe and appropriate.
Mateen says he is working to roll out the tool globally.
The article didn’t investigate whether AI could increase access to care, but Chen is very excited about that possibility. As AI improves, he says it could actually create a draft to edit the healthcare worker’s notes, which could allow them to save time so they can see more patients. Or the AI tool could even interact with the patients themselves.
According to him, “timely and consistent access to healthcare” is where AI can have the greatest impact.
Mateen says he is working to bring this tool to health systems around the world. But there are still concerns about nascent artificial intelligence tools. Healthcare AI researcher Dr. Nicolás OkumuAn orthopedic surgeon at Kenyatta National Hospital who was not involved in the study fears that new AI systems could make mistakes or give bad advice, leading to serious errors. It warns that “even approved AI can cause harm, so oversight must remain active.”
Keep following us for the latest insights.
