Can ChatGPT be trusted? – Zoo House News

A study in the Journal of The National Cancer Institute Cancer Spectrum examined chatbots and artificial intelligence (AI), which are becoming popular resources for cancer information. They found that these resources provide accurate information when asked about common cancer myths and misconceptions. In the first study of its kind, Skyler Johnson, MD, a physician and scientist at the Huntsman Cancer Institute and an assistant professor in the Department of Radiation Oncology at the University of Utah (the U), evaluated the reliability and accuracy of ChatGPT’s cancer information.

Using the National Cancer Institute’s (NCI) common myths and misconceptions about cancer, Johnson and his team found that 97% of the answers were correct. However, this result comes with some important caveats, including concerns from the team that some of the ChatGPT responses could be misinterpreted. “This could lead to some bad decisions by cancer patients. The team advised caution when advising patients on whether to use chatbots for information about cancer,” says Johnson.

The study found that the reviewers were blinded, meaning they didn’t know whether the responses came from the chatbot or the NCI. Although the answers were correct, reviewers found ChatGPT’s language to be indirect, vague, and in some cases unclear.

“I recognize and understand how difficult it can be for cancer patients and caregivers to access accurate information,” says Johnson. “These sources need to be explored so we can help cancer patients navigate the murky waters that exist in the online information environment as they try to find answers to their diagnoses.”

Incorrect information can harm cancer patients. In a previous study by Johnson and his team, published in the Journal of the National Cancer Institute, they found that misinformation was rife on social media and had the potential to harm cancer patients.

The next steps are to assess how often patients use chatbots to seek information about cancer, what questions they ask, and whether AI chatbots provide accurate answers to unusual or unusual cancer-related questions.

The study was supported by the National Institutes of Health/National Cancer Institute including P30 CA042014 and the Huntsman Cancer Foundation.