Large language models (LLMs) like ChatGPT are making waves in the medical field, with one in 10 doctors using them in their practices and even patients turning to them for diagnoses. This rapid adoption comes despite concerns about the safety and effectiveness of generative AI (GenAI), with even the FDA struggling to regulate these technologies. An evaluation of LLMs shows that while they can generate responses, the ability to substantiate claims with reliable sources is lacking. The models often fail to provide valid sources, particularly for more lay-oriented inquiries, highlighting the need for further research and regulation in the field.
https://hai.stanford.edu/news/generating-medical-errors-genai-and-erroneous-medical-references