Large language models (LLMs) can store and recall large amounts of medical information, but their ability to process this information in rational ways remains variable. A new study led by investigators at Mass General Brigham has demonstrated a vulnerability in which LLMs are designed to be flattered, or overly helpful and agreeable, causing them to fail to appropriately challenge illogical medical questions despite having the information needed to do so.
Large language models prioritize helpfulness over accuracy in medical contexts, finds study

