The hallucination problem in large models can have serious consequences in medical settings; AI outputs must undergo clinical validation and cannot replace physicians' professional judgment.