Prescription Errors: The Role of Language Models in Patient Safety in an Expert Evaluation Study

Abstract

Objective: This study investigates the potential of Large Language Models (LLMs) to support medical prescription processes and enhance patient safety. Methods: Six LLMs answered four prescription-related questions on contraindications, drug interactions, and dosage. A panel of 34 physicians blindly evaluated 24 responses based on consistency, focus, coherence, completeness, and detail. Results: LLM performance varied by criteria and question type; LLM6 excelled in completeness and detail, especially in complex cases. Simpler questions, like contraindications, scored higher overall, while complex queries showed more variation. Conclusion: LLMs show promise as digital assistants in prescription tasks, improving access to medical info and reducing errors. However, reliability depends on question complexity. They should support, not replace, clinical judgment and require ongoing validation for healthcare adoption.