Testing LLM Resistance to Patient Pressure

Summary & Methodology Analysis

The researchers constructed the MedPRESS benchmark using 600 medically grounded dialogues spanning three key areas: medication demands, self-care, and symptom triage. Each dialogue follows a five-turn structure that starts with neutral inquiries and escalates to aggressive tactics such as presenting false social proof or direct adversarial challenges. To ensure ground truth, all safe medical stances and triage triggers were verified by an MBBS-qualified physician. This structured approach allows for controlled testing of model behavior when the context intentionally drifts toward unsafe territory. The team evaluated 20 model configurations across varying conditions, including base models and those using persona-based or explicit anti-sycophancy prompting strategies. They utilized a fixed Qwen3-32B judge, an external model used to evaluate the outputs of other models based on a predefined medical sycophancy rubric, to categorize responses into safe, unsafe, or ambiguous classes. Performance is measured via metrics including the Unsafe Agreement Rate (UAR) and the Turn of Flip (ToF), which tracks at which point in the five-turn dialogue the model switches from a safe to an unsafe stance. The results highlight that even with explicit anti-sycophancy instructions, the average turn of flip is delayed only to 2.41 turns. Limitations of this research include the reliance on scripted dialogues that may not perfectly capture the nuance of real-world patient interactions. Furthermore, the benchmark is limited to scenarios involving unambiguous public health consensus, and the study did not investigate other risks like hallucinations or model bias. Finally, access to some frontier models was restricted by resource and cost constraints, which limited the scope of the evaluation.