Objective Conformance checking in healthcare aims to verify whether patient care traces comply with clinical guidelines, but it typically requires formal, machine-interpretable guideline models, such as Computer-Interpretable Guidelines (CIGs), that are rarely available in practice.

Methods This work presents a modular framework that leverages orchestrated Large Language Models (LLMs) to support medical conformance checking directly from unstructured data, without the need for CIGs. The proposed architecture combines multiple LLMs and auxiliary components to extract patient traces from clinical discharge letters, derive normative rules from textual clinical guidelines, formalize rules into executable scripts, and compute a Trace Conformance Indicator that quantifies conformance on the event log.

Results The framework has been implemented and evaluated in the stroke care domain at the neurological ward of Alessandria hospital, where hundreds of patient traces were automatically extracted from hospital data. In particular, most of the available traces proved to be conformant to the 50 rules derived from the guideline. Conclusion Our work has demonstrated the feasibility of LLM orchestration in our domain, while, at the same time, verifying the good rule conformance checking results in Alessandria.