Structured Annotation and Information Extraction of Epilepsy Clinical Texts

Objective To develop a fine-grained, highly comprehensive Chinese clinical texts named entity annotation schema tailored to the needs of epilepsy specialty clinical practice and research, and to validate its effectiveness in named entity recognition (NER) tasks.

Methods A three-level annotation schema covering 25 entity types was designed across seven major dimensions, including disease, disease course timeline, clinical manifestations, medical examinations, non-pharmacological treatments, medication, and influencing factors, with explicit label boundary definitions and rules for handling special expressions. De-identified inpatient records of epilepsy patients admitted to West China Hospital, Sichuan University from 2009 to 2023 served as the data source. Three annotators with epilepsy clinical backgrounds completed high-quality annotation of 804 cases, with annotation quality ensured through double annotation, expert arbitration, and entity-level inter-annotator agreement (IAA) evaluation. NER performance was validated using 10 model combinations comprising five Chinese medical pre-trained language models (Base-BERT, chinese-bert, chinese-Roberta, MC-BERT, MedBERT) paired with two sequence labeling frameworks (BiLSTM-CRF and GlobalPointer), with additional cross-domain generalization evaluation on an external case dataset established on the basis of published literature.

Results The final corpus contains 25 categories of epilepsy-related entities, 804 annotated cases, and a total of 28 400 entities. The IAA among the three annotators ranged from 0.86 to 0.88, indicating annotation consistency that met the accepted standards in computational linguistics. In NER validation, the GlobalPointer framework outperformed BiLSTM-CRF, achieving an overall Micro-F1 score of 0.906 and Macro-F1 score of 0.760. High-frequency core entities (e.g., seizure symptoms, drug names, and temporal information) all yielded F1 scores exceeding 0.90. In cross-domain validation on literature-based cases, high-frequency entity F1 scores remained above 0.80, while low-frequency entities (e.g., factors with incomplete/ambiguous induction fac-inc-amb, treatment information trt, and adverse drug reactions dru-adv) achieved F1 scores of 0.31-0.57, primarily attributable to limited sample size and high linguistic variability.

Conclusion The epilepsy-specific Chinese clinical annotation schema developed in this study demonstrates broad coverage, fine granularity, and high inter-annotator consistency. It effectively supports the training and evaluation of NER models and provides a reusable corpus foundation for the structured analysis of epilepsy medical records and the development of downstream intelligent diagnostic and therapeutic tools.