Objective  In this study, we used artificial intelligence (AI) technology to explore for automated medical record quality control methods, standardize the process for medical record documentation, and deal with the drawbacks of manually implemented quality control.

  Methods  In this study, we constructed a medical record quality control system based on AI. We first designed and built, for the system, a quality control rule base based on authoritative standards and expert opinions. Then, medical records data were automatically collected through a data acquisition engine and were converted into structured data through a post-structured engine. Finally, the medical record quality control engine was combined with the rule base to analyze the data, identify quality problems, and realize automated intelligent quality control. This system was applied to the quality control of medical records and five quality control points were selected, including similarities in the history of the present illness, defects in the description of chief complaints, incomplete initial diagnosis, missing in formation in the history of menstruation, marriage, and childbirth, and mismatch between the chief complaints and the history of the present illness. We randomly selected 2 918 medical records of patients discharged in January 2022 to conduct AI quality control. Then we organized medical record quality control experts to conduct an accuracy review, made a comparison with previous manual quality control records, and analyzed the results. The number of quality problems that were verified in the accuracy review was taken as the gold standard and receiver operating characteristic (ROC) curves were drawn for the 5 quality control points.

  Results   According to the accuracy review performed by medical record quality control experts, the accuracy of AI quality control reached 89.57%. For the sampled medical records, the results of AI quality control were compared with those of previous manually performed quality control and only one problem detected by manual quality control of the sampled medical records was not detected by the AI quality control system. The number of medical record quality problems correctly detected by AI quality control was about 2.97 times that of manual quality control. Analysis of the ROC curves showed that the AUC of the five quality control points of the AI quality control system were statistically significant (P<0.05) and all the AUC values approximated or exceeded 0.9. In contrast, results obtained through manually performed quality control found significant AUC (0.797) for only one quality control point—similarities in the history of present illness (P<0.05). Comparison of the AUC values of the two quality control methods showed that AI quality control system had an advantage over manually performed quality control for the five quality control points.

  Conclusion  Through the application of medical record quality control system based on AI, efficient full quality control of medical record documentation can be achieved and the detection rate of quality problems can be effectively improved. In addition, the system helps save manpower and improve the quality of medical record documentation.