Authors | Mahdavi, M., Thomas, N., Flood, C., Stewart-Lord, A., Baillie, L., Grisan, E., Callaghan, P., Panayotova, R, Hothi, S, Griffith, V., Jayadev, S. and Frings, D. |
---|
Abstract | Introduction The use and value of Artificial Intelligence (AI)-driven tools and techniques are under investigation in detecting coronary artery disease (CAD). EchoGo Pro is a patented AI-driven stress echocardiography (SE) analysis system produced by Ultromics Ltd. (henceforth Ultromics) to support clinicians in detecting cardiac ischaemia and potential CAD. This manuscript presents the research protocol for a field study to independently evaluate the accuracy, acceptability, implementation barriers, users' experience and willingness to pay, cost-effectiveness, and value of EchoGo Pro. Methods and analysis ‘Evaluating AI-driven stress echocardiography analysis system’ (EASE) study is a mixed-method evaluation, which will be conducted in 5 work packages (WPs). In WP1, we will examine the diagnostic accuracy by comparing test reports generated by EchoGo Pro and 3 manual raters. In WP2, we will focus on interviewing clinicians, innovation/transformation staff, and patients within the NHS, and staff within Ultromics, to assess the acceptability of this technology. In this WP, we will determine convergence and divergence between EchoGo Pro recommendations and cardiologists’ interpretations and will assess what profile of cases are linked with convergence and divergence between EchoGo Pro recommendations and cardiologists’ interpretations and how these link to outcomes. In WP4, we will conduct a quantitative cross-sectional survey of trust in AI tools applied to cardiac care settings among clinicians, healthcare commissioners, and the general public. Lastly, in WP5, we will estimate the cost of deploying the EchoGo Pro technology, cost-effectiveness and willingness to pay of cardiologists, healthcare commissioners and the general public. The results of this evaluation will support evidence-informed decision-making around the widespread adoption of EchoGo Pro and similar technologies in the NHS and other health systems. Ethics approval and dissemination This research has been approved by the NHS Health Research Authority (IRAS No: 315284) and the London South Bank University Ethics Panel (ETH2223-0164). Alongside journal publications, we will disseminate study methods and findings in conferences, seminars, and social media. We will produce additional outputs in appropriate forms, e.g., research summaries and policy briefs, for diverse audiences in NHS. Strengths and limitations of this study - The main strength of this study lies in its mixed-method approach to evaluation, which offers comprehensive assessment, data triangulation, and increased validity of, and confidence in, findings. - The mixed-method approach will yield diverse forms of empirical evidence, financial evidence, and stakeholder evidence. - By measuring willingness to pay from different users, this study will inform decisions around investment in healthcare AI applications. - One limitation is that the impact of this AI application at the health system level, in terms of improved outcomes or quality of care, will not be measured in this study. |
---|