Artificial intelligence–driven decision support systems (AI–DSS) have the potential to help physicians analyze data and facilitate the search for a correct diagnosis or suitable intervention. The potential of such systems is often emphasized. However, implementation in clinical practice deserves continuous attention.

A new paper published in Frontiers in Medicine journal sheds light on the needs and challenges arising from the use of AI-DSS from physicians’ perspectives. The basis for this recent study is a qualitative content analysis of expert interviews with experienced nephrologists after testing an AI-DSS in a straightforward usage scenario, aiming to provide insights on the basics of clinical decision-making, expected challenges when using AI-DSS as well as a reflection on the test run.

D. Samhammer and colleagues could confirm the somewhat expectable demand for better explainability and control. However, other insights highlight the need to uphold classical strengths of the medical profession when using AI-DSS as well as the importance of broadening the view of AI-related challenges to the clinical environment, especially during treatment.

With the progress of AI-DSS performance, the question of how to implement relevant systems in practice becomes more central. The purpose of this study was to explore physicians’ perspectives after testing an AI-DSS for predicting risks in kidney transplant care. In general, the findings indicate a positive attitude toward AI-DSS. Concerns include a potential loss of expertise and autonomy. Of high importance is the clinical environment in which AI decision-support as well as patient involvement takes place. This is in line with the desire for further means of establishing explainability and control.

These findings resonate with current debates on AI in medicine. The question of how to implement AI-DSS in current organizational structures as well as the importance of ensuring shared decision-making remain equally central. Explainability must be context-specific and AI-DSS designed to ensure meaningful interaction. In this sense, expertise and autonomy of the treating physicians may be maintained when implementing AI-DSS. The researchers hope the present study supplements conceptual discussions surrounding AI-DSS in medicine with empirical evidence and provides a starting point for further research.

Access to the whole study.