Artificial intelligence tool development: what clinicians need to know?

Abstract

Digital medicine and smart healthcare will not be realised without the cognizant participation of clinicians. Artificial intelligence (AI) today primarily involves computers or machines designed to simulate aspects of human intelligence using mathematically designed neural networks, although early AI systems relied on a variety of non-neural network techniques. With the increased complexity of the neural layers, deep machine learning (ML) can self-learn and augment many human tasks that require decision-making on the basis of multiple sources of data. Clinicians are important stakeholders in the use of AI and ML tools. The review questions are as follows: What is the typical process of AI tool development in the full cycle? What are the important concepts and technical aspects of each step? This review synthesises a targeted literature review and reports and summarises online structured materials to present a succinct explanation of the whole development process of AI tools. The development of AI tools in healthcare involves a series of cyclical processes: (1) identifying clinical problems suitable for AI solutions, (2) forming project teams or collaborating with experts, (3) organising and curating relevant data, (4) establishing robust physical and virtual infrastructure, and computer systems’ architecture that support subsequent stages, (5) exploring AI neural networks on open access platforms before making a new decision, (6) validating AI/ML models, (7) registration, (8) clinical deployment and continuous performance monitoring and (9) improving the AI ecosystem ensures its adaptability to evolving clinical needs. A sound understanding of this would help clinicians appreciate the development of AI tools and engage in codesigning, evaluating and monitoring the tools. This would facilitate broader use and closer regulation of AI/ML tools in healthcare settings.

Background

The transformation and realisation of digital medicine [1] and smart healthcare [2] hinge upon the active and cognizant participation of clinicians in the entire development process and cycle [3, 4]. Clinicians educated in these aspects could provide invaluable insights during the design stage of digital health technologies, including artificial intelligence (AI)-enabled tools and systems [5]. This could ensure that these solutions meet their unique needs and preferences, emphasise patient safety and quality of care and facilitate seamless integration into clinical workflows [3]. Additionally, their endorsement and support foster adoption and acceptance among other healthcare providers, drive innovation and ultimately optimise clinical outcomes [6, 7].

AI is the cognitive ability of machines made possible by mathematically designed neural networks (see the Glossary in the Additional File 1). The electronic neural networks are built to mimic the human neuronal plexus and are programmed to manage a myriad of data according to their categories and to assign different factors of data different weights. The weights are decided from given data on specified outcomes, and this process is continuously being improved with ongoing receival of data. When vast amounts of data are interconnected, it enables new discoveries and creates opportunities that can transform personal experiences and advance science in nearly all aspects of human life [8]. For example, interconnected health data can lead to early detection of diseases through predictive analytics, while personalised education platforms use linked learning data to tailor teaching methods to individual needs. In science, combining datasets across disciplines can uncover patterns such as predicting climate change impacts or accelerating drug development through AI-driven simulations. The learning computer models are machine learning (ML) versions of AI, and deep learning (DL) is a version with multiple layers of neural networks. The predictive ability of such models is evaluated against the annotated or labelled outcome via weights applied to each variable when they are included in the models. The models are self-learning, improving their performance through repeated adjustments (iterations) using a method called backpropagation which optimises the model by minimising errors via a process known as stochastic gradient descent. This process in gauging the best weights for variables in the model as they progress through their different levels of complexity at the different neural layers. Other ML models include natural language processing and computer vision. Transformers are present when DL models differentially weigh the importance of each part of the input data and make natural-language processing possible (ChatGPT stands for the Chat Generative Pretrained Transformer). Augmented intelligence (AugI) is an AI that supplements and enhances humans’ ability instead of substituting it. There are FDA-approved software, applications, programmes and devices that use AI to interpret a broad range of imaging modalities and diagnostic and prognostic assistance and help outline possible treatments for clinicians [9, 10]. It is important to discern between programmed computer systems and applications that mimic AI tools and systems but are not considered true AI. Table 1 shows examples of similar tools in these two categories in the healthcare industry.

Table 1 A comparison of programmed computer systems and similar AI tools or systems

The integration of AI into routine clinical care is accelerating, which is now reviewing patient histories, drafting physician notes, offering patient instructions, and not just reading X-rays and histopathological images [11,12,13,14]. The appropriate use of AI technology in healthcare defined as ethical, clinically validated and seamlessly integrated applications that enhance patient care and efficiency which is also potentially cost-effective [15]. This is when improved quality care by reducing variation, being safe and expedient [16, 17], transforming reactive healthcare to a more proactive approach, and focusing on health promotion, disease prevention and health management rather than disease treatment, resulting in fewer hospitalisations, fewer doctor visits and fewer treatments [18]. AI applications are projected to reduce annual healthcare expenditures in the USA by USD 150 billion by 2026, primarily through increased efficiency, improved diagnostics, and optimised care delivery [18]. However, all this AI advancement is not without great challenges from development to deployment [19,20,21], integration in clinical workflows [22, 23] and influences on doctor‒patient consultation [24, 25]. Persistent concerns about integrating AI systems into existing clinical consultations include alert fatigue [26, 27], data quality, data security, transparency and accountability, alignment with standards and guidelines and unintended consequences along with model design requirements, and retention of clinician autonomy [28]. The human factors and AI systems that may affect medical professionals’ interactions with technology could be related to perceptions of training data quality, performance of AI systems, explainability, adaptability, medical expertise (young versus experienced clinicians), technological expertise, personality, cognitive biases (proper understanding and use of AI outputs) and trust in the whole ecosystem [29]. Table 2 shows the real present challenges of AI technology in healthcare and its more certain progress in the near future. Globally, the AI healthcare market was valued at USD 20.9 billion in 2024 and is anticipated to grow at a compound annual growth rate of 48.1% reaching an estimated USD 148.4 billion by 2029. This growth reflects expanding investments in AI-driven technologies and services across the healthcare sector [30].

Table 2 The present challenges and future uses of AI technology in healthcare

This review explains the usual path for new AI tool development and deployment in healthcare and clinical services. It concurs with other evaluation frameworks [31, 32] (Table 3) and could be extended to include assessments of health economic benefits [33]. When selecting an evaluation framework, users should consider the specific objectives of their study as some frameworks focus on quality evaluation, transparent reporting or risk of bias, while others address specific stages, designs or disciplines. Depending on the purpose, a single framework or a combination can guide study planning, implementation and reporting to ensure robust and impactful outcomes. However, articles with sufficient and clear technical explanations of the AI development process for clinicians are scarce [31, 32, 34]. Clinicians with a sound understanding of the whole development process of AI tools would help them engage in codesign, effective collaboration [35], evaluation and monitoring of the tools, and further facilitate broader use and closer regulation of these tools in healthcare settings [36].

Table 3 Evaluation frameworks of AI studies

Some reporting guidelines are study design specific (TRIPOD-AI for prognostic and diagnostic studies, STARD-AI for diagnostic test studies, SPIRIT/CONSORT and SPIRIT/CONSORT-AI for clinical trials), stage specific (DECIDE-AI for early clinical studies) or discipline specific (CHEERS-AI for health economy, IDEAL for surgery, and CLAIM and FUTURE-AI for radiology) [39].

How are AI tools and models developed according to clinicians’ needs?

This focused integrative review attempts to update and delineate practical knowledge on AI tools or model development throughout the whole process for clinicians. The review questions are as follows: What is the typical process of AI tool development in the full cycle? What are the important concepts and technical aspects of each step? The approach includes a targeted literature review and synthesised summaries from online courses, including but not limited to the AI for healthcare by the National University of Singapore (https://nusmed.emeritus.org/ai-for-healthcare), the No Code AI and Machine Learning: Building Data Science Solutions by the Massachusetts Institute of Technology (https://professionalonline2.mit.edu/no-code-artificial-intelligence-machine-learning-program) and the European Information Technologies Certification Academy (EITCA) Artificial Intelligence Academy (https://eitca.org/certification/eitca-ai-artificial-intelligence-academy/). It strives to provide adequate technical knowledge that is immediately useful for clinicians to appreciate the development of AI tools and is able to engage with developers, vendors and researchers when considering clinical adoption and codesigning a new tool. ChatGPT 3.5, 4o and o1 (OpenAI, San Francisco, CA, USA) were used to assist in drafting and language editing of portions of this review. The authors have reviewed and edited the content produced by ChatGPT for accuracy and integrity, and accept full responsibility for the final version of the manuscript.

Overview of the AI development process

The development of AI in healthcare involves a series of cyclical processes (Fig. 1). It begins by identifying clinical problems suitable for AI solutions, forming project teams or collaborating with experts, and organising and curating relevant data. The establishment of robust infrastructure and architecture supports subsequent stages, including the exploration of AI neural networks on open access platforms and the validation of AI/ML models. Following registration procedures, clinical deployment and continuous performance monitoring occur. Finally, a commitment to improving the AI ecosystem ensures its adaptability to evolving clinical needs.

Fig. 1

AL/ML tool development process

Clinical problem identification

This first step is the most important starting point for the rest of the development process (see Fig. 2). Some clinical and biomedical problems in healthcare services could be best resolved with the help of an automated solution. These are problems or challenges that are technically factual, mechanical, repetitive and complex in nature owing to the need to process multiple aspects of healthcare services, people in the health system or patient characteristics (see tips and examples in Table 4). DL/AugI/ML does not help address personal values, health beliefs or emotions that change until these constructs are measured in certain ways. Identifying the problem includes deciding on the level of the problem for the AI technology to solve. This method is descriptive, diagnostic, predictive or prescriptive and uses either assistive or autonomous AI algorithms (Table 4). Descriptive AI models are about estimating the quantity of a certain condition, diagnostic models are about the probability of occurrence of certain conditions, prognostic models are predicting certain outcomes, and prescriptive models suggest the most likely efficacious treatment. The order denotes an incremental level of value and complexity to be expected in the development of the tool. Be as specific and clearly defined as possible with all the variables, especially the outcome variable (supervised learning models).

Fig. 2

The process of identifying AI tools for clinical problems

Table 4 Tips for identifying clinical problems for AI solutions

Forming a project team or collaborating with experts

A successful team must consist of individuals with the right skillsets. This includes data scientists for data validation, transformation, curation, and visualisation for AI/ML models; data engineers to implement data workflows, such as storage; data architects to design the system architecture for data repositories; and a chief data officer to establish the data governance structure and policies. Clinicians, healthcare administrators and relevant stakeholders, including patients and public groups, are essential for the planning, development, deployment and sustained use of the AI/ML tool. Additionally, health informatics professionals and business or industry partners should be considered. To secure funding, the project must address the tool’s ethical aspects, ensuring professional integrity, a clear balance of benefits over harm, justice and trustworthiness, with designated accountability for its implementation.

Data availability, organisation and curation

Relevant real-world data sources must be explored, annotated and preprocessed (Table 5). The availability of high-quality and sufficient variables in the target population is crucial for AI solutions to address clinical problems. These data must be diverse and representative [58], properly labelled and curated to minimise bias and errors. Data will need to go through several stages before becoming useful for AI algorithmic models. These stages include standardisation (coding structured and unstructured data) for interoperability, cataloguing, deidentification (pseudo or anonymisation), cleaning/transformation (validation), and linking and combining different sources into a single dataset. Managing a large amount of quality data within credible data governance structures remains a significant challenge [59].

Table 5 Data curation for AI tools

Infrastructure and architecture for data repository and AI technology

An adequate capacity of computers and servers and accessories for operating large amounts of data at the optimal speed are needed (Table 6). Intelligence processing units (IPUs) are rarer, especially on certain clouds, and are best used in graph-based AI algorithms. In scenarios where high-performance, energy-efficient hardware acceleration is required to handle demanding AI workloads, enabling faster training, inference and deployment of AI models across various applications and industries. Another specialised hardware accelerator developed by Google is the tensor processing unit (TPU). It is specific to ML tasks, particularly those involving TensorFlow and Google’s open-source ML framework. Compared with traditional CPUs and GPUs, they offer significant speedups and cost savings, particularly for large-scale AI workloads running on TensorFlow-based frameworks. The operations of these data servers include strong cybersecurity (data encryption), data privacy, controlled access and updated regulatory policies on the proper use of the data, and supervised incremental learning of the AI/ML tools. In addition to security and proper governance, the ease and speed of access to different users are paramount. The physical and virtual infrastructure, and computer systems’ architecture must be scalable to meet the increasing needs and demands of the tools. Alternatively, cloud-based infrastructures offer more feasible services in AI tool development from algorithm building to deployment and scale AI applications by providing access to a rich ecosystem of resources and tools. The three main cloud service providers are Amazon Web Services (AWS), the Google Cloud Platform (GCP) and Microsoft Azure (Table 7). In addition to providing scalable infrastructure, it also provides robust data storage, management solutions and a wide range of AI development tools and frameworks, such as TensorFlow and Azure Machine Learning, and application programming interfaces (APIs) to streamline the development workflow. Cloud services also facilitate collaboration among team members and simplify the deployment of AI models in production environments. Additionally, they offer monitoring and optimisation tools to ensure the optimal performance of AI applications.

Table 6 Different levels of computing power and AI technologies

Table 7 The three main cloud service providers

AI neural networks on open access platforms

Many AI algorithms are readily available on open-access platforms (Table 8), with similar algorithms often already developed and tested. Choosing appropriate AI/ML models and methods is essential for resolving clinical challenges. The model selection framework should balance performance requirements with cost, risk, deployment needs and stakeholder expectations [60]. The choice of algorithm depends on the input type, whether speech, language, vision, decision-making, or a combination of these. For example, convolutional neural networks are ideal for image data, whereas recurrent neural networks are best suited for text and numerical data [61]. The development of new AI neural networks requires data scientists with advanced skill sets and is time-consuming.

Table 8 Examples of open sources for AI algorithms

AI/ML model validation

The selected or newly developed algorithm must undergo training, validation and testing on a curated dataset (Table 5). Its performance should be evaluated and compared with that of the baseline model or standard of care before external validation, especially if the model is applied in different settings from where it was developed and then deployed in practice [37]. Table 9 shows the classification tasks and ML strategies on data [60, 61]. Both nonclinical and clinical validation are essential to establish its performance, ensuring its integration into routine clinical workflows, usability and positive effects on clinical outcomes. Properly designed clinical research, including clinical trials, may be necessary to assess its real-world clinical impact. Table 3 outlines recommendations for evaluating AI tools in clinical settings, whether as diagnostic or prognostic tools. Once finalised, the results are published for broad dissemination and peer scrutiny. It is also critical to explore and address any ethical and legal implications associated with using these tools in healthcare, as liability risks may arise from sources of error, error identification, potential harm and legal redress [62].

Table 9 ML learning, techniques and evaluation metrics

Registration

The registration of tools with relevant authorities is typically carried out by the manufacturer, developer or the organisation responsible for the AI tool. In many cases, this involves collaboration between technical and regulatory teams within the organisation to ensure compliance with the regulatory requirements of the target market. Tools registration with relevant authorities such as medical device authorities could increase the likelihood of successful implementation and deployment in the real world [63]. The evaluation criteria differ across countries, which may include an effectiveness trial [32]. AI tools in the UK are classified as medical devices and therefore require Medicines and Healthcare products Regulatory Agency (MHRA) approval bearing the “United Kingdom Conformity Assessed” (UKCA) logo. However, AI tools in Europe are regulated by the EU Medical Device Regulation (EU MDR) and bearing the “Conformité Européenne” logo to be marketed in Europe. In the USA, AI tools are regulated by the Food & Drug Administration (FDA) (https://www.fda.gov/science-research/science-and-research-special-topics/artificial-intelligence-and-medical-products). FDA classifies AI tools based on their risk level and intended use following pathways such as 510(k) premarket notification, De Novo classification or premarket approval. Many AI tools are categorised as Software as a Medical Device and must meet rigorous criteria for safety, effectiveness and transparency including Good Machine Learning Practices. Post-market surveillance is often required to monitor real-world performance while labelling must clearly define intended use, performance metrics and limitations. European AI Act [64] prohibits AI systems that collect sensitive personal information that causes discrimination, manipulates human behaviour or exploits vulnerabilities of certain groups of people at all social places. The core principle is that AI “… should be a human-centric technology. It should serve as a tool for people, with the ultimate aim of increasing human well-being.” Clinicians play a vital role in evaluating AI tools’ suitability for their practice and ensuring their safe and effective use.

While regulatory frameworks vary across regions, a unifying principle among global authorities is the emphasis on ensuring that AI tools align with ethical standards, prioritising human well-being, fairness and accountability. These principles not only guide the evaluation and approval processes but also ensure that the implementation of AI tools remains consistent with societal values and promotes trust among users and stakeholders. The ethical principles of nonmaleficence, beneficence, autonomy and justice with added governance and associated principles of privacy, diversity, inclusiveness, transparency, reliability, fairness, social good, well-being, sustainability, auditability, explicability, interpretability and quality data are referred to in high-level policy documents [65,66,67,68] (see Fig. 3, and Additional File 2: AI Ethics and Policy Frameworks from the United Nations Educational, Scientific and Cultural Organization 2022 [7, 65, 69], UN Resolution on AI 2024 [66], International Scientific Report on the Safety of Advanced AI: Interim Report 2024 [70], Diversity, INclusivity and Generalisability: STANDING Together project team 2023 [58], US Executive Order on AI 2023 [67], Artificial Intelligence Act European Parliament/2024 [68], Harmonised Standards for the European AI Act: European Parliament 2024 [71], Ethics and governance of artificial intelligence for health: World Health Organization guidance 2021 [72], Organization for Economic Cooperation and Development AI Principles 2019 [73], Universal Guidelines for AI: Center for AI and Digital Policy 2018 [74], Asilomar AI Principles: Future of Life Institute 2017[75]). These principles, in the form of typology according to the different stages of the AI life cycle and sources, are available here (https://ricardo-ob.github.io/tools4responsibleai/#title-cite) [76] and foster the development of responsible AI tools and systems by technical and nontechnical persons, balancing the risk and benefits to the public [77]. AI tools and systems are prohibited by the European Union’s AI Act if they manipulate cognitive behaviours, classify the traits and status of people through facial or emotion recognition and collect sociobiological characteristics such as sexual orientation or religious beliefs into various forms of social scoring or biometric categorisation [68].

Fig. 3

AI ethics that may determine progressive or regressive outcomes. *Different age groups, cultural systems and language groups, persons with disabilities, girls and women, and disadvantaged, marginalised and vulnerable people or people in vulnerable situations. SDG = Sustainable development goals.

Clinical deployment and monitoring of performance

Deployment is the method by which the tested AI tools are integrated into an existing clinical workflow to make practical healthcare decisions (outputs) on the basis of data (input). The best deployment strategy would consider the software systems or applications environment where the AI tools are to be deployed. If this system is a web service or electronic health records system, then it will require an API to enable data pipeline integration where the input and output could be executed. The easier the deployment process, which includes having the same API endpoint references, the faster the model improvements are. The design of the user interfaces must allow alerts or notifications to be displayed noninterruptively but effectively to achieve practice efficiency and provider acceptance and adoption. This could be tested via “silent” or “shadow” deployment, which is deployed in the actual environment but not fully for routine use. Another important step before deployment to production is the quality testing of scalability and performance optimisation in scenarios when high data flow occurs. The final deployment approach is likely a decision between the budget, availability of the infrastructures and the required performance of the AI tools.

Another important task is to train clinicians and team members in healthcare facilities where tools are used or integrated into the electronic medical records system. Promoting human-AI teaming would augment performance and safeguard autonomy but require calibrated design, support and monitoring [34]. Be prepared to explain the decision process of the AI/ML tools and be present to support their use. This responsibility often lies with a collaborative effort between the developers and the clinical staff, with oversight from regulatory bodies. Neural networks in AI are often called “black boxes” because their internal workings are complex and not easily understood. This lack of transparency can be problematic in healthcare where understanding how decisions are made is crucial. To address this, techniques like SHAP (Shapley Additive Explanations) and LIME (Local Interpretable Model-agnostic Explanations) help identify which input features most influence the AI’s decisions. Visualisation tools such as Grad-CAM (Gradient-weighted Class Activation Mapping) can highlight areas in medical images that the AI focuses on, providing insights into its decision-making process [78].

The tool’s effect in clinical care should be measured, and the incremental learning of the tool should be supervised. Additionally, monitoring should continue for any unwanted effects arising from its use including data integrity, cybersecurity and impact mitigation in cases of breach and functional recovery [79]. A medical algorithmic audit could and should be conducted if it was not performed before real-world deployment [80]. Changes in the clinical practices such as changing guidelines, treatments or diagnostic protocols can render previously trained models less effective over time. Be vigilant and prepared to retrain the AI/ML algorithm if the performance has drifted to below expectations. Clinicians, developers and IT teams must remain vigilant in monitoring AI performance for signs of drift, with clinicians reporting inconsistencies and developers tracking key metrics. This includes prediction accuracy, response time and resource utilisation. Developers are primarily responsible for retraining models, using updated data and collaborating with clinical staff to ensure relevance, while compliance teams ensure adherence to regulatory standards. Retraining involves collecting new data, refining the model, validating updates and carefully redeploying the system with ongoing performance monitoring. Any update done to the tools may require a notification to the regulatory body, and it is essential to consult specific guidelines and maintain open communication with the relevant regulatory body.

AI ecosystem improvement

Actively engage stakeholders and the public with the tool throughout the development-deployment-monitoring process. This could improve confidence and progress the AI ecosystem in the country [17]. This approach aims to improve broader and better perceptions of AI/ML technologies, invite more keen interest and training from experts, and establish central governance, trusted custodian, ethical value and proper regulation. This may increase the investment and uptake of AI technologies in healthcare and research that are supported by sufficient funding and infrastructure to allow freedom to innovate and implement more AI/ML tools. Ethical considerations around data privacy and patient safety are well-known challenges that must be addressed [81, 82]. Similarly, traditional medical principles remain crucial as they uphold patient dignity and foster mutual trust among doctors, patients and society. Establishing a successful partnership between technology companies, which provide technological expertise, and healthcare facilities which offer data and expert input, is essential. This partnership must be both regulatory compliant and economically beneficial to ensure the effective implementation and deployment of algorithms [83].

Experiences in National University Health System, Singapore

This research shares the experience and valuable lessons of the National University Health System, Singapore (NUHS), in obtaining AI tools for production in healthcare services. NUHS’s experience in implementing AI-driven healthcare systems offers valuable insights for institutions pursuing similar transformations. Success in AI implementation extends beyond the technology itself, requiring four critical elements: (1) establishing robust data infrastructure, (2) building organisational trust, (3) ensuring continuous human oversight through committees and (4) committing to long-term engagement with AI technology [84].

They developed the ENDEAVOUR AI platform, a comprehensive AI system that integrates various tools to streamline operations [85]. Additionally, they established DISCOVERY AI, a private AI training cloud featuring NVIDIA DGX A100 s to support the development of AI models, which functions as both a production system and a research sandbox for modular machine learning tools [16, 85]. It adheres to local and international regulatory guidelines, with data anonymized by removing identifiers such as names, addresses and identification numbers. A robust master governance structure ensures equitable data access, centralised anonymisation and differential data linkage. Data access and sharing are overseen by custodians of specific databases and a dedicated committee. This governance framework also manages research administration, including institutional review board processes and collaborative agreements. Integrated with the electronic health record system, the platform leverages multiple clinical data and research databases through embedded algorithms to enable many AI predictions.

With both the ENDEAVOUR AI platform and DISCOVERY AI, a series of AI tools have been developed and successfully deployed for clinical care, while some have undergone internal validation within the institution and are pending full peer-reviewed publication [84, 85]. An AI-driven system, the Pathfinder Dashboard (Additional File 3 shares the experience of Pathfinder Dashboard AI tools development and challenges according to the nine steps mentioned in this review) predicts patient wait times and manages patient inflows at the emergency department, enhancing care quality and patient satisfaction. Should patients have to be admitted for inpatient care, the estimated length of stay model predicts patient hospital stays, ensuring timely and appropriate care and thus optimising the effective planning and allocation of resources [86]. When discharge is possible or decided upon, the 30-day readmission prediction model could personalise patient care to prevent readmission and reduce hospital costs. The Disease Progression Modelling tool enables earlier intervention by anticipating disease progression, particularly for chronic conditions, and the Pharmacogenomics Alerts System tailors medication recommendations on the basis of genetic profiles, enhancing precision medicine and reducing adverse drug reactions. NUHS has enhanced patient communication with various chatbot systems, including RUSSELL-GPT [87], which provides instant responses and personalised health information. These chatbots use advanced GPT models to cater to both patients and researchers while maintaining data security. For managing chronic diseases, all AI tools at NUHS are integrated with the Epic EMR System, providing a unified AI dashboard that offers comprehensive insights. This integration enhances decision-making and patient care by consolidating information and streamlining hospital operations.

There is the CURATE.AI to optimise chemotherapy treatments for prostate cancer [88] and solid tumours [89]. It has also been applied to personalise dose selection [90] and to tailor immunosuppressant drug dosages for liver transplant patients to prevent organ rejection [91]. NUHS introduced the Chronic Disease Management Programme (CHAMP) Chatbot System, which engages patients with reminders and follow-ups via WhatsApp. Compared with similar programmes, this tool aims to improve patient adherence to treatment plans, leading to higher enrollment rates and lower dropout rates.

Discussion

Developing and translating AI innovations from research to clinical practice faces significant challenges often referred to as the “valley of death” [92]. These include the complexity of identifying the right pain point, clinical validation, regulatory hurdles and the need for robust evidence of efficacy and safety, registration with the regulatory body and communication with trust with healthcare stakeholders for integration into an existing clinical workflow [93, 94]. Additionally, the lack of standardised reporting and evaluation frameworks complicates the explainability and interpretability for the integration of AI tools into healthcare settings [95]. Clinicians who are more than aware of the full cycle of AI tools development delineated in this paper could facilitate the development, reporting, assessment and smoother transitions from bench to bedside of these tools [96, 97].

The most important step and challenge to tackle is the biases in training data. This could perpetuate healthcare disparities such as the underrepresentation of specific demographic groups or the reinforcement of historical biases in data collection [98]. Compounding this issue would be a dataset shift post-deployment where the model’s operational environment differs from its training environment causes a degrade performance and compromise generalisability [99]. Mitigating these challenges requires careful dataset curation to have diverse and representative samples, along with the deployment of bias detection and mitigation strategies [100]. Rigorous external validation across varied populations and settings is essential to ensure the reproducibility and generalisability of AI models, both of which are foundational to achieving fairness, equity and clinical adoption [101].

Beside the strategies alluded to when faced with a lack of data quantity or poor data quality, there are several strategies generative AI can offer. This includes creating synthetic data based on the electronic health records, omics data and bioimages to train diagnostic and predictive models [102, 103]. This transformative potential alleviates data scarcity, enhances patient privacy and enables the simulation of rare or complex clinical scenarios. However, challenges remain in ensuring that synthetic data maintains the variability and complexity of real-world datasets to achieve reproducibility [104]. For AI models trained on synthetic data, rigorous testing and validation are necessary to confirm that they generalise accurately to diverse populations and clinical realities. Addressing these challenges allows generative AI to significantly enhance the robustness and utility of healthcare AI systems.

Evaluating AI models against professional clinicians is crucial to understanding their clinical utility and assessing their algorithmic quality [105]. While some AI models achieve expert-level accuracy, a lack of rigorous study design often leads to overestimated performance claims. Comparative studies and standardised evaluation frameworks are critical for determining whether AI tools can complement or enhance human decision-making in healthcare [93]. Such evaluations are vital for building confidence among stakeholders and ensuring the safe and effective deployment of AI in clinical practice.

Reproducibility and generalisability are critical pillars for ensuring the effective translation, application, and evaluation of AI models that has been developed for healthcare. Reproducibility demands consistent results through transparent documentation of data collection, preprocessing and model training, fostering trust and reliability [106]. Generalisability ensures that AI models perform accurately and equitably across diverse populations, clinical settings and evolving medical practices, addressing key challenges such as dataset biases and shifts [107]. These principles are essential for validating all AI tools and ensuring robust, scalable solutions. By integrating these considerations across the entire AI lifecycle, from development to deployment, clinicians and developers can create innovative, equitable and reliable tools that meet the demands of real-world healthcare.

All AI systems, regardless of type, follow a similar life cycle as described in this review. The primary differences lie in complexity, scalability, interpretability, resource demands and training methodology. Traditional AI/ML models that utilise established statistical or rule-based techniques with manually chosen features are typically simpler, more interpretable and less computationally demanding. Neural networks and deep learning models that employ layered neuron-like architectures that learn patterns from raw data could scale well with large data but require more computational resources and are harder to interpret. Generative AI models implement advanced frameworks that generate original outputs by modelling data distributions do push these challenges further, often requiring massive data and compute resources, more complex training regimes (pretraining plus fine-tuning), and specialised evaluation and monitoring strategies.

Conclusions

This review presents a straightforward explanation of the entire development process of AI tools, outlined in nine cyclical and iterative steps, which could enhance understanding among clinicians. More importantly, the presentation with many infographics and examples, combined with adequate technical details, has the potential to reach a broader audience, particularly in countries that face greater inequities in the health AI/ML literature [108, 109] and are at risk of health disparities from this technology [110, 111]. Notably, other great challenges include win‒win partnerships between technology companies such as technological know-how, health-care facilities, as data sources, and expert inputs to algorithms [112, 113]. This is to be regulatory, acceptable and economically rewarding to the two stakeholders [83, 114]. Robust AI tools are those that resolve real-world clinical problems, are developed by a team of relevant stakeholders, are trained on broad-based high-quality data; are validated externally, prospectively and in controlled trials or equivalently. They perform in real time, are unbiased, safe and trustworthy with acceptable human‒AI tool interactions [20], are quick in algorithm updates to cover emerging diseases, are controllable by human agents, are acceptable to target users who are either explainable or unexplainable [5, 115], and are ethically justifiable and legally compliant [116]. Challenges in attaining high-performing AI systems include having high-quality infrastructures in terms of computing power, memory and storage capacity, high-speed internet connectivity, low-latency networking, more energy efficient computing technology (quantum computing and optical computing) [117], and scalability and elasticity that are supported by ethics and regulatory compliance data governance [118].

Ultimately, AI/ML tools offer significant benefits by reducing systemic, sporadic medical errors and enhancing patient care quality. These tools streamline healthcare processes, integrate seamlessly into health systems and are continuously monitored to ensure safety and effectiveness. Having legal framework that ensure compliance with data security, protection and privacy policies, positive economic impacts or at least an oversight by an established data governance body including representation from the public and patients could further strengthens accountability and trust in their use [119]. Accordingly, success clinical integration and implementation of AI tools must include building trust and confidence among clinicians in the development process, having quality data, and risk levels are understood by all stakeholders and mitigated as a team with clinicians [120], satisfying fairness, equity, robustness, privacy, safety, transparency, explainability and accountability with assured benefits for patients, healthcare providers and the organisation involved [121].

As the field of AI is anticipated to evolve quickly with new technologies and algorithms, it is essential for all stakeholders including clinicians, to stay informed about new guidelines, reporting standards for AI tools and systems, and the application of AI in medicine (see Table 2 on some of the important organisations on AI-related matters in Additional File 1) [116].