Navigating artificial intelligence in healthcare: Hurdles and hindrances

Dear Editor,

We wish to communicate our observations on the concept of Generative AI and its limitations.

Generative artificial intelligence (AI) is an emerging concept in terms of healthcare. It finds its use in various applications related to the world of healthcare and medicine. The concept of AI and, most significantly, Generative AI has secured its place in different modalities, right from patient data sheet management to diagnostics and imaging.¹

Generative AI refers to the application of AI models in deriving information regarding varied subject matter. It consumes and collects the required information from the vast data pool fed into the system. In the discussion of Generative AI, one significant term comes to light, which is Large Language Models (LLMs).

LLMs are, in simple words, a vast computer program that holds a vast set of data developed in it. It derives required information based on the queries fed into its input branch through its application of Chatbots like ChatGPT, Bard etc.

For example, we can assume LLM to hold a set of data related to different diagnostic methods available in a tertiary care hospital. Now, through one of its applications of chatbots like ChatGPT, we can provide prompts or queries to make a list of diagnostic methods available for tuberculosis in that center.

Now, comes the question as to why we have any relevance of using such AI tools in the first place. The foremost usage is as simple as it makes the task at hand easier to comprehend, coupled with faster derivations, and in a better-organized manner, the content is available.

It also finds its use in non-native English speakers as a middle ground in the language barrier coming across in scientific writing. Various algorithms can be maintained for the easier implementation of management protocols in remote healthcare set-ups for untrained staff.

As every new technology has its flaws, generative AI comes with its own set of limitations. The most common being ‘AI hallucinations,’ i.e., defined as the set of misinformation presented in a way to appear factual and reliable. There are various types of AI hallucinations, including – sentence contradiction, prompt contradiction, factual contradiction, and irrelevant hallucination.^2,3

The various case scenarios we encounter in healthcare related to AI hallucinations like:⁴

1. Misinformation regarding scientific journal references to the presented paper.

2. Wrong facts regarding patient statistics are presented as facts.

3. Disregarded values of accuracy or compatibility of diagnostic or therapeutic measures related to patient data.

The various methods derived to ensure minimal misinterpretation of data can include:

Correction of source data fed to the LLMs, training to put across legible and correct queries to Chatbots in order to derive information, sensitization among users to reconfirm the information acquired through extensive research, holding CME to put forward the message of applied values of AI and its flaws.

Though AI has its flaws in ethical concerns, like breach of privacy by sharing of data of the patients, or the healthcare decisions being made by relying upon machine expertise.

In conclusion, we believe that AI finds immense potential in the healthcare sector but should be treated with utmost caution in relying totally on its utility as a sole guiding power for future healthcare decisions.

Thank You.