AI in GUTB: The concept of a holistic multi-modal approach

Part 1: Introduction and the basis of this revolutionary approach to AI in healthcare

TB is a worldwide scourge. MDR-TB remains a major concern, with only 44 per cent of 2023’s estimated cases receiving treatment. GUTB accounts for approximately 9 per cent of EPTB cases, worldwide (1). The goal of TB elimination by 2050, can be facilitated immensely by AI, vide a major contribution in all key metrics. Although AI will soon be ubiquitous, it is still far from replacing radiologists. Since the early days of AI, I have been persistent about concept-based AI incorporating a holistic multimodal approach.  Make the machines think like us. Not the other way around. After years of perseverance, I got this concept published in 2022, in both Radiology and AI journals (1,2). 

Much of the published literature in Radiology-related Artificial Intelligence (AI) focuses on single tasks,  such as identifying the presence or absence or severity of specific lesions. Large Language Models (LLMs) are impressive with that. However, as humans use multiple problem-solving strategies, applying each as needed, realistic AI solutions must combine multiple approaches. Also, since clinical radiologists rarely achieve a diagnosis through images alone, radiology AI must be able to employ diverse strategies that consider all available evidence, not just imaging information (1). I’ve always maintained Artificial  Intelligence needs real Intelligence to guide it! (1,2) Truly intelligent humans are distinguished from the merely smart by intellectual humility and flexibility: as noted in Robson’s “The Intellect Trap”(3), they constantly consider the possibility of being wrong and abandon long-held beliefs when these are invalidated by new evidence. AI approaches must be flexible too. 

We are drowning in information while starving for wisdom. To maximise AI applications’ accuracy and utility in medical diagnosis and treatment modalities, AI must incorporate experiential wisdom accumulated over decades of clinical and radiological experience time; namely time-tested key medical  ‘teaching’ and/or key ‘clinical’ parameters, including prognostic indicators (1,2). TB is no exception.  Emphasis is thus laid upon the concept of adding Wisdom (experience/expertise acquired over the years) to Knowledge ( Big Data etc.), for better outcomes.  

Let’s look at some examples to help understand this better. 

[A] Childhood pulmonary TB (< 15 years), represents 12 per cent of new cases, but 16 per cent of the estimated 1.4  million deaths (4). This higher mortality highlights the urgency for improving case detection. Early diagnosis and prompt treatment will prevent spread to other children at school, or in community settings,  especially in resource-limited settings (4). Imaging algorithms can thus play an important role in screening strategies. 

While DL excels at recognising individual patterns (most artificial vision applications use it), a higher level of knowledge of key imaging and clinical signs allows for integrating individual patterns into a diagnosis. Such “Holistic” multi-modal algorithms that integrate all the available information – not just on a single patient; but also, on molecular and epidemiologic knowledge – can significantly improve, not only early detection of TB, including MDR-TB, but more effective management and significant improvement in healthcare outcomes (1,2). Incorporating data from novel imaging modalities too, or from translational applications of bench-science research (e.g., detection of resistance mutations through  PCR, augmented optionally by CRISPR), will make this concept of a holistic multi-modal approach more useful. It is also worth looking into the role of GenXpert and other such DNA amplification techniques, in not only detecting TB within 2 hours but also using them to detect MDR TB upfront  (rather than wait for 4 months); as well as the vital need for Vit D to be utilised, for better healing as well as complementing the various anti-TB regimens used (1).  

[B]. Here we see an example of human wisdom completely changing the course of a young girl’s life. I  encountered a young girl who had been repeatedly evaluated under general anaesthesia for her urinary incontinence, searching for a possible ectopic ureter as a cause for the same. Both, surgeons as well as the child and her family were completely exasperated, after 5 unsuccessful attempts at examining under  GA; not to mention the trauma and the scars that would be left behind in the young girl’s mind. The girl was then referred to our department for a Multidetector CT exam to look for an ectopic ureter. My colleagues excitedly came to me stating they had solved this girl’s riddle – announcing there was no evidence of an ectopic ureter, but she had evidence of a vesicovaginal fistula (VVF) on MDCT. I asked them a basic question – “Since when was she incontinent”? When they said- “since birth”, I politely told them a congenital vesico-vaginal fistula is almost unheard of and began my review of the case; starting with the Plain Film ( as I usually do [with an abdomen & pelvis film in this case] ). As soon as  I saw the plain film, I promptly diagnosed Female Epispadias! They were flabbergasted. But Sir that’s a clinical diagnosis; and you haven’t even seen the patient, besides the CT showing a VVF? I calmly told them although it was appearing thus, the visualisation of contrast in the vagina on CT was not due to a  VVF, but due to back- spill of contrast from the ‘open urethra’, due to the hypospadias; and to get the girl examined by a senior Pediatric Surgeon to confirm the same. They reluctantly did that and when the senior Pediatric Surgeon confirmed a diagnosis of female hypospadias, an exceedingly rare diagnosis, they came rushing back to me to enquire as to what secret clue did I see? I showed them the very mild increase in the inter-pubic distance that my eagle eyes (loaded with the wisdom acquired over 4 decades)  had picked up and emphasised that any increase in the inter-pubic distance in such cases could indicate an underlying Exstrophy-Epispadias Complex ( of which hypospadias is the least extreme of cases; and can easily remain hidden in a girl ) and that the simple plain film should never be ignored as it can provide invaluable clues. 

It is thus easy to see what wisdom can add to just knowledge and maybe it’s time to move on from uni-modal LLMs to multi-modal AI, especially those based on Large Concept Models (LCMs). Despite our suggestions in the literature as early as 2022 (1,2), the implementation of the idea of LCMs was published by Barrault L, et al; from META’s FAIR (fundamental Advancing AI through Fundamental and Applied Research) team, only in December 2024 (6). Albeit this paradigm shift from LLMs to LCMs will bring about a dynamic change in the way AI is utilised in Healthcare, as LCMs work more like the human brain. In the words of world-renowned AI scientist Eliot B. Lance, LCMs can devour sentences and adore concepts (7). They allow for cross-modality integration and conceptual reasoning and prediction (8). Unlike LLMs, an LCM goes deeper, attempting to emulate human cognitive processes by constructing its frameworks from the very building blocks of human thought (9). 

These LCM-based AI systems will be designed to learn from experience, and adapt to new situations, without being explicitly programmed; getting one step closer to the current ultimate goal of AI, i.e. to create machines that can simulate human intelligence, including reasoning, problem-solving, and creativity. Hopefully, Large Language Models (LLMs) could be replaced by Large Concept Models (LCMs),  starting in 2025 itself.  

Pathognomonic and other vital imaging signs related to GU TB, along with clinical signs, lab tests and the rest can be easily incorporated into such algorithms, albeit with the wisdom acquired over the years.  Additionally, for our TB elimination goal, utilising epidemiological data and dashboards that summarise the data therein ( amongst other data ) will facilitate timely decision-making; along with enhanced computing Infrastructure to facilitate all the above; from optimised data gathering to more sophisticated algorithms, to more powerful hardware architectures. This integration must be guided by policies developed by the coordinated actions of international consortia (including bodies like WHO, UNICEF,  Big Pharma, national health ministries, philanthropists, etc.) that make use of diverse expertise around the globe, including those available through leading-edge technologies (1). An outline for the implementation of such policies, will be provided, vide a pneumonic ‘TB-REVISITED’ (1) (Figure 2). Imagine dashboards monitored by LCM-based algorithms, hooked up worldwide, focusing on TB REVISITED. Would help immensely in our fight to eliminate the scourge of TB, right? What’s next?  Quantum Computer-based AI, true Artificial Super Intelligence! 

Albert Einstein famously said, “Creativity is intelligence having fun.” Concept-based multi-modal AI is here—no need to learn coding etc. The machines will do that. Let your intelligence have fun. Get going with your concepts. The World is yours !!

References: 

1. Merchant SA, Shaikh MJS, Nadkarni P. Tuberculosis conundrum – current and future scenarios: A  proposed comprehensive approach combining laboratory, imaging, and computing advances. World J  Radiol. 2022 Jun 28;14(6):114-136. doi: 10.4329/wjr.v14.i6.114. PMID: 35978978; PMCID:  PMC9258306. 
2. Nadkarni P, Merchant SA. Enhancing medical-imaging artificial intelligence through holistic use of time-tested key imaging and clinical parameters: Future insights. Artif Intell Med Imaging 2022; 3(3):  55-69. DOI: 10.35711/aimi.v3.i3.55 
3. Robson D. The Intellect Trap. W. W. Norton & Company; 2019. ISBN: 978-0393651423 

4.Vonasek B, Ness T, Takwoingi Y, Kay AW, van Wyk SS, Ouellette L, Marais BJ, Steingart KR,  Mandalakas AM. Screening tests for active pulmonary tuberculosis in children. Cochrane Database Syst  Rev 2021; 6: CD013693 [PMID: 34180536 DOI: 10.1002/14651858.CD013693.pub2] 

5. Lee MS, Leung CC, Kam KM, Wong MY, Leung MC, Tam CM, Leung EC. Early and late tuberculosis risks among close contacts in Hong Kong. Int J Tuberc Lung Dis 2008; 12: 281-287 [PMID: 18284833]. 
6. Barrault L, et al. Large Concept Models: Language Modeling in a Sentence Representation Space.  Cornell University: Computer science>computation and language. https://arxiv.org/abs/2412.08821.  https://doi.org/10.48550/arXiv.2412.08821 
7. Eliot L. AI Is Breaking Free Of Token-Based LLMs By Upping The Ante To Large Concept Models  That Devour Sentences And Adore Concepts. Forbes>Innovation>AI at forbes.com. Jan6, 2025. 

https://www.forbes.com/sites/lanceeliot/2025/01/06/ai-is-breaking-free-of-token-based-llms-by upping-the-ante-to-large-concept-models-that-devour-sentences-and-adore-concepts/ 

8. Shrikhande A. A deep dive into Large Concept Models ( LCMs ). The Association of Data Scientists.  Published on January 6, 2025. 

https://adasci.org/a-deep-dive-into-large-concept-models 

lcms/#:~:text=Real%20World%20Applications-,What%20is%20a%20Large%20Concept%20Model% 20(LCM)%3F,for%20conceptual%20reasoning%20and%20prediction. 

9. Danda R. Large Concept Models: The Next Revolutionary Frontier In AI. Forbes Technology Council  post. Jan 23, 2025. 

https://www.forbes.com/councils/forbestechcouncil/2025/01/23/large-concept-models-the-next revolutionary-frontier-in-ai/