Published GenAIMed-related articles

While drug combinations are increasingly important in disease treatment, predicting their therapeutic interactions remains challenging. This review systematically analyzes computational methods for predicting drug combination effects through multi-omics data integration. We comprehensively assess key algorithms including DrugComboRanker and AuDNNsynergy, and evaluate integration approaches encompassing kernel regression and graph networks. The review elucidates artificial intelligence applications in predicting drug synergistic and antagonistic effects.

The commentary discusses the study by Chang et al. on the use of ChatGPT-4 for providing colonoscopy follow-up recommendations based on clinical data. The authors express concerns regarding the randomness inherent in large language models (LLMs) like ChatGPT-4, which can generate inconsistent or contradictory responses to the same input. They emphasize the importance of evaluating the consistency of AI models through repeated queries to capture variability and ensure reliable outputs for clinical decision-making. Additionally, the commentary critiques the method of converting continuous or categorical variables into numerical ones by taking the mean of a range. This simplification may introduce inaccuracies and fail to reflect the clinical flexibility needed in some situations. The authors suggest that more accurate handling of variable transformations and addressing the randomness of LLM responses are crucial for improving the clinical applicability and reliability of AI systems. In conclusion, the commentary advocates for further research to ensure that AI models like ChatGPT-4 provide consistent and precise recommendations before being integrated into clinical practice.

This study evaluated three Gen-AI models (GPT-4-turbo, Gemini-pro-vision, and Claude-3-opus) for lung tumor recognition in CT images, finding Gemini performed best in single-image analysis (92.21% accuracy), followed closely by Claude (91.49%), while GPT lagged (65.22%); performance declined with increased complexity for all models, though Claude showed the most stability; feature analysis revealed strengths in recognizing tumor morphology and margins but significant errors in complex cases; after optimization through regression techniques, performance improved substantially (AUC ~0.89), suggesting promising potential for Gen-AI in simplified lung CT analysis while highlighting current limitations for complex clinical applications.

Biosafety laboratories are crucial for safe research with pathogenic microorganisms, enabling significant advances in microbiology and epidemiology to address emerging infectious diseases. However, laboratory work carries inherent risks without stringent biosafety protocols and training. This study explores whether ChatGPT and other generative AI models can effectively answer biosafety-related questions and enhance laboratory management. Using a dataset of 62 text-based and 8 image-based questions, the performance of several large language models (LLMs) was evaluated. Models such as Gemini Pro, Claude-3, Claude-2, and GPT-4 achieved high accuracy in text-based tasks, while image-based performance was slightly lower. The study discusses the transformative potential of AI-driven systems for real-time monitoring, personalized training, workflow documentation, and predictive maintenance in biosafety labs. Despite their promise, current AI limitations include bias, limited specialized data, and real-time constraints, along with ethical concerns regarding data security and transparency. Recommendations are provided for overcoming technical and ethical challenges to optimize the integration of generative AI in biosafety laboratory management.

The rapid advancements in large language models (LLMs) such as ChatGPT have raised concerns about their potential impact on academic integrity. While initial concerns focused on ChatGPT’s writing capabilities, recent updates have integrated DALL-E 3’s image generation features, extending the risks to visual evidence in biomedical research. Our tests revealed ChatGPT’s nearly barrier-free image generation feature can be used to generate experimental result images, such as blood smears, Western Blot, immunofluorescence and so on. Although the current ability of ChatGPT to generate experimental images is limited, the risk of misuse is evident. This development underscores the need for immediate action. We suggest that AI providers restrict the generation of experimental image, develop tools to detect AI-generated images, and consider adding “invisible watermarks” to the generated images. By implementing these measures, we can better ensure the responsible use of AI technology in academic research and maintain the integrity of scientific evidence.

The commentary emphasizes the potential of artificial intelligence (AI) chatbots in providing empathetic and readable responses to cancer-related questions but raises significant concerns regarding their accuracy and consistency. While AI models are effective in certain aspects like engagement and communication, the authors stress that inaccuracies in the information provided could pose serious risks to patient safety. The inherent randomness of large language models (LLMs) can lead to inconsistent responses, further complicating their reliability. The authors argue that for AI to be safely integrated into clinical settings, models must prioritize accuracy, consistency, and safety, ensuring that they consistently deliver reliable information to patients.

The commentary explores the potential of large language models (LLMs) in combating medical disinformation, acknowledging both the risks and opportunities they present. While agreeing with concerns about LLMs enabling the spread of false medical information, the authors highlight that the problem predates AI technology. They argue that rather than focusing solely on restricting LLMs, efforts should be made to empower users to assess the reliability of online health information. The commentary demonstrates how well-trained LLMs can be powerful tools in identifying and correcting inaccurate medical claims. The authors’ experiment with multiple popular LLMs showed that most responses flagged misinformation, particularly in areas like vaccine safety, by providing evidence-based explanations. These models’ ability to reference the latest authoritative information, such as CDC data, reinforces their potential to challenge health misinformation, including emerging threats like novel infectious diseases. The commentary stresses that the positive potential of LLMs in addressing medical disinformation should not be overlooked.

This paper proposes a comprehensive framework for standardized evaluation of generative AI in medicine, addressing the current lack of standardized assessment methods. The authors recommend a three-pronged approach: establishing standardized scoring criteria with multiple complementary evaluation methods (including Reference Answer Accuracy Rate, Subjective Answer Accuracy Rate, and Strict Accuracy Rate), implementing rigorous evaluation processes (including pre-review alignment, multi-reviewer scoring, and independent audits), and utilizing statistical analysis to quantify scoring differences and refine evaluation methods. The recommendations aim to enhance reliability and consistency in assessing generative AI’s capabilities in healthcare applications, acknowledging both the technology’s potential and the need for careful validation before clinical implementation.

The introduction of ChatGPT-4V’s ‘Chat with images’ feature represents the beginning of the era of large multimodal models (LMMs), which allows ChatGPT to process and answer questions based on uploaded images. This advancement has the potential to transform how surgical teams utilize radiographic data, as radiological interpretation is crucial for surgical planning and postoperative care. However, a comprehensive evaluation of ChatGPT-4V’s capabilities in interpret radiological images and formulating treatment plans remains to be explored. Three types of questions were collected: (1) 87 USMLE-style questions, submitting only the question stems and images without providing options to assess ChatGPT’s diagnostic capability. For questions involving treatment plan formulations, a five-point Likert scale was used to assess ChatGPT’s proposed treatment plan. The 87 questions were then adapted by removing detailed patient history to assess its contribution to diagnosis. The diagnostic performance of ChatGPT-4V was also tested when only medical history was provided. (2) We randomly selected 100 chest radiography from the ChestX-ray8 database to test the ability of ChatGPT-4V to identify abnormal chest radiography. (3) Cases from the ‘Diagnose Please’ section in the Radiology journal were collected to evaluate the performance of ChatGPT-4V in diagnosing complex cases. Three responses were collected for each question. ChatGPT-4V achieved a diagnostic accuracy of 77.01% for USMLE-style questions. The average score of ChatGPT-4V’s treatment plans was 3.97 (Interquartile Range: 3.33-4.67). Removing detailed patient history dropped the diagnostic accuracy to 19.54% (P<0.0001). ChatGPT-4V achieved an AUC of 0.768 (95% CI: 0.684-0.851) in detecting abnormalities in chest radiography, but could not specify the exact disease due to the lack of detailed patient history. For cases from ‘Diagnose Please’ ChatGPT provided diagnoses consistent with or very similar to the reference answers. ChatGPT-4V demonstrated an impressive ability to combine patient history with radiological images to make diagnoses and directly design treatment plans based on images, suggesting its potential for future application in clinical practice.

Prostate cancer (PCa) is a common malignancy in males. The pathology review of PCa is crucial for clinical decision-making, but traditional pathology review is labor intensive and subjective to some extent. Digital pathology and whole-slide imaging enable the application of artificial intelligence (AI) in pathology. This review highlights the success of AI in detecting and grading PCa, predicting patient outcomes, and identifying molecular subtypes. We propose that AI-based methods could collaborate with pathologists to reduce workload and assist clinicians in formulating treatment recommendations. We also introduce the general process and challenges in developing AI pathology models for PCa. Importantly, we summarize publicly available datasets and open-source codes to facilitate the utilization of existing data and the comparison of the performance of different models to improve future studies.

This correspondence discusses the assessment of generative AI in the field of medicine, specifically focusing on ChatGPT’s capabilities in passing the American Heart Association (AHA) Basic Life Support (BLS) and Advanced Cardiovascular Life Support (ACLS) exams. The authors critique earlier studies that attempted to evaluate ChatGPT-3.5’s performance on these exams, noting that the lack of repeated testing may have underestimated its abilities. The authors’ subsequent study, which involved multiple rounds of questioning, found that ChatGPT-3.5 was capable of passing both exams. However, earlier limitations such as the inability to process image-based questions were overcome with the release of ChatGPT-4V, which was tested on complete AHA exams. While acknowledging the importance of exams in assessing knowledge, the authors argue that evaluating AI’s clinical potential should extend beyond these exams. Just as medical students progress through various stages of evaluation—beginning with written exams and advancing to case-based assessments and clinical practice—the authors suggest that generative AI should also be assessed through simulated clinical scenarios that reflect real-world applications. They emphasize the need for comprehensive evaluations of AI tools in healthcare, moving towards practical, real-world testing to better understand their potential impact in clinical practice.

This study highlights the significant language and cultural biases inherent in large language models (LLMs), particularly in the context of Traditional Chinese Medicine (TCM). The research compares the performance of eight prominent LLMs—four developed by Chinese companies and four by Western companies—on the National Medical Licensing Examination for TCM. The results revealed that Chinese-developed models, such as Qwen-max and GLM-4, significantly outperformed their Western counterparts, such as ChatGPT-3.5 and ChatGPT-4, in answering TCM-related questions. The Western models, with their training predominantly on English-language data, struggled with the cultural nuances and specific terminology used in TCM. The study emphasizes the need for localized models that are trained on culturally relevant data to ensure better accuracy and applicability in specific fields, like TCM. It suggests that AI models tailored to local languages and medical practices can offer more precise solutions and meet the unique needs of different populations. Additionally, these localized models could contribute to preserving cultural knowledge, improving data security, and addressing specific healthcare needs. In conclusion, the research advocates for the development of AI systems that integrate multilingual and culturally diverse training to enhance their effectiveness in global healthcare settings.

This study evaluated the capabilities of the newly released ChatGPT-4V, a large language model with visual recognition abilities, in interpreting electrocardiogram waveforms and answering related multiple-choice questions for assisting with cardiovascular care.

The increasing interest in the potential applications of generative artificial intelligence (AI) models like ChatGPT in health care has prompted numerous studies to explore its performance in various medical contexts. However, evaluating ChatGPT poses unique challenges due to the inherent randomness in its responses. Unlike traditional AI models, ChatGPT generates different responses for the same input, making it imperative to assess its stability through repetition. This commentary highlights the importance of including repetition in the evaluation of ChatGPT to ensure the reliability of conclusions drawn from its performance. Similar to biological experiments, which often require multiple repetitions for validity, we argue that assessing generative AI models like ChatGPT demands a similar approach. Failure to acknowledge the impact of repetition can lead to biased conclusions and undermine the credibility of research findings. We urge researchers to incorporate appropriate repetition in their studies from the outset and transparently report their methods to enhance the robustness and reproducibility of findings in this rapidly evolving field.

Ensuring the authenticity and reliability of scientific research is a critical responsibility in academic medicine, as image manipulation severely undermines scientific integrity and may misdirect resources and further investigations. Recent data indicate that approximately 1 in 25 biomedical articles contains inappropriate duplicate images, prompting a rising number of retractions. Advances in artificial intelligence (AI), such as the large multimodal model ChatGPT-4 Vision (ChatGPT-4 V), offer novel tools for detecting image manipulation. In this study, we assessed ChatGPT-4 V’s capability to identify manipulated images by testing it on representative samples from 12 biomedical articles suspected of containing image duplication or fraudulent modifications. The model performed well in identifying simple forms of image duplication and splicing, but its effectiveness declined with more complex manipulations such as rotation and flipping. While the integration of such AI systems holds promise for enhancing research oversight, important challenges remain, notably the potential for false positives and negatives and the limited representativeness of training data for certain research fields. Close collaboration between medical experts and AI developers is needed to optimize these tools’ reliability and applicability. Our preliminary findings underscore the potential of multimodal AI in protecting scientific integrity and highlight the continuing essential role of human expertise in the research validation process.

ChatGPT-4V model with image interpretation tested for distinguishing kidney & prostate tumors from normal tissue.

The study by Fijačko et al. tested ChatGPT’s ability to pass the BLS and ACLS exams of AHA, but found that ChatGPT failed both exams. A limitation of their study was using ChatGPT to generate only one response, which may have introduced bias. When generating three responses per question, ChatGPT can pass BLS exam with an overall accuracy of 84%. When incorrectly answered questions were rewritten as open-ended questions, ChatGPT’s accuracy rate increased to 96% and 92.1% for the BLS and ACLS exams, respectively, allowing ChatGPT to pass both exams with outstanding results.

This correspondence explores the best approach to assessing generative AI, particularly in the context of medical applications like ChatGPT. The authors discuss previous studies evaluating ChatGPT’s ability to pass the American Heart Association (AHA) BLS and ACLS exams, noting that earlier assessments underestimated its potential due to methodological limitations, such as the lack of repeated testing and exclusion of image-based questions. With the release of ChatGPT-4V, these limitations were overcome, allowing for more comprehensive testing. The authors argue that evaluating AI’s capabilities should not be confined to exams alone. Similar to how medical students progress from basic knowledge assessments to real-world clinical practice, AI should also be assessed in simulated clinical scenarios to better understand its application in healthcare. The authors advocate for comprehensive evaluations of generative AI that move beyond theoretical exams to real-world clinical simulations, enabling a deeper understanding of its potential impact in clinical settings.