Build an AI-Powered Clinical Patient
And then get feedback on how well you diagnosed their illness.
Of all the potential uses for AI in education, I like the potential to create more low-stakes, high-availability practice the most. Practice for the kinds of high stakes assessments that make you lose sleep the night before and make your palms sweaty the day of. AI can't replace these assessments but if it can offer some practice that takes the edge off and creates more confident students...awesome!
Take the high stakes world of clinical scenarios at medical schools. For the important practice of actually seeing and having a diagnosis conversation with a patient, medical schools have had only a few options:
Hire actor patients which require training, scheduling and facilities.
Use peer-to-peer role play which lacks some authenticity.
Pay for some of the technology-enabled clinical scenario simulators, perhaps using VR or similar technologies. These solutions can be quite rigid and expensive.
With AI, the ability to spin up low-cost virtual patients is almost at the level that most faculty can do it. And the simulation in this article costs about $.03 per run using GPT-4o.
Steps to Build your Interview Practice tool:
Before creating this clinical scenario simulation, I had to actually add a chatbot feature to my AI MicroApps repository. I had avoided chatbots in the past because the world has a lot of chatbot tools at this point and I have built MicroApps to be targeted, precise and guided. But in this case, I needed the ability to have a free-form conversation with a patient before getting into more specific diagnosis and feedback.
Check out the quickstart to build your first “hello world” app: https://jswope00.github.io/AI-MicroApps-Docs/quickstart/
Edit your version of the “app_clinical_chatbot.py” file.
Modify the SYSTEM_PROMPT for your own interview. My prompt for this demo PM interview is “You are an assistant for a clinical simulation exercise for a student user who is playing the role of a doctor. You will answer the user's questions and sometime assess their accuracy.”
The fields in phase 1 are:
Markdown introduction to the scenario
Image of the patient
Chat Input with a max_messages of 10 to simulate a short conversation.
And the phase 1 instructions include all the background that your patient should have. I’ve included patient details, patient history, and primary symptoms. But you might include more information about their immunization history or temperament.
In phase 2 we ask the user for two things:
Text Input for the primary complaint
Long Text Input for a full differential diagnosis
And in the phase 2 instructions we provide the AI with a lot of details about what potential diagnoses and evaluations could be.
Finally there is a scoring rubric that checks for an accurate primary complaint and diagnosis. I haven’t made it very specific but it could be!
Test your app. Once happy with it, deploy your app!
Hi John, thank you for sharing this fascinating approach to leveraging AI for low-stakes, high-availability practice in education. Your example of using virtual patients for medical training is inspiring, and the detailed steps you provided to build the interview practice tool are incredibly helpful. I’m especially drawn to how this can increase student confidence while reducing costs and logistical challenges.
I’ll definitely explore creating something similar! For now, I’ve been using ChatGPT’s customization options to create chatbots, but your approach with AI MicroApps has me thinking of the possibilities. I wanted to ask:
- Is there a way to collect user input (e.g., for analysis or to improve the tool) using the chatbot framework you described?
- If I have about 500 users for my app, do you have an estimate of what the usage cost might be, considering the approximate cost of $0.03 per run using GPT-4?
With that volume of users, do you recommend building and deploying the app from a local machine, or would hosting it via GitHub suffice?
I know I could do more research on these questions, but I thought checking with you might provide the most accurate and practical insights.
Thank you so much for sharing your expertise! Looking forward to your thoughts.
Best regards,
Minh