Build an AI-Powered Case Study Assessment

An AI-Powered assessment, guided by the instructor voice, that can give students more practice on their own time with critical thinking

One of the most exciting opportunities I see with the recent popularization of genAI is the ability to create this new category of “AI-powered, Instructor-guided” assessment. Instructor feedback remains the pinnacle of academic feedback and the basis many times for student learning. But instructor time is limited, so we supplement feedback with automated feedback i.e. with MCQ exams and also by things like peer feedback. Now, we can offer AI feedback but ground that feedback in the instructor voice.

Case studies can be a good example for this. Case studies are a valuable learning tool, but generally require instructor feedback, in-class discussion, or something similar to deliver them.

That's where AI comes in, offering a unique opportunity for AI-powered, yet instructor-guided feedback. This not only enhances the learning experience but also allows students to engage in critical thinking practice at their own pace. Imagine being able to build up a store of practice case study assignments that your students could use to improve their critical thinking or build their confidence in between the more high stakes graded assignments.

This AI powered case study assessment tool gives students asynchronous practice by:

• Asking specific questions (created by the instructor) to the students about a case study

• Providing feedback based on the instructor guidance for student responses.

• Optionally, scoring responses to determine if the student did well enough on the question to progress.

Steps to Build your Case Study Feedback tool:

1. Check out the quickstart to build your first “hello world” app: https://jswope00.github.io/AI-MicroApps-Docs/quickstart/

2. Edit your version of the “app_hotel_case_study.py” file.

3. Modify the SYSTEM_PROMPT for your own case study. For short case studies, we can place the whole case study in the system prompt which will be more accurate than using RAG.

4. Review the phases and fields, and customize as needed.

   1. Phase I

      1. What is the case study about?

   2. Phase II

      1. Identify two specific risks that can arise from scaling a boutique hotel, and how each risk might be mitigated.

   3. Phase III

      1. What strategies should Sarah and the Harpers prioritize to scale the hotel’s operations without diluting the personalized guest experience that distinguishes The Grand Horizon? How can they allocate limited resources effectively to sustain growth?

5. Review the prompt, and customize as needed. I could probably have done this prompt without conditionals, but I opted to use conditionals so that we only ask the AI for hints if the user checked that option.

   1. Phase I

      1. phase_instructions: “I will summarize the case study. Please critically review my response for accuracy and effort. You are looking for some combination of the facts that the case study is about a hotel, and the hotel needs to differentiate itself in order to grow, and that growth presents challenges that need to be overcome.”

      2. user_prompt: “{about}”

   2. Phase II

      1. phase_instructions: “I will offer two specific risks that can arise from scaling a boutique hotel, and how each risk might be mitigated. Provide me feedback based on:

         1. I provide two risks (not less)

         2. My risks are specific and grounded in the case study.

         3. I provide realistic mitigation strategies for each

         As the expert, you would speak to the challenge of reputation damage for a small hotel with only one location, mitigated by prompting customers for feedback before they get to the internet. You might also speak to employee burnout mitigated by good training and flexible scheduling. But don't penalize me for answers that are different from yours. “

      2. user_prompt: "{risk}”

   3. Phase III

      1. phase_instructions: “I will offer suggestions on how Sarah and the Harpers can prioritize to scale the hotel’s operations without diluting the personalized guest experience that distinguishes The Grand Horizon. Review my answers based on the following criteria:

         1. I reference the case study

         2. My arguments make sense.

         3. My arguments are realistic.

         As the expert, you would speak to hiring and onboarding the right people as being critical to maintaining a culture of personalization as we scale. But don't penalize me for answers that are different from yours.”

      2. user_prompt: “{scale}”

6. Test your app. Once happy with it, deploy your app!