Build an AI-Powered Question Feedback Generator

I have a thing for question feedback. On the student side, few thing are more frustrating than getting a question wrong and not knowing why you got it wrong. From a pedagogy perspective, immediate feedback significantly improves student retention and outcomes (source). And when we’ve already written the question, the answer and the distractors, it’s not much more effort to add an explanation! But I’d say that only 20% of questions I get from subject matter experts initially include feedback.

Thankfully, AI does a good job at generating question feedback. By providing the context of the question itself and the answer choices, AI can generate a fairly thorough set of feedback for each correct and incorrect answer without going off script and hallucinating as much. To be clear, it still can and will hallucinate, and so question feedback requires a human in the loop just like most AI processes. But writing feedback that just needs to be verified is a huge improvement over writing feedback from scratch.

This AI powered question feedback generator is pretty simple and just capture a few things:

• The question text with answers (make sure to mark the correct answer somehow)

• Source material so that feedback is tied to lesson content.

• The LMS option I use has a nice option to include hints, so I added a checkbox to include hints too.

Steps to Build your Custom Rubric Generator:

1. Check out the quickstart to build your first “hello world” app: https://jswope00.github.io/AI-MicroApps-Docs/quickstart/

2. Edit your version of the “app_question_feedback” file.

3. Review the phases and fields, and customize as needed.

   1. Phase I

      1. Question and answer Text

      2. Source Material

      3. Hints Checkbox

      4. Number of Hints

4. Review the prompt, and customize as needed. I could probably have done this prompt without conditionals, but I opted to use conditionals so that we only ask the AI for hints if the user checked that option.

   1. "user_prompt": [

      {

      "condition": {},

      "prompt": "Please provide feedback"

      },

      {

      "condition": {"hints": True},

      "prompt": "and {num_hints} hints"

      },

      {

      "condition": {},

      "prompt": "for the following questions(s): \n {question_text}"

      },

      {

      "condition": {},

      "prompt": "Adjust your tone based on the tone of the question text."

      },

      {

      "condition": {},

      "prompt": """If provided, please consider the source material in your hints and feedback. If no source material is provided, ignore this:

      Source Material:

      {source_material}

      """,

      }

5. Deploy our app