Evaluating AI outputs

The checks you need depend on what you plan to do with the output and what could happen if it is wrong. Before relying on it, ask:

• What am I using this output for?

• Which parts will I learn from, repeat, act on or include in my work?

• Does it contain factual claims, recommendations, calculations or instructions?

• Could an error affect my assessment, research, placement or wellbeing?

• Can I check the important parts using appropriate evidence or expertise?

A suggestion used for early brainstorming may only need to be checked for relevance and usefulness. A claim used in an assessment, a technical instruction or advice with serious consequences needs direct evidence, independent testing or help from someone with appropriate expertise.

You do not need to verify every word of a low-risk brainstorming response. You do need to check anything important that you intend to rely on, learn, repeat, act on or submit.

AI outputs can sound authoritative whether they are accurate or not. None of the following proves that an answer is reliable:

• confident, fluent or polished writing

• technical language or precise-looking detail

• links, quotations or references

• the same answer appearing more than once

• a similar answer from another AI tool

• the AI saying that it has checked its work

Links and references give you something to investigate, but they do not show by themselves that the source exists, is suitable or supports the claim. Two AI tools may draw on the same widely repeated material and make similar mistakes.

You can ask an AI tool to identify assumptions, uncertainties or possible objections in its answer. This may reveal questions worth investigating, but it is not independent verification.

Work through the parts of the output that matter to your task:

1. Check the task. Compare the output with your original question, instructions, assessment brief or marking criteria. Look for requirements it has ignored, changed or invented.

2. Identify the claims. Separate checkable facts from suggestions, interpretations and conclusions. Do not judge the whole answer as simply reliable or unreliable.

3. Verify important information externally. Use an appropriate original or authoritative source, course material, test, calculation or person with relevant expertise.

4. Check the connection. Make sure the evidence supports the precise claim being made. Correct facts can still be combined into an unsupported conclusion.

5. Check scope and certainty. Look for missing qualifications, limitations, exceptions or uncertainty. AI summaries can make findings sound broader or more certain than the original material.

6. Look for gaps. Consider whether relevant evidence, alternative explanations, affected groups or areas of disagreement have been omitted.

Your own academic and disciplinary judgement is essential. If you do not understand an output well enough to explain and defend it, do not rely on it in your work.

For the fuller process of checking sources, claims and an AI-generated synthesis during research, see AI for researching a topic.

Different outputs need different forms of checking.

• An explanation or answer: Compare important claims with course materials and appropriate original or authoritative sources. Check that examples really illustrate the concept and that relevant exceptions have not been lost.

• A summary: Compare it directly with the original material. Check the purpose, scope, population, method, findings, limitations and degree of certainty where these are relevant.

• Suggested sources or a source-based answer: Find and read the originals. Check that each source exists, is appropriate and supports the claim attached to it. Use AI for researching a topic for the complete process.

• Feedback on your work: Compare it with the assessment brief, marking criteria and guidance from your tutor. Decide whether the suggestion improves your work without changing your intended meaning, argument or voice.

• Ideas or a plan: Check relevance, feasibility and fit with the task. Look for predictable suggestions, missing alternatives and assumptions about the time or resources available.

• A calculation, code or data analysis: Check the inputs, units, formulae, assumptions and method. Reproduce the calculation, run and inspect the code, or test the result using known cases. Seek subject-specific help if you cannot check it yourself.

• A translation: Check important terminology, meaning and tone against the original. For specialist or consequential material, use an authoritative translation or someone proficient in the language and subject.

Generated images, audio and video also require checks for accuracy, misleading presentation, permitted use and whether their AI origin needs to be made clear. Ask your tutor for subject-specific guidance where these outputs form part of an assessment.

For guidance on using AI to understand or summarise an academic paper, see AI for academic reading.

A reliable source does not automatically make the AI's answer reliable. The tool may misunderstand the source, attach it to the wrong claim, remove important context or select only evidence that supports one view.

First evaluate the original source itself. Then check how accurately and fairly the AI output has used it. Evaluation Frameworks Explained can help you investigate webpages, publications and other sources, but a source-evaluation framework does not validate the generated answer or synthesis for you.

Do not rely on an AI output when:

• you cannot verify an important part and accuracy matters

• you do not understand it well enough to identify or check possible errors

• the task requires comprehensive research, direct engagement with original material or your own analysis

• authoritative, professional or subject-specific advice is required

• the proposed use is not permitted by your assessment brief

Use an original or authoritative source, complete the required research or testing, or ask your tutor, supervisor or another appropriately qualified person. Sometimes the responsible decision is to discard the AI output.