0.7.4 AI Output Is Generated, Not Guaranteed

A Fluent Answer Can Still Be Wrong

Generative AI can produce text that is:

  • clear;
  • confident;
  • detailed;
  • well organized.

Those qualities describe how the answer is presented.

They do not prove that the information is correct.

An AI-generated response can contain an error while sounding completely certain.

That is why appearance and accuracy must be evaluated separately.

The System Is Producing a Likely Response

A generative AI system uses its model, prompt, context, and available tools to generate an output.

It is not automatically checking every statement against an authoritative source before presenting it.

A response can therefore include:

  • correct claims;
  • mistakes;
  • missing details;
  • invented specifics;
  • information that does not apply to the exact situation.

The user should treat the response as generated output, not guaranteed evidence.

Specific Details Can Be Especially Risky

Imagine an AI response says:

The sentence is specific.

That specificity does not prove that the system had access to the real schedule.

The time and field could be:

  • correct;
  • outdated;
  • inferred;
  • or generated incorrectly.

The more a task depends on an exact fact, the more important it becomes to notice whether that fact has real support.

AI Can Fill Gaps With Plausible-Sounding Information

Suppose a user asks:

but the AI does not have the instructor's announcement.

A generated answer may still sound plausible if the system tries to be helpful.

The correct response depends on actual course information that may not be available.

A useful habit is to ask:

Generated Citations Can Also Be Wrong

An AI response may sometimes mention:

  • an article;
  • a book;
  • a website;
  • an author;
  • a quotation.

A citation-shaped detail is still generated output unless the system actually retrieved or was given the source.

Do not assume that a source exists because its title looks realistic.

Current Information Requires Current Access

Some questions depend on information that changes:

  • schedules;
  • prices;
  • weather;
  • software versions;
  • rules;
  • current events.

An AI system may or may not have access to current external information.

A model that cannot access current information cannot reliably know a change that occurred after the information available to it.

Always distinguish:

general knowledge

from:

current factual state.

The Prompt Can Also Cause Problems

If the prompt contains an incorrect assumption, the output may continue from that assumption.

For example:

Activity diagram showing AI generates, then human reads, then human evaluates, then human decides whether the output is usable.
AIGenerates.
  • New
  • Changed
  • Removed

The next Learning Activity focuses on three qualities to examine during that evaluation: accuracy, credibility, and relevance.