How to Critically Evaluate GenAI Output

As GenAI tools become more widely used in education, it is imperative that students remain mindful of the capabilities and limitations of GenAI tools. Specifically, when using GenAI to generate content, it is imperative to exercise caution and critical thinking regarding its accuracy and reliability. It is highly advisable to verify the factual accuracy of any generated content, and to supplement it with information from other credible, trustworthy sources. This approach not only ensures the integrity and credibility of the information being presented, but also mitigates the risk of potential biases or misinformation that may be present in AI-generated content.

Everyone should approach GenAI content with the understanding that while these technologies can produce content that seems to be accurate and appropriate, it may not be. This critical analysis is something everybody needs to practice. Humber College identified the following essential considerations:

Content Accuracy

Although GenAI tools can generate grammatically correct text that may seem logical and informative, they can produce content that is misleading and/or incorrect. Always cross-verify AI generated content with trusted resources to ensure accuracy.

Bias Awareness

GenAI can (and will) generate biased or skewed content. It is vital to critically evaluate the output for gender, racial, cultural, and ableist biases.

Ethical Considerations

GenAI is powerful and can create content that seems incredibly human-like. This can lead to misuse, such as generating deepfakes or disseminating false information. As responsible learners, it’s crucial to understand and respect the ethical boundaries surrounding the use of these technologies.

Source Verification

Always verify the source of the information. GenAI tools do not generate information based on new data after their training cut-off. Furthermore, GenAI is widely known to “hallucinate” meaning it will make things up and present the content as the truth.

This 15 sec video is an example of ChatGPT3.5 (free version) ‘hallucinating’ requested academic references:

The following short video includes useful information to help you identify and verify ‘hallucinated’ references:

Attribution: Adapted from Considerations for Your Course (Talk to Your Students) by the Centre for Teaching and Learning at Queen’s University is licensed under CC BY SA 4.0

Attribution: Adapted from Generative Artificial Intelligence in Teaching and Learning Copyright © 2023 by Centre for Faculty Development and Teaching Innovation, Centennial College is licensed under a Creative Commons Attribution 4.0 International License.