Courses · FAC 105 The ethical use of AI in teaching and research · Lesson 2 of 4
Verify everything
Lecture 2 of 4 · about 7 minutes
Before you begin, ask yourself. The lecture answers these questions, and you will meet them again at the end.
- What was wrong with the brief in Mata v. Avianca?
- What is the second of the four checks?
- What do you do with a statement you cannot verify?
In 2023 two lawyers in New York filed a brief in a case called Mata v. Avianca. It cited half a dozen earlier decisions. None of them existed. An artificial-intelligence tool had supplied them, complete with quotations, and had assured the lawyer, when asked, that they were real. The court fined the lawyers and their firm. The judge’s view of the matter is printed below, and it is balanced: using a tool is not improper, and the person who signs is responsible for what is filed.
Canada had its own case within months. In Zhang v. Chen, decided in British Columbia in 2024, a lawyer had included two invented cases in a notice of application and was ordered to bear costs personally. In December 2023 the Federal Court of Canada began asking parties to declare any content in their filings that was generated by artificial intelligence.
The lesson reaches well beyond lawyers. Anyone who puts their name to a document vouches for it. A teacher who gives a class a date, a quotation or a reference vouches for it to thirty people who will repeat it.
So verify. The procedure has four checks, and none can be skipped. First, does the source exist? Find it yourself, in a library catalogue, a court database or an archive. Second, does it say what the tool claims? Open it and read the passage. Tools often cite a real work for something it does not say. Third, is the quotation exact? Compare it word for word. Tools paraphrase and then put quotation marks round the result. Fourth, is the source any good? A real article may be wrong, out of date or written by someone with no standing on the question.
Check figures and dates in the same way, against a primary record wherever one exists. Check names. Be wary of the answer that arrives suspiciously complete, since a gap in the record is often filled with invention.
A useful habit is to treat everything a tool produces as you would a rumour from a clever stranger: possibly right, worth following up, and not to be repeated until you have seen the evidence. If you cannot verify a statement, leave it out, or say that it is unverified. Never soften an unverified claim into the text and hope.
Verification takes time, and that changes what the tools are good for. They save effort in drafting and arranging. They save none in establishing what is true. An instructor who lets a tool write a lesson and then checks every line properly will often find it would have been quicker to begin with the sources.
Teach learners the same four checks, and show them a failure. Ask a tool, in front of the class, for three sources on a point of local history, and then look for them together. Nothing you could say would make the point as well.
In this University the habit has a particular weight. Its officers decide who is entered on the Register from documents. An invented source in an essay is an embarrassment. An invented record in a line of descent is a harm to a family and to the Register.
The source
Technological advances are commonplace and there is nothing inherently improper about using a reliable artificial intelligence tool for assistance. But existing rules impose a gatekeeping role on attorneys to ensure the accuracy of their filings.Mata v. Avianca, Inc., opinion and order on sanctions, 22 June 2023 · United States District Court, Southern District of New York; Judge P. Kevin Castel.
Key terms
- Verification
- Checking that a source exists, says what is claimed, is quoted exactly, and is sound.
- Primary source
- An original record or document, as distinct from an account of it.
- Gatekeeping
- The duty of the person who signs a document to ensure it is accurate.
Check yourself
Summary
Go back to the questions at the top. You should now be able to answer each in a sentence, and to explain these terms: Verification, Primary source, Gatekeeping.