Summarising a legal document with ChatGPT

Legal documents are a good fit for this in one narrow way. They are long, densely cross referenced, and written so that the part you need is rarely where you would look for it. Finding things in that kind of text is exactly what a language model does well.

The risk is that the output reads like advice. A summary of a judgment comes back in the same confident prose whether the model understood the ratio or invented it, so checking has to be a habit rather than a reaction to something looking wrong.

Confirm it has the whole document

Legal drafting puts the expensive material at the back. Schedules, annexes, definitions, appendices of judgments. Text files are truncated at 2 million tokens with no visible warning, and files themselves are capped at 512 MB, so a long agreement or a bundled judgment can arrive as a fragment that still summarises confidently.

Before summarising: list every numbered section, schedule and annex in this document, in order, with the first few words of each.

If that list stops short of what you can see in your own PDF, stop. Everything after this point is a summary of a fragment, and you now know which part it never read.

Ask it to quote and locate, not to conclude

The difference between a useful summary and a dangerous one is usually the shape of the prompt.

Summarise this in plain language. For every point you make, give the clause or paragraph number and quote the sentence it rests on. If you cannot find a sentence that supports a point, say so rather than filling the gap.

Quotes are checkable. Conclusions are not. A quoted sentence either appears in your file or it does not, and finding out takes one search. Asking for source text also pushes the model towards retrieval and away from what a document of this kind normally contains.

Never trust a negative answer

This is the single most important habit, and it applies to every legal document you will ever put through this.

When you ask whether there is an arbitration clause and the answer is no, the model has not read the document end to end and confirmed an absence. It has searched, failed to retrieve, and phrased the failure as a fact about the document. Those are different claims and it cannot tell them apart.

The failure modes stack up. The clause sat past the truncation point. It was worded in a way the search did not match, an agreement to refer disputes to a tribunal rather than the word arbitration. It lived in a schedule that came through as an image and was discarded, since text only, except Enterprise. A confident no covers all of these identically.

So a negative answer is a prompt to check, not an answer. Search your own copy for three or four words that any version of the clause would contain. Absence is only ever established by you.

Different documents need different questions

The last column matters more than the first. Everything in it is a judgement call rather than a reading task, and judgement is where the output stays fluent while the accuracy quietly leaves.

The verification pass

Four checks, two or three minutes, and they catch nearly everything.

1. Search for one quote

   Take a quoted sentence from the middle of the summary and search your own file for a distinctive phrase inside it. If it is not there, discard the summary and start again.

2. Check the numbering runs continuously

   Ask it to list the section numbers it can see. Gaps in the sequence usually mean truncation rather than unusual drafting.

3. Ask the same question twice, worded differently

   Ask about the liability cap, then ask what limits either party's exposure. Two different answers mean it is reconstructing rather than retrieving.

4. Verify every absence yourself

   Anything the summary says is missing gets a manual text search before you act on it.

Where the line sits

Reading is a task. Advice is a judgement. The model does the first and imitates the second.

It can save you an hour on a ninety page judgment by pointing at the twelve paragraphs that carry the reasoning. It can turn a policy stack into the four obligations that apply to your team. All of that is locating and rephrasing text that is in front of it.

What it cannot do is tell you whether a term is enforceable where you live, whether the deal is worse than market, or whether something is unusual enough to argue about. Those need someone who knows your jurisdiction and your facts, and the tone of the answer never changes at the point where you cross from one category into the other.

The practical decision

Use it for the first read of anything long, always with quotes and clause numbers attached. Use it to build a list of what you need to look at properly. Verify every absence by hand. Then take the marked up list to whoever is qualified to act on it, which costs less and goes faster than handing them an unread document and a vague worry.