Access Without Understanding

# Access Without Understanding > Why low AI literacy is a problem inside legal teams [Read on Substack](https://lawwhatsnext.substack.com/p/access-without-understanding) · 2026-04-21 · Law What's Next --- Something I find myself saying more and more, in training sessions, in AI adoption discussions, and directly with GCs: there is a meaningful AI risk inside most legal teams right now that is barely being discussed. Low AI literacy. The conversations that dominate the AI risk space in legal tend to be driven by people who are AI-literate. They are talking about data privacy, about hallucination rates in legal research tools, about agentic AI and what happens when you let a system take actions autonomously. Important conversations. But they are being had at one end of the capability spectrum, and they are largely missing what is happening at the other end. What is happening at the other end is this. A very large number of lawyers are now using AI tools every day who do not really understand them. Not in a blameworthy way. They have been handed enterprise AI platforms, often with a session of training that focused on what the tool can do rather than where it breaks, and they have been sent off to get on with it. In many cases there has been top-down pressure to adopt, a sense that resistance is professionally unwise. And so they are using them. Confidently. Without a particularly clear picture of what they are actually doing. What low AI literacy actually looks like If you talk to users about how they use these tools, which I would strongly recommend doing if you are responsible for AI adoption and governance in your function, you start to hear things that should give you pause. A lawyer describes their workflow for reviewing third-party MSAs. The agreement comes in, maybe 100 pages or more, and they drop the whole thing into Copilot alongside a 30-page company playbook and prompt it to review the contract against the playbook. It produces a response. They make the edits and send the document back. They are broadly happy with how this works. This is not wrong in principle. But Microsoft’s own published guidance recommends aiming for documents shorter than around 20 pages, or roughly 15,000 words, when you need Copilot to consider a document as a whole. Above that threshold, their literature flags partial coverage, weaker handling of the middle sections, and more generic answers. Many lawyers using Copilot this way are working well outside the parameters Microsoft themselves have set out, and have no idea. I wrote about a version of this directly in When The Copilot Forgets. The moment when you spot an error in output the tool should have caught, and realise it was never across the context you assumed it was, is a jarring one. It is also an entirely avoidable one if you know where the limits are. Another pattern involves lawyers stretching AI tools across use cases those tools were not built for. A lawyer identifies what feels like a creative use for the AI in their CLM system and starts feeding it compliance data files and investigation materials, asking it to build timelines and surface patterns. The outputs look reasonable. What they do not know is that the CLM’s AI has been tuned specifically for contract work. The system prompt, model selection, and evaluation criteria all optimised around contract language and structure. The outputs may look fine, but the probability of errors is meaningfully higher than they assume, and a greater level of scrutiny is warranted than they are probably applying. The pattern underneath it all What connects these examples is something I have started calling capability generalisation. The user sees the tool perform impressively in a context it was designed for and forms a belief about its general capability. It did a great job on X, so it should do a great job on Y. It worked brilliantly on the five-page NDA in the training session, so it will work brilliantly on this 200-page agreement. This is a logical inference. It is also the wrong one, and the tools themselves do nothing to correct it. This is a quieter version of a problem I explored in The Icarus Directive, which looked at agentic AI and the futility of being told to deploy powerful tools while simultaneously being warned not to use them in any sensitive context. The common thread is tools being used beyond their safe operating range, with the risk neither visible in the output nor acknowledged in the guidance. It is a theme Tom and I explored in depth with Rok Popov Ledinski in this conversation. An LLM operating within its tested competence and an LLM operating well outside it produce outputs that can look functionally identical. Same register, same confidence, same apparent authority. There is no flag in the interface that says this task is outside validated scope. The tool just answers, because answering is what it does. On tasks it was built for, plausibility and accuracy tend to travel together. On tasks it was not built for, you will get plausibility and you have to hope for the rest. The compounding problem of memory and context There is another layer to this. Many lawyers have figured out, quite sensibly, that if they are working on a particular matter they should keep coming back to the same conversation thread so the AI retains context. This is a very human and very logical way to think about it. The problem is that it is not really how these tools work. Performance degrades over long conversation threads. The model’s ability to hold and accurately use earlier context diminishes as the conversation grows, and most providers recommend starting fresh threads and re-providing key context rather than relying on an ever-lengthening conversation to hold everything. Most users do not know this. They are building working practices on a foundation that is slowly becoming less reliable, with no visible indication that this is happening. What to do about it I am not arguing against using these tools, or against stretching them into adjacent use cases. That can be legitimate, and more literate users do it regularly because they can navigate and accept the risk knowingly. But it should be a knowing choice. If you are using a tool outside its stated capability, do it with your eyes open, with a corresponding increase in human oversight, and treat the output as a first pass rather than a work product. The harder problem is that most of the users I am describing are not making a knowing choice. They do not know they are outside the guardrails. They think they are using the tool correctly because it has not told them otherwise and the outputs look fine. In a sense, they have become their own parents navigating social media for the first time, guilelessly encountering the pitfalls, quirks and traps that any informed user would sidestep without thinking. The scam post that looks completely legitimate. The Facebook competition that is definitely real. The AI output that is absolutely thorough. Same dynamic, higher professional stakes. Raising general AI literacy does not require putting every lawyer through a technical education in machine learning. It requires helping people understand that these are probabilistic systems, not intelligent ones in the human sense, that document length and format matter, that conversation threads degrade, and that a tool optimised for contracts is not automatically a tool for compliance files. The users are not the problem to solve. They are the symptom. The problem is the organisations that have prioritised getting people onto these tools over getting people ready to use them. Access without literacy is not a feature of a good rollout. It is a liability. There is also a professional dimension that does not get mentioned enough. Lawyers are trained to know the limits of their own competence and to act accordingly. That obligation does not suspend itself when the work is being assisted by AI. If anything it sharpens, because the tool will never tell you it is out of its depth. That job falls to the lawyer. Which means the lawyer needs to know enough to do it. The legal profession spent years being sceptical about AI. Now that it has largely converted, the risk has not gone away. It has just changed shape. The question is no longer whether lawyers will use these tools. It is whether they understand what they are using well enough to use it safely. On current evidence, for a significant number of them, the answer is not yet. Thanks for reading Law://WhatsNext! Subscribe for free to receive new posts and support our work.