From Curiosity to Capability: the early enterprise generative AI adoption playbook

Paper One argued that early advantage in enterprise generative AI belonged to the organisations that learned fastest, and that speed of deployment was a poor proxy for it.

What did this paper say companies should do about generative AI in early 2023?

It said they should treat the first quarter of a technology’s public life as a learning problem and run a small number of bounded experiments under a named owner. The paper carries the date 31 January 2023, and everything in it describes the position as it stood on that day, two months after OpenAI made a conversational system publicly available as a free research preview. Within a week, its chief executive said publicly that the preview had passed a million users. The paper was written for organisations discovering that their employees had already begun, without being asked to and without being told they could not.

The argument ran against the instinct of the moment. An organisation that moved quickly to deploy would find it had deployed something it could not evaluate, because the evaluation capability existed nowhere at that point, including inside the companies building the systems. An organisation that prohibited use would find that use continued and that it had lost the ability to see it. The position the paper recommended sat between those two, and it worked as a learning system.

• Conversational access changed who could reach a large language model, while accuracy, provenance and accountability stood where they had stood in October 2022.
• Fluency was a property of the output’s surface, and the system’s own documentation said it wrote plausible-sounding but incorrect answers.
• Identical questions phrased differently produced different results, which weakened any control that depended on a reader spotting an error by eye.
• The proposed filter screened tasks by the consequence of error, scoring value, data sensitivity, verifiability, reversibility and consequence.
• Task time, rework, error type and incident count were observable within 30 days, and any return on investment figure offered that quarter was a projection wearing the clothes of a result.

Why was rapid deployment the wrong bet in early 2023?

Deployment at speed put a system into production that the organisation had no means of assessing. Evaluation methods for generative output were immature everywhere that quarter, and the paper made the point that they were immature inside the companies building the models too. A pilot producing fluent output and no quality evidence told its sponsor little. The paper’s word for that was a demonstration, and in early 2023 demonstrations were the cheapest thing in the field.

Prohibition failed for a different reason. Use had already begun, so a ban converted visible use into invisible use and removed the organisation’s only source of evidence about what was actually happening on the ground. What the paper asked for instead was a small set of experiments with owners, baselines and written stop conditions, together with a record of what happened kept somewhere the next person could find it.

What did the use-case filter screen for?

The filter scored value, data sensitivity, verifiability, reversibility and consequence, and it rejected any task failing the last three however attractive the first two looked. That ordering carried the argument. A task whose output could be checked, whose effects could be undone and whose failure cost little was a task an organisation could learn from safely, whatever its headline value happened to be. Screening on enthusiasm produced the opposite selection, because the applications that excited people most were often the ones where an error travelled furthest before anybody caught it.

What could a 30-day pilot honestly measure?

Four measures were observable inside a month. Those were task time, rework, error type and incident count, and each of them required a baseline taken before the pilot started, which is why the paper treated a missing baseline as the defect that turns a pilot into a demonstration. A named owner and a written stop condition did the same work, since they fixed in advance who would call a halt and on what evidence.

Return on investment sat outside that window. Any figure offered for it in the first quarter was a projection dressed as a result, and the paper said so plainly. The measurements it did endorse were modest, and their modesty was the point, because a small honest number about rework tells a board more than a large invented number about productivity.

Does a paper written in January 2023 still hold value?

Its value now lies in being a record of what could responsibly be claimed at a particular moment, before the evaluation tooling, vendor assurances and regulatory frameworks of later years arrived. Its observations about model behaviour, user numbers and the state of practice belong to that quarter alone. They describe January 2023. What survives is the method, which is the habit of screening on consequence of error, insisting on a baseline, naming an owner and writing down what happened.

The paper’s claim about cost is worth highlighting. What it proposed cost nothing in new technology, budget lines or reorganisation. It required somebody named to be accountable for a handful of experiments, and a written record kept where a successor could find it. The proposal belongs to governance, which is why it reads as calmly now as it must have read oddly at the time.

Read in 2026, the paper is a snapshot of a quarter in which very little could be known and a great deal was asserted. Its usefulness lies in how it handled that gap, by setting a deliberately low bar for what counted as evidence and then refusing to step over it. The wager it made was that an organisation able to say what it had tried, what that cost, where it failed and who had been responsible would hold something durable, and that the record itself was the capability the title points to. Everything else in the paper serves that single idea.

Frequently asked questions

What changed for organisations when OpenAI released its chatbot in November 2022?

Access changed. On 30 November 2022 OpenAI made a conversational system available as a free research preview, and a capability that had existed in research settings for several years became reachable by anyone who could type. The paper called that a distribution event. Reaching a large language model stopped requiring an API key, a budget line and somebody who could write the calling code.

Does fluent AI output mean the answer is accurate?

Fluency is a property of the surface of the text, and quality has to be established separately. The paper made that point using the system’s own documentation, which stated that it wrote plausible-sounding but incorrect answers. In early 2023 accuracy, provenance and accountability stood where they had stood in October 2022, and only the means of access had moved. Readers checking output by eye had a weak control.

Why did the same question produce different answers?

Identical questions phrased differently produced different results, a behaviour recorded in the system’s own documentation at the time. The paper drew a governance conclusion from it. Any control that depends on a reader noticing an error by reading is already weak, since a second run may reproduce neither the wording nor the fault. Instability of that kind pushed the paper towards screening tasks by the consequence of error.

Who should be accountable for an organisation’s AI experiments?

One named person should hold accountability for the experiments. The paper asked for somebody named to be responsible for a small number of them, and for a written record of what happened, kept where the next person could find it. A named owner sits alongside a baseline and a written stop condition, fixing in advance who calls a halt and on what evidence. That arrangement needed no new budget line.

What happened when organisations banned staff from using generative AI?

Use continued out of sight. The paper’s position was that an organisation prohibiting use in early 2023 would find that employees carried on and that it had lost its ability to see what was happening. Staff in most large organisations had begun within weeks of the preview opening, without being asked to and without being told they could not. A ban removed the evidence.

Paul Forrest is a fractional Chief AI Officer and AI strategist working with UK boards, private equity portfolio companies and growing businesses. Read how the fractional CAIO engagement works, or see the full range of Ecaveo AI services.