Guardrails Before Scale: an enterprise governance blueprint for generative AI

Guardrails are a working system of decisions, evidence and review cycles, and an organisation that produces only a policy document has documented an intention.

Nothing in this article constitutes legal advice. It is offered to support discussion and further review with qualified advisers.

Did the executive order of late 2023 create obligations for companies deploying AI assistants?

An ordinary enterprise deploying an assistant acquired no obligation from it, on my reading of the text as it stood. Guardrails Before Scale was written on 30 November 2023 and records the position at that date, so everything below is a snapshot of late 2023. Three instruments arrived inside three days. The executive order on safe, secure and trustworthy artificial intelligence was issued on 30 October 2023 and published in the Federal Register on 1 November. On the same day the Group of Seven published a leaders’ statement, international guiding principles and a code of conduct for organisations developing advanced systems, and on 1 November, 28 countries and the European Union signed a declaration on frontier risk. The order itself was an instruction to federal agencies, and its principal reach into the private sector was a reporting duty on developers of the largest dual-use models and on operators of large computing clusters. All three instruments nonetheless changed the environment in which an enterprise would be asked to explain itself.

• The executive order directed federal agencies, requiring guidelines for red-team testing from the standards institute, a report on content authentication and synthetic content labelling, an interagency council, and budget office guidance to agencies on their own use.
• The Group of Seven principles and code of conduct published on 30 October 2023 were voluntary and addressed to organisations developing the most advanced systems, which placed a deploying enterprise outside their stated scope.
• Five risks most commonly defeated controls designed for deterministic systems, namely probabilistic output, prompt injection through untrusted content, leakage of material the user never meant to disclose, a supplier changing a model without notice, and human overreliance on fluent text.
• Four tiers, running from prohibited uses through high-consequence and controlled productivity to low-risk experimental, were set by the consequence of a wrong output and by how far it could be reversed.
• Supplier assurance covering change notice, testing access, logging, data terms and an exit plan was negotiable before a deployment became embedded.

What does it mean to treat guardrails as an operating system?

Guardrails, in the paper’s definition, are decisions with evidence attached, renewed on a cycle. An acceptable use policy is one artefact of that system, and my judgement then, which I still hold, was that producing only the artefact documents an intention. The distinction matters because a board asked how a system was tested, who approved it and when the decision was last reviewed will reach for evidence, and a policy text holds none of that.

The risks sat outside the conventional software category, which is why the paper treated lifecycle controls as continuing work. Probabilistic output, prompt injection, quiet leakage, an unannounced model change and human overreliance on fluent prose all behave differently from a defect in deterministic code, and a control that tests a fixed input against a fixed output has little purchase on any of them. The difficulty appears to have persisted.

How should systems be tiered when the technology keeps moving?

Tier by consequence and reversibility, meaning by what happens when the output is wrong and how easily the error can be undone. The paper set out four tiers and argued for keeping the number small, since a scheme with a dozen categories produces argument about classification and very little else. Tiering by technology ages badly, because the model sitting behind a given use changes far more often than the consequence of getting that use wrong. Few tiers also keep the scheme usable.

What goes wrong when governance is treated as the first step of four?

An organisation that works through govern as step one completes a step and leaves the result unowned. In the framework most organisations were adopting at the time, three of the four functions describe work done on a system, and the fourth describes the conditions under which that work happens at all. Govern is therefore a condition holding across the other three, which is my own reading of the framework’s structure.

Decision rights are where the same point becomes practical. The paper gave a section to them, including one decision that in my experience commonly sits unassigned, on the argument that governance becomes observable at the moment a named person holds each call. An unowned control generates no record, and a control with no record looks, to anyone reviewing it later, like one that was never operated.

What could be agreed with a supplier while the terms were still open?

Change notice, testing access, logging, data terms and an exit plan were all negotiable at the point of purchase, and my position was that the bargaining power sits there. Once a deployment is embedded in a workflow, the cost of moving falls on the buyer. The supplier knows it. Of the five, change notice probably carried the most weight, because a model updated without warning can alter the behaviour of a system that passed its tests a month earlier.

The final section of the paper set out a 90-day launch for the same reason. Getting a small number of decisions recorded, owned and reviewable inside a quarter gives an organisation something it can show, while the deployment is still small enough to change. A longer programme tends to arrive after the systems it was meant to govern.

Read now, the paper is a record of a moment when the policy environment moved quickly and the obligations on ordinary deployers moved hardly at all. The environment has moved a good deal since, and anyone picking up the blueprint today should check every instrument it cites against its current form. What has held up, on my reading, is the organising claim. An enterprise asked to explain a deployment is asked for decisions, owners, tests and dates, and those come out of a habit that runs on a cycle. That was true before the three instruments appeared in 2023, and I suspect it will outlast the texts the paper was written to interpret.

Frequently asked questions

Were the G7 AI principles binding on companies using AI tools?

The Group of Seven principles and code of conduct, published on 30 October 2023, were voluntary and addressed to organisations developing the most advanced systems. An enterprise deploying an assistant sat outside their stated scope, which matters a great deal before anyone adopts them as an internal compliance baseline. Reading the scope of each instrument first was, on my view then, the opening governance task.

What is prompt injection, and why do ordinary software controls struggle with it?

Prompt injection reaches a system through untrusted content that the system reads and then acts on. It was one of five risks identified in late 2023 as defeating controls built for deterministic software, alongside probabilistic output, leakage of material a user never meant to disclose, a supplier changing a model without notice, and human overreliance on fluent text. A test matching one fixed input to one fixed output has little purchase on any of them.

What did the countries that signed the declaration on 1 November 2023 agree to?

On 1 November 2023, 28 countries and the European Union signed a political declaration on frontier risk. It was the third instrument in three days, after a United States executive order and the Group of Seven material. A declaration of that kind placed no obligation on an ordinary enterprise deploying an assistant, on my reading at the time. Its effect fell on the environment in which such an enterprise would be asked to explain itself.

What should an organisation do in the first 90 days of an AI governance programme?

Get a small number of decisions recorded, owned and reviewable inside the quarter. The 2023 blueprint closed with a 90-day launch, on the argument that an organisation then holds something it can show while the deployment is still small enough to change. A longer programme tends to arrive after the systems it was meant to govern. That reasoning still stands as a method, whatever the regulatory position now is.

What will a board ask for when it reviews an AI deployment?

A board asks for decisions, owners, tests and dates. Asking how a system was tested, who approved it and when the decision was last reviewed is a request for evidence, and a policy document carries none of it. My position in 2023, which I still hold, was that an unowned control generates no record, and a control with no record looks to a later reviewer like one that was never operated.

Paul Forrest is a fractional Chief AI Officer and AI strategist working with UK boards, private equity portfolio companies and growing businesses. Read how the fractional CAIO engagement works, or see the full range of Ecaveo AI services.