Paper three of six in the Ecaveo Working Papers on the higher-order issues in AI adoption, written in the month that embedded assistants were announced across mainstream office software.
What does an organisation actually adopt?
Never a model. It adopts a configuration of seven elements, of which the model is one, and because the other six vary across a business, the same model can succeed in one setting and fail in the next with nothing about the model changing at all.
The paper made the point through an invented case. A distribution business licenses one general-purpose language model for two teams. Customer service staff reach it through a browser tab beside the order system, at quiet desks, pasting in emails and correcting drafts, and within weeks routine replies take noticeably less time. Warehouse staff get the same model through the same interface on shared handheld scanners with small screens. They wear gloves, the floor is loud, questions arise with both hands full, and supervisors quietly discourage standing still to type. Usage falls away after the first week, and the project board concludes that warehouse staff are not ready for AI.
They were never offered the same thing. One team received a capable tool through a good route into suitable work with permission to use it. The other received the same capability through a route that made it unusable.
What are the seven elements of a configuration?
The model groups into four clusters, which is the form most useful to a pilot team.
- The tool: capability, meaning what the model can do and how reliably; and interface and modality, meaning typed text, speech or image.
- The route: channel and device, meaning browser tab, office application, phone or handheld scanner; and physical setting, meaning noise, light, movement, whether hands are free and who can overhear.
- The work: user capability, meaning knowledge of the task, of the tool, and of when to distrust it; and the task itself, meaning the speed required and the cost of an error.
- The institution: organisational legitimacy, meaning whether use is permitted, expected, rewarded or quietly frowned upon.
Capability is the only element that model comparisons measure. A capable model delivered through a poor channel is experienced as a poor model, and the evaluation that produced the purchase says nothing about it.
Why do acceptance models miss this?
The acceptance tradition has measured technology adoption for 30 years, and its consolidated form accounts for a substantial share of the variance in people’s intention to use a system. Its blind spot is structural. The material circumstances of use are compressed into a single construct measured as a belief, so the model records that warehouse staff did not feel supported without capturing that the screen was too small and their hands were full.
A practice lens does better. Technologies become different technologies in use, routines and tools shape each other over time, and organisations put boundaries around what a system is allowed to decide. The paper also borrowed from multiple resource theory to argue that modality is a design decision rather than an implementation detail. A picker already using eyes and hands is being asked for more of both by a text interface, and for neither by a spoken one.
Exposure studies of the period were widely read as adoption forecasts. They estimated how many tasks could be affected, which is a different question from where, through what device and with whose permission the work actually happens.
How should a pilot be run instead?
Six steps. Describe the baseline by recording all seven elements before anything changes. Change one element at a time. Run the pilot where the work actually happens, at normal pace, rather than in a quiet room. Keep a configuration log covering devices, settings, model versions, dates and any change made mid-pilot. Read the results by configuration rather than by average. Adopt a named configuration, give it an owner, and set boundaries and checks around it.
Pilot results should carry their configuration the way laboratory results carry their conditions. A finding reported without one cannot be transferred, and most of the disappointment in enterprise AI comes from transferring it anyway.
What should a board ask before approving an AI adoption?
Four questions. Which configuration are we adopting, element by element? Where was it tested, by whom, and in what setting? Which elements differ between the pilot and the rollout? Who owns the envelope around it, and how will they learn when the vendor changes the model or the interface?
The fourth has become the most pressing of the four since the paper was written, and not in a way the author welcomed. Embedded assistants arrived inside software that organisations already owned, switched on through routine updates. A board may find that it has adopted a configuration without ever deciding to, and the paper’s warning that this would happen has become the ordinary condition of enterprise software rather than the edge case it described.
Two of its evidence gaps are worth repeating, because they remain open. No published study varied the channel or the physical setting of a generative AI tool while holding the model constant. Frontline deployment evidence is still thinner than desk-work evidence, which means the warehouse in the invented case is still, three years later, the part of the workforce about which least is known.
Frequently asked questions
Is a model evaluation the same as an adoption decision?
No, and treating them as the same is the error the paper is about. A model evaluation measures one element out of seven. An adoption decision commits to all seven, and the other six are where most failures happen.
Why does the physical setting matter so much?
Because it determines which of a person’s resources are already occupied. A worker using eyes and hands is poorly served by a tool that asks for more eyes and hands. For frontline work, the difference between hands-free and hands-busy will usually tell you more than the difference between two models.
What belongs in a configuration log?
Devices, settings, prompts, model versions, data sources, who used it, in what setting, and every change made during the pilot with its date. It should be detailed enough that someone reading the result a year later can tell what was actually tested.
Does an embedded assistant still need an adoption decision?
Yes. A vendor switching on a feature is a change of configuration whoever initiated it. Somebody should describe the resulting configuration and decide to accept, restrict or delay it, and record that decision.
Has the seven-element model been validated?
No, and the paper says so. It is a synthesis of published research and observation, offered for pilot teams to use and to correct. The author expected some of the seven to merge or split once people started recording them.
Free download
Get the full paper
Read The Unit of Adoption in full, with every claim carrying an evidence grade and the full reference list. Give your name and email and the PDF opens straight away.
