There's always someone at the controls

Under the hood

From the outside, a language model seems smart. It talks back and understands your question. On the inside, it's a big file with billions of numbers, adjusted during training.

What does that file do? It predicts which piece of text is likely to come next. Then the piece after that, and so on. That's how a full answer takes shape.

That computation runs on GPUs, chips originally designed for computer graphics. A model can also only work with the text you give it. That's why RAG exists: you first hand the model the right passages from your own documents, so it doesn't have to guess.

For me, this was the start of demystification. AI became technology. And technology is something you can understand, measure and improve.

In practice

An AI platform needs its own hardware, and that's hard to get. We're talking about GPUs like Nvidia's H100 and the newer H200. They're scarce and expensive, and the big suppliers have long waiting times. Getting your own rack space in a data centre isn't easy either.

How you connect cloud and AI may be the most important question right now. Almost everything you do with AI touches data. You don't want to send that data outside lightly. Yet most computing power sits outside your walls. You have to make a choice there, and it's rarely simple.

A pleasant surprise: many tasks work fine with a small model. Think of summarising a text, classifying a document or pulling a field from a form. A small model is fast and cheap and runs on modest hardware. A big model then mostly costs more money.

I also learned something about trust. A language model always sounds sure of itself, even when it's wrong. At scale, small mistakes quickly become big ones.

Anthropomorphism

We all say it. "AI decides." "AI thinks that." "The model panicked." I've caught myself doing it too. It happens naturally, because the thing talks back. There's a word for it: anthropomorphism. We give human traits to something that isn't human.

We've done this for a long time. We name storms and swear at the printer. With a language model the temptation is stronger, because it speaks our language. "I understand you" comes out of the same calculation as any other sentence. The feeling of being understood arises in us, the reader.

The philosopher Daniel Dennett called this the intentional stance. We act as if something has desires and beliefs, because that makes its behaviour easier to predict. That's useful. It becomes a problem when we forget we're pretending.

This debate is very much alive in the Netherlands right now. In September, a group of Dutch AI scientists and journalists, including Felienne Hermans, Antal van den Bosch and Iris van Rooij, published an open letter in the Dutch weekly Vrij Nederland (in Dutch; you can also read the full letter). It was aimed at Alexander Klöpping, a tech journalist and regular guest on Dutch talk shows. They're tired of having to repair the damage after each of his appearances. One of their objections: his stories are full of anthropomorphisms. Like an AI system that panics. According to the authors, that's impossible, because panic doesn't follow from computation. The Dutch linguist Marc van Oostendorp responded (in Dutch) by asking whether a bot might be able to panic after all. A nice question for philosophers.

In my work, another danger weighs more heavily. As soon as you treat AI as a person, the real people disappear from view. People chose the data the model was trained on. People wrote the instructions that shape how it responds. People drew the lines of what it may and may not say. People decided where it runs, who has access and what happens to your questions.

If you see AI as a person, you stop seeing those choices. A mistake then looks like something the machine did. That's how responsibility gets lost. It always lies with an organisation, a team or a person.

The news about 'escaped' OpenAI agents is a good example. In July, an agent broke into Hugging Face. In September, another model found its way onto the internet. OpenAI then paused training for the second time.

The word 'escape' brings to mind an animal breaking out of its cage. A textbook case of anthropomorphism. Read the reports and you mostly see human work. People had deliberately lowered the model's safety limits to measure what it could do. People had set up the network rules wrongly. An automatic emergency stop failed. Only after two and a half hours did someone stop the training by hand. OpenAI itself admitted there was a gap in its network controls.

That doesn't make it any less serious. A nuclear power plant can also get out of control, with disaster as a result. Yet under every nuclear disaster lie human decisions and mistakes. A design, a budget cut, a test that should never have gone ahead. AI is no different. People build it, people switch it on, and people are responsible when it goes wrong.

For me as a Product Owner, this was the most important lesson. The model is a tool. The choices around it are ours.

What sovereignty really is

Sovereignty sounds like a postcode. As long as the servers are in Europe, you're fine. At first I thought so too. That turned out to be far too simple.

Sovereignty has layers. Each layer comes with its own question:

1. Where is the data stored?
2. Which law applies? Data in a European data centre owned by an American company also falls under American law.
3. Who runs the system? Who has access, and from which country?
4. Where does the model come from? What do you know about the data it was trained on? Every model carries the bias of that data. And the maker's choices about what it may and may not say.
5. Is it open source? That applies to every component: the model, the operating system, the database and the software that runs everything. Open software is something you can run, inspect and change yourself. With closed software, you depend on the maker.
6. Where does the hardware come from? And can you still get it tomorrow?

You can score well on one layer and badly on another. A European model on American GPUs. Your own servers running software you can't change. Each layer has a price, in money, time or convenience.

The European Commission is trying to make this measurable. Its Cloud Sovereignty Framework assesses cloud services on eight points, such as law, operations, supply chain and security. The outcome is a level on a five-point scale: the SEAL levels. SEAL-0 means no sovereignty at all. SEAL-4 means the whole chain is European, from chip to software. For its own procurement, the Commission requires at least SEAL-2, sovereignty over data. Most of the providers it selected in April came out at SEAL-3.

Full sovereignty hardly exists. I found that disappointing at first. Now I see it as an honest starting point. You always depend on someone. So know on whom, and make that choice deliberately.

What I take away

Do you work with AI, or do you have to make decisions about it? These are the questions I now always ask:

1. Who's at the controls? Who chose the model, wrote the instructions and set the limits?
2. Is this the smallest model that does the job well?
3. Where does the data go, and which law applies there?
4. Who do we depend on, per layer? And are we choosing that deliberately?
5. Which components are open source? And what do we know about the bias in the model?

Every AI system is chosen, set up and switched on by people. They decide what it may do, where it runs and who has access. As a Product Owner, I'm one of those people. There's always someone at the controls.