Weekend Reflections #19 | The Missing Shape

[Views are my own]

I was looking at some of the new AI agents and devices appearing lately.

And something started bothering me.

They look like toys.

Not all of them, obviously. But enough that it feels like more than a coincidence. Small. Soft. Cute. Sometimes with eyes. Sometimes almost like something you would give to a child.

Except there is an important distinction.

Some of them are not physical objects at all. They are characters on a screen. Representations of an AI rather than devices containing one. Others actually are physical products.

But the design language is surprisingly similar.

Which made me imagine a product conversation.

"People will never carry something that watches them every second."

"But they already carry a smartphone."

"Yes, but a smartphone is normal. You don't look at it and think about the sensors, microphones and data collection. People had years to get used to that. We don't have years."

"Fair. So you think carrying an AI with you all day would make people uncomfortable?"

"Maybe."

"Then let's make it look like something people are already comfortable carrying."

"A pendant?"

"Doesn't work for everyone."

"A small metal box?"

"Too technological."

"What do you mean?"

"You look at it and you remember what it is."

"So... a toy?"

"Hmm."

I obviously don't know whether any conversation like this actually happened. Probably not.

But I keep coming back to that sentence.

You look at it and you remember what it is.

We have spent years talking about dark patterns in software. The button that makes cancelling harder than subscribing. The cookie banner that makes "accept" obvious and "reject" work. The interface that changes how a decision feels without technically removing the decision.

But perhaps the interface does not have to be a button or a screen.

It can also be the way the intelligence is represented. And, when that representation becomes hardware, the object itself.

An AI device can listen. It can remember. It can know where you are, who you speak to, what you ask, perhaps eventually what you see. And increasingly, it can also act.

That is a lot of agency for something presented as a small, friendly character. Or sitting in your pocket or on your desk.

Maybe character design and industrial design have a role here that I had not really thought about before. Because representation also tells us what something is.

A camera looks like a camera. A microphone looks like a microphone. A computer looks like technology.

A toy says something very different.

Harmless. Playful. Not serious. Something you don't need to worry about.

But there may be another explanation for why this is happening.

For a while, AI was mostly being added to objects we already understood. Glasses became AI glasses. Watches became AI watches. Cars and homes became more intelligent too.

The pattern made sense. Take an existing object, give it intelligence, and make it better at the thing it was already there to do.

The object still explained the product.

Glasses see. A watch knows something about your body and your day. A car understands where you are going. There is a natural boundary around each of these things.

But general-purpose AI systems, especially agentic ones, don't really have those boundaries. Their promise is almost the opposite. They are supposed to understand anything, help with anything, remember anything, and eventually perhaps do anything they are given access to do.

And that creates a strange design problem.

What is the form of something whose function is not specific?

You can put AI into glasses, but then it is still, at some level, an AI that lives in your glasses. You can put it into a watch, but the watch still comes with a particular place on your body, a particular interaction model and a particular set of expectations.

A general-purpose AI does not have an obvious form.

But what should something that listens, remembers, answers, observes and potentially acts actually look like?

There is no established object for that.

And maybe that is why the distinction between digital character and physical device matters less than I first thought.

Before we decide what AI should physically look like, we already have to decide how it should appear to us at all.

Maybe this is why some of these agents and devices seem to be moving away from a functional identity and towards a social one.

A face. Eyes. A name. A personality.

Sometimes that identity stays on the screen. Sometimes it gets a body.

Something that feels less like equipment and more like a character.

Perhaps we are not turning toys into AI.

Perhaps we are giving AI the language of toys because we do not yet have another familiar metaphor for a general-purpose intelligence.

I think that makes the original question more interesting, not less.

When AI lives inside glasses, the glasses constrain how we think about it. When it lives inside a watch, the watch constrains it. The object gives the intelligence a boundary.

The companion gives it a different kind of boundary. A social one.

It can potentially be anything.

And the more general the system becomes, the less its representation tells us what it can actually do.

Which brings me back to dark patterns.

Maybe manipulation is no longer only in choice architecture. Maybe it can also exist in representation.

Not making you click "yes".

Making the thing you are saying yes to feel less consequential.

I don't know if I would call every cute AI character or device a dark pattern. That would be too easy. And there is nothing inherently wrong with making technology approachable, familiar or even like-able.

But I think there is an interesting gap worth watching.

The gap between what a system can actually do and what its representation makes us feel it can do.

AI systems are becoming more capable, more autonomous and more present in our lives.

At the same time, some of the identities we give them are becoming smaller, softer and more social. Sometimes only visually. Sometimes physically too.

Maybe that is simply good design.

Maybe it is what happens when we have to give form to something that has no obvious form of its own.

Or maybe those two things create a problem precisely when they work.

The more the machine can do, the less its form tells us what it does.

We cannot show you what the machine does.

So we show you who it is.