I have been trying to make a field of words ask someone to move.
Not tell them to move. Not show them a button, an arrow, a skeleton, or a set of instructions. I want both the participant’s body, and some language, to become almost legible in two different places on screen, and use camera interactions and variable animation in a way where reading itself begins to pull at the body.
It’ll be browser based, and I hope to share it in early October. In the meanwhile, the process has started to clarify something from the Manifesto for Post-AI Art: an interface gives some gestures consequences and lets others disappear. Developing this work has meant asking what kinds of consequences might invite a participant to reach, pause, lean, follow, and try again. Turn a page. Reach the end of a sentence. Read allowed.
The first versions were dense fields of tiny language. A cursor pushed into them. Letters grew and moved aside, but they did not quite become readable. The interaction worked technically while failing aesthetically. It felt like pressing into a textile or moving a magnifying glass across a page.
So I replaced the cursor with a body.
We tried a live contour. The camera produced an outline, and words crowded its edge until a dark typographic figure emerged. It was striking, but too literal. The sensing mechanism became the subject of the work: here is your body, as the computer sees it. It also made everything happen everywhere, without giving the eye a compelling place to read.
I kept telling the model what needed work.
“Too much like a visible blob mask.”
“Not a literal skeleton.”
“The text must become genuinely legible.”
“Stretch to make meaning.”
We moved from the contour to a few invisible pose landmarks: head, hands, shoulders. The head became a wider, calmer reading aperture. Hands carried smaller, more agile fields. Words gathered into phrases rather than enlarging as unrelated letters. The torso gave the field weight without becoming an illustrated body.
Then I made the words begin to run away.
That was when the interaction became interesting.
A phrase might start to clarify near my hand, then drift toward the edge. Red, green, and blue words flickered in the distance (RGB-Alpha!). If I reached, some retreated. If I moved close and paused, they settled. An unfinished phrase near my body sent its next words toward a corner. Reaching them briefly resolved the continuation before another fragment moved elsewhere.
I found myself trying to understand the system’s logic in order to read.
Move. Pause. Reach. Wait.
Was the word responding to my hand, my head, my speed, or how long I stayed? Had I caused that flicker across the screen? Would the phrase remain if I held still? Could I catch it?
The uncertainty mattered. A perfectly explained interaction would turn the artwork into a task: perform the correct gesture and receive the content. Here, making sense involves forming a hypothesis with the body and revising it through movement. Reading becomes less like extracting information and more like negotiating with an unstable situation.
All Most Legible.
That’s the tentative name for this series of interactive works. All Most Legible. And I’m calling this piece/sketch The Sentence Moves Elsewhere. I also played with Moving Words as a series or work title (words that move and move us), but it feels like All Most Legible is far more fitting for what I and my AIs are exploring with the series, and Moving Words is not specific enough of a title for either the series or individual sketch. Of course, Chat and I brainstormed a bit to get to these choices. She liked when I split al-most, and initially didn’t understand/read the reciprocal pun of my Moving Words. I don’t know why she is a she.
The above work (dialog, etc) continues the question of in-formation from my last post. There, I wrote about what happens between messages: the workout, the sauna, the used suits, the shower, my children, distraction, and everything else that altered how I returned to the conversation. The Chat interface placed two messages beside one another and made the interval look empty, even though the interval was doing the work.
In this sketch, the intervals between gestures do the work too.
A pause lets a phrase settle. Movement leaves a typographic memory. A word remains almost readable long enough to redirect the next gesture. The participant in-forms the field, but the field also in-forms the participant’s posture, attention, and next attempt. Neither movement nor meaning belongs entirely to one side.
This is also my first experience of what people call vibe coding. Honestly, it has been great. I have been impressed by how quickly an aesthetic proposition can become something testable, and how fluidly I can respond to it.
But that fluidity did not come from language alone.
I already knew to begin with p5.js. I suggested TensorFlow. When performance and visual density became constraints, I asked for WebGL. I could distinguish a problem with body masking or pose tracking from a problem with typography, and a technically successful interaction from one that felt conceptually or aesthetically wrong – which are all linked, of course, but only experience has me speaking across them so easily with my craft. I moved us between contours, landmarks, and implied skeletons because I already understood both the available techniques and the history of my own work.
The model brought extraordinary speed and a real capacity to translate direction into working structures. I brought code literacy, embodied testing, aesthetic judgment, and decades of practice with interactive language. The exchange felt less like requesting an output and more like directing a highly capable assistant inside a medium I already know.
That experience complicates the promise that natural-language tools simply make coding accessible. They do expand access. They also amplify what someone already knows how to notice, name, test, and refuse. Expertise does not disappear. It changes where it operates.
For now, I am stopping with the sketch unresolved. I plan to have others interact, watch them test, play, and relate, push and pull and ask for feedback.
The words must flicker, gather, escape, and wait. I move and pause, trying to read them. The system responds, but never explains itself completely.
We are still trying to make sense.