Better than the last one

limitations of vibecoding vs the need for something meaningful for a birthday video

I had 167 photographs of my family and a birthday coming up for my mom, and I wanted to show them off in a grand way. The first idea was to take all 167 and rearrange the pixels of each of them into a single photograph of her. I had vibe-coded something like it before, and I had one rule for the whole project: nothing on screen could be generated by AI.

Getting that first build to work was mostly about how it looked. If the pixels move too fast you can’t see them travel, and then it doesn’t read as a transformation at all. And the edge of the face in the final photograph has to be preserved, or the face is unintelligible. Both of those are 2D or 3D transformations I couldn’t have written. But I could tell they were wrong by looking at what came out, and fix them by asking for the specific thing I wanted instead. That is why this one worked: I had a vision for the end. It was a success.

Then I made three more, and the only spec any of them carried was better than the last one. No direction.

With no end to aim at, the gaps got filled in by the model. It gave me a Droste portal, a depth dive, and a set of high-dimensional solids with photographs mounted on their faces, and it tuned each of them however it felt like. None of it satisfied me, and I couldn’t say why, because there was no end goal to compare it to.

The depth dive is the one place I could still see what was missing. It pushes a camera into a still photograph so the room gains volume, and the parallax reveals the pixels behind people — pixels that do not exist. That is missing information a photo does not carry. A technique called quantile depth bands put boundaries through bodies, so a person landed on two layers at two scales and tore. I switched to double-Otsu, which I can’t explain properly — only that it split each photo into three layers, and the tearing stopped. Neither version fixes the hole behind people. Filling it needs an inpainting model or a generative dolly, which is the rule I’d set, so I retired the dive. The missing thing was literally missing, which is the only reason I caught it.

The other two I couldn’t argue with. I didn’t know what a Droste effect was when the AI decided on it, so yes-or-no was the only judgement I could make, based on feeling. It is supposed to be a conformal twist, but it requires the whole plane to be self-similar and a photo is self-similar nowhere; applied anyway, the photo smears and is unintelligible. The other idea, multi-dimensional solids, were striking and unreadable — foreshortened faces, half of them pointing away, nothing holding still long enough to be looked at. Both are known properties of the construction, and I found them one render at a time instead of in an afternoon of reading.

I could tell a build was wrong by looking at it, and I could explain what was wrong about it visually. But, I could not tell whether the code or the math underneath was wrong, because I don’t have that knowledge. Those are two different kinds of wrong and I only had access to one of them.

What worked is a fifth build: traversal of the co-occurrence graph. I don’t count it part of the experiment, as it started directionless, but I think it supports my point. It started off as just a graph of faces, but then I connected them by which photos they appear in together, a co-occurrence graph. I decided the video had to traverse this graph from person to person, starting and ending at my mom. A full circle showing all our family members centering around her. It only came together once that direction finally arrived.

Fifty-three seconds of the finished piece, looping, and rendered entirely from stand-ins: each white rectangle is one photograph, carrying a square head per face at the position and size that face actually occupies in it. Colour is identity — the five most-photographed people, everyone else grey. The first forty-one seconds are the traversal, where every hop crosses one edge of the co-occurrence graph and moves the way that edge actually runs, so the neighbours drifting past are the real ones, and the person about to be travelled through lights up before each departure. The last twelve are the closing act, a wide view of the entire graph. This is the same arrangement as the real graph, with the photographs replaced by stand-ins.

I see the first build as a success and the next three as a disappointment, and the three that disappointed me took much longer than the one that didn’t. All four were vibecoded, and I knew nothing about image transformations going into any of them. So the one thing that changed between them was whether I had a direction, and the results split along it.

But there’s still something on my mind. I could always see what was wrong, but never where. Vibecoding trades control for convenience, and that matters more when you care deeply about the outcome. A birthday video for my mom had a specific requirement: it had to be meaningful and emotional. But you can’t do that without a certain amount of control. Knowing the domain well enough to make the decisions is that control. What I still don’t know is what that actually looks like in practice. I’ve handcoded with a model advising, and I’ve vibecoded outright. I haven’t yet worked in the place where only the direction stays mine — the closest I came here drifted off the moment the problems went past what I understood.

I should say plainly that this is thinner than I first made it sound. Direction beats no direction is not a discovery. And the four builds were four different tasks, so I can’t really attribute the split to direction — a Droste portal and a pixel sort are not the same job attempted twice under different conditions. What I have is one project where the difference happened to be legible to me, not an experiment.

One more thing, since it’s the thing a reader notices first. I wrote the original version of this note as a paper, because I’m hoping to get into research-driven work this fall and thought a small project would be good practice at the form. I used AI to condense it and push it toward that tone, and so it read like a model wrote it, which is fair. It also skipped the setup — no introduction, no terms established and carried through, and too many different words for the same thing. This version puts the story back. I’m leaving the note up on my LessWrong account, to look at again once my writing is better.

Code: photo-constellation. The traversal runs on any folder of photographs; the superseded rounds are there to read rather than to run. No photographs, no embeddings, no derived data.