Snappish Productions

One thing I have come to realize over the years is that I am very prone to falling for gadgets. This means I have a graveyard of electronics that has followed me from Bicester to Manchester, across the Atlantic to Chapel Hill and back again…and then back again via shipping container and moving van to Cincinnati. There’s all sorts of weird and wonderful things; animal toys that click together and play different tones depending on the arrangement, my Sony Cyber-Shot DSC-U10 (still so cute!), the period of time where I’d take a Handspring Visor on the bus to Oxford every day before I gave in and finally bought my first real mobile phone (and that iPhone is probably still around somewhere too).

There’s all sorts of Bluetooth things, and then a spurt of things from 2022 when Tammy and I were throwing around ideas for another escape room, this time having one room across three different time zones. We really went all out on this; I have rolls of NFC sticker tags hanging about, and I had built a working Pepper’s Ghost effect that would have been used, plus I had already had the idea to pull in (then quite early) models capable of voice cloning and facial recognition. We probably should have written more of it down…maybe when Maeryn is a little older, we can run a version of it…

Then there’s the more recent things. The RFID/NFC battery-less e-ink photoframe (which promises a lot more than it can deliver, but some photos come out really nicely on it), the Xteink mini e-readers, and then the influx of ESP32 stick devices. These were Matt Webb’s fault. An absolutely tiny form-factor (smaller than a pack of chewing gum!) and yet it has 8Mb of memory. Anyway, I now have 5 of these because…of reasons. Finally, the most recent arrival is the Cardputer ADV, which is similar to one of those sticks but it has a “full” keyboard and looks so 90s-hacker-coded that I feel like I’m in Sneakers every time I look at it.¹ Easily worth $30.

And then there’s Maeryn’s toys. When she got the Toniebox a couple of years ago, I did have a poke around at the state of reverse engineering, although sadly I came to the conclusion that it probably made sense just to play around with Creative Tonies instead of trying to flash (and likely brick) the Toniebox itself.

There’s also a competitor to Toniebox called Yoto, and one of the things that Yoto has going for it is that their Yoto Mini system is considerably smaller than a Toniebox², and while it doesn’t have the cute range of figurines, the cards are so much easier to pack for travel. Tammy found a great deal on the Mini recently, and it arrived this weekend. I hadn’t really paid that much attention to the system, but when she took it out of the box, I was struck by how fun and little it was, with a simple 16x16 LCD panel that can be used by stories for little icons as stories are playing.

Anyway, more out of idle curiosity than anything else, I searched to see if there was a similar reverse engineering effort. And instead I landed on the developer API page. I’m fairly sure my eyes widened as I saw API contracts, MQTT events detailed, and quickly found a Python wrapper. You can even point to content that isn’t on their servers…and you can alter that content and get button push information from the box itself over the internet. Five minutes later, I had the idea of “Daddy’s London Travelogue”, where I can send an update (with fun icons!) every day I’m going to be in the UK in September, and Maeryn can put in a card to get the latest update every evening.

Naturally, before 13:00 on Saturday, Claude and I had this working. But there’s even more. The way that the Yoto system works…it doesn’t really care if you swap files about. Even if you do it dynamically. You could use the two button presses from the device to make a simple game work. Which in itself would be fun, but there’s nothing stopping you from using a separate controller system (e.g. a tablet or phone or dedicated ESP32 device…) and using that to control the webapp that you host to alter what gets sent to the Yoto Mini.

It was at that point I had to physically separate myself from the computer, as I was already plotting something akin to an aural version of Knightmare³, and it would have sucked up the whole rest of my day. And funnily enough, a good part of the day I should be actually spending with Maeryn…

…but there’s always nights…

I built another thing. I don’t think it’s useful, not really, but I still built it. If, by some chance, you had always wanted a online search engine dedicated to Visions of Heaven and Hell, a 1994 documentary broadcast on Channel 4, then…congratulations? And we should probably say hello at some point?

The search engine in question.

Most of this was two afternoons, and the second was spent fiddling with CSS and picking an embedding model that would be fine in a browser process¹.

I still can’t really tell you why I did it, either. I spent a good part of my free evenings in early July mainlining a lot of mid-90s BBC/C4 docs on the Internet and what it was going to do to us. Smiling at some bits and going “Oh, well, damn, you nailed that one.” Simultaneously re-reading The Invisibles to fully bask in that there’s-a-new-world-coming feeling². I’d never seen Visions before, somehow, and it just felt like something I wanted to cut up and process with 2026 open source technology.

If you really want to be afraid, I have over 500 editions of Horizon on my NAS and I think the world needs them to be searchable…

When I came here, a decade-and-a-half ago, I am sure I did not imagine that fifteen years later, I’d be lying on the floor in a completely different state, singing Appetite to my three year old because she wanted to hear the songs “you sang me when I was little.”

And yet, here I am, and I can’t imagine anything better. In another fifteen years, Maeryn will be eighteen, and I can’t imagine that, either, but I know it’ll be here faster than an eyeblink.

In what was otherwise a pretty terrible week, it was wonderful to see a section of my social media follow to completely lose their top over discovering that one of our long-term Holy Grails sneaked onto YouTube a few months ago. Yes, that’s right, the complete, unedited, and unabridged Statutory Right of Entry is now available for your unbridled CSO enjoyment.

(there’s even a comment on the page from one of the VT people that made it happen!)

Ever since I first read the World Models paper, there has been a nagging irresistible thought at the back of my mind: “I want to try that out on Deathchase". And, to be fair, I have over the years spent a few Sunday nights trying to make it work. The RNN or the VAE code wasn’t the problem; I have working code on my old 1080Ti PC tower that trains well enough on the example toy racing world. The problem was somehow hooking it up to a Spectrum. I tried a lot of different approaches, including some major surgery to MAME, and I got tantalizingly close, to the point of hand-disassembling Z80 code to work out where information like lives and scores were being kept but never enough to really make it work in a fashion I could run a full training loop without it keeling over and dying a few iterations in.

(you know how this is going to go)

It popped up in my head again recently, and this time I asked Claude. It laughed, saying: “Why do you not just use zx, as it’s basically built exactly for what you want to do?” because I didn’t know it existed, you smug little SkyNet! And then we set to work.

The World Model paper is actually three different models. Firstly, a variational autoencoder (VAE) takes the pixels from the screen (resized in the paper to 64x64, in my code to 84x84) and compresses them down to a latent vector. Just 64 numbers to describe everything going on in every single frame fed into the model. This is then chained into a LSTM network, which takes the latent vectors and learns to predict what comes next. These two models together form the world model; eventually, you can feed a starting frame into the autoencoder and then ‘dream’ the entire sequence of the game inside the LSTM. And this is the trick that seemed magic in 2018; it’s annoyingly hard to do reinforcement learning or any sort of gradient descent work when you have to keep going back to the game state, perhaps in an awkwardly coded emulator. There’s a lot to go wrong. Here, the controller model trains directly on the dream world and then you transfer the trained controller model back to the real world…and it works! Mostly.

%%{init:{'theme':'base','themeVariables':{'fontFamily':'ui-monospace, monospace','fontSize':'14px','lineColor':'#8b97ac','edgeLabelBackground':'#0f1420','tertiaryTextColor':'#c7d0de'},'flowchart':{'curve':'basis','nodeSpacing':46,'rankSpacing':60,'padding':10}}}%%
flowchart LR
  o["obs oₜ
64×64 frame"]
  subgraph WM["world model · runs with no environment (the dream)"]
    direction LR
    V["V · VAE encoder
compress → zₜ ∈ ℝ³²"]
    M["M · MDN-RNN
predict P(zₜ₊₁ | zₜ, aₜ, hₜ)"]
    V -->|"zₜ"| M
  end
  o --> V
  M -->|"zₜ, hₜ"| C["C · controller
1 linear layer · ~900 params"]
  C -->|"aₜ"| E(["real environment"])
  E -->|"next frame"| o
  classDef ha fill:#2a2214,stroke:#eaa63f,stroke-width:2px,color:#f3e7d4;
  classDef ctl fill:#1b2130,stroke:#cfd6e2,stroke-width:2px,color:#eef2f8;
  classDef io fill:#141a26,stroke:#6b7688,stroke-width:1.5px,color:#c7d0de;
  class V,M ha; class C ctl; class o,E io;
  style WM fill:transparent,stroke:#eaa63f,stroke-width:1px,stroke-dasharray:5 5,color:#c69a52;
      

flowchart LR
  o["obs oₜ<br/>64×64 frame"]
  subgraph WM["world model · runs with no environment (the dream)"]
    direction LR
    V["V · VAE encoder<br/>compress → zₜ ∈ ℝ³²"]
    M["M · MDN-RNN<br/>predict P(zₜ₊₁ | zₜ, aₜ, hₜ)"]
    V -->|"zₜ"| M
  end
  o --> V
  M -->|"zₜ, hₜ"| C["C · controller<br/>1 linear layer · ~900 params"]
  C -->|"aₜ"| E(["real environment"])
  E -->|"next frame"| o
  classDef ha fill:#2a2214,stroke:#eaa63f,stroke-width:2px,color:#f3e7d4;
  classDef ctl fill:#1b2130,stroke:#cfd6e2,stroke-width:2px,color:#eef2f8;
  classDef io fill:#141a26,stroke:#6b7688,stroke-width:1.5px,color:#c7d0de;
  class V,M ha; class C ctl; class o,E io;
  style WM fill:transparent,stroke:#eaa63f,stroke-dasharray:5 5,color:#c69a52;

After spending an afternoon with Opus a couple of weeks ago, the first discovery was that Ha’s world model…just didn’t work very well for Deathchase. It did learn a grand plan of surviving…which was to slowly go through the first day, shoot at a few things, and simply sit tight when night falls. Which is a strategy, but one that would have got you slapped if you tried it back in the day round your friend’s house. Also, it does have a tendency to hit the trees a lot. But then I have that problem as well.

Despite that, looking at the dream sequences of the VAE model is spooky; you can see exactly what the model is seeing and, yep, well, it’s certainly Deathchase.

Anyhow, I was not to be stopped by that little failure. Because I kept reading these sorts of papers, and I specifically remembered DreamerV3 from DeepMind. Back in 2023 it pretty much set the new state of the art for autonomously playing Atari games. So Claude, myself, and the DGX Spark worked for a day or two to train a new model based on this new approach.

The DreamerV3 framework is a lot more complicated (see the image, yikes!), but I’d say that the too key points are: a CNN model to better capture what’s going on versus the older autoencoder, and a training regime that keeps the world model anchored to reality, with joint training focused on reconstructing the frame, optimising the reward, and whether the model has left the game in a playable state. And this works a lot better.

%%{init:{'theme':'base','themeVariables':{'fontFamily':'ui-monospace, monospace','fontSize':'13.5px','lineColor':'#8b97ac','edgeLabelBackground':'#0f1420'},'flowchart':{'curve':'basis','nodeSpacing':40,'rankSpacing':58,'padding':10}}}%%
flowchart LR
  prev["hₜ₋₁ · zₜ₋₁ · aₜ₋₁"] --> GRU["sequence model
GRU"]
  GRU --> H["hₜ
deterministic memory"]
  X["obs xₜ"] --> ENC["encoder
CNN"]
  ENC --> Q["representation q
posterior zₜ | hₜ, xₜ"]
  H --> Q
  H -.-> P["dynamics p
prior ẑₜ | hₜ · no obs"]
  Q -. "KL: pull prior → posterior" .-> P
  H --> S["state sₜ
(hₜ, zₜ)"]
  Q --> S
  S --> DEC["decoder → x̂ₜ"]
  S --> REW["reward → r̂ₜ"]
  S --> CON["continue → ĉₜ"]
  classDef dr fill:#123039,stroke:#43c0d0,stroke-width:2px,color:#dff3f6;
  classDef prior fill:#123039,stroke:#43c0d0,stroke-width:2px,stroke-dasharray:5 4,color:#dff3f6;
  classDef neut fill:#1b2130,stroke:#8b97ac,stroke-width:1.5px,color:#e9edf6;
  class GRU,H,Q,S dr; class P prior; class prev,X,ENC,DEC,REW,CON neut;
      

flowchart LR
  prev["hₜ₋₁ · zₜ₋₁ · aₜ₋₁"] --> GRU["sequence model<br/>GRU"]
  GRU --> H["hₜ<br/>deterministic memory"]
  X["obs xₜ"] --> ENC["encoder<br/>CNN"]
  ENC --> Q["representation q<br/>posterior zₜ | hₜ, xₜ"]
  H --> Q
  H -.-> P["dynamics p<br/>prior ẑₜ | hₜ · no obs"]
  Q -. "KL: pull prior → posterior" .-> P
  H --> S["state sₜ<br/>(hₜ, zₜ)"]
  Q --> S
  S --> DEC["decoder → x̂ₜ"]
  S --> REW["reward → r̂ₜ"]
  S --> CON["continue → ĉₜ"]
  classDef dr fill:#123039,stroke:#43c0d0,stroke-width:2px,color:#dff3f6;
  classDef prior fill:#123039,stroke:#43c0d0,stroke-dasharray:5 4,color:#dff3f6;
  classDef neut fill:#1b2130,stroke:#8b97ac,stroke-width:1.5px,color:#e9edf6;
  class GRU,H,Q,S dr; class P prior; class prev,X,ENC,DEC,REW,CON neut;

This is Dreamer dreaming of trees and motorbikes, learning how to ride in just 12,000 iterations.

And this is Dreamer playing the actual game, having learnt to ride safely and shoot things with 300k steps played in the imagined worlds¹

I think we can say it’s mostly solved at this point…but if you don’t believe me, then here’s five minutes of it playing without losing a life.

Of course, now that I had Deathchase sorted, I started thinking about other games. We’ve got another post coming that stays firmly in the 16K era, but is a touch more complicated…and another classic. We’ll be going to Ashby-de-la-Zouch…

Current status:

View this post on Instagram

A post shared by Ian Pointer (@carsondial)

(80s bit is delayed a little for reasons. It’s done, but I just can’t face writing it up right now)

This episode of Horizon from 1978 is an interesting historical look at microprocessors and how they were starting to filter through into the world at large. As a piece of its time, it’s an interesting documentary all by itself, but I was intrigued/amused at the back half, which was all about how white and blue-collar jobs will be eliminated by our 6502 overlords, showing examples of a doctor creating an expert system that would replace consultants, and a building contractor who only had to tell a system how to do something once and then a robot would be able to perfectly replicate the process. Again, 1978. It’s interesting how we now seem to be in the same position, only this time…maybe it’s more real? Or are we getting ahead of ourselves once more, and the advent of Claude/etc. just means a new normal, like how coding in C or Pascal is often much faster than trying to write assembler.

I find I can do more; this weekend, I trained a new model for work, yes, but I did two other things that I could have done myself, but it would have taken weeks and weeks of effort. Firstly, I finally started setting up the smart solar bird feeder I bought during a recent sale. But it wasn’t the brand I somehow assumed it was, and the only way to communicate with it is to use an iOS or Android app, and they want an extra $20/year for sharing the account. I sat down with Claude just after 13:00 on Saturday, and by 15:00, we had mapped out the entire API, pulled out a long-lived JWT token that allows me to authenticate whenever and where-ever, and built a small web server that produces a child-friendly set of HTML cards ready to show Maeryn just what birds have visited today, and hooking up the audio to BirdNET so it’s strictly speaking better AI than what the company is offering through the app alone.

Then, for a laugh on Sunday afternoon, I pointed Claude at the source code for Grid Wars. Every so often, I remember how fun that game was and then I rediscover that the Intel builds currently crash on MacOS. I told Claude to port it to a Rust WASM runtime that could be played in the browser. And damn if it didn’t just go ahead and do it. It required a little hand-holding to get the blur and gravity mesh effects just right…but I can now just play Grid Wars by serving up just a few files on a website¹. It is very strange having all these horizons just open up in front of you. And it is something of a siren call; just one more idea that you can run before bedtime, just one more run and you’ll be fixed. I feel like I have stepped away from the edge a couple of times already, but every Friday night, the AI abyss comes around again…

(Next week? Back to the 1980s)

It might be surprising, considering I grew up in the irony-poisoned Britain of the 90s, but the last 10 years in this country has made me somewhat patriotic. But that’s patriotic in the sense that I’m now a person who reads the Federalist papers, letters from the Civil War and the biographies of LBJ, and remembering that day in 2009 on the Mall. Patriotism in the sense of I’ll be damned if I let these people desecrate the founding documents of this country and the nation of immigrants that gave rise to the American Century. Patriotism in the service of bending the arc of the moral universe to justice. Patriotism in living our lives as a form of resistance, a happier life than I could ever have imagined, and worth fighting for.

Five words that changed America.

Honestly, I don’t remember exactly when I joined ILX. I think it was likely in either late 2002 or early 2003, when I was living in Chapel Hill, and for some reason I vaguely believe it was due to searching for information on Saturday Looks Good To Me, which is a little odd, as they’re not exactly an ILM or ILX band. Anyway, I loved hanging out in the background¹, making a comment here and there, but mostly being a lurker as Poptimism took root there and I rediscovered my younger Smash Hits reading self (heavens above! — Ed.)

What I really wanted to do is meet up with everybody at Club Popular. But…while I had got to the point where I was relatively okay with going to concerts by myself (seriously, the hours spent awkwardly hanging about before the advent of mobile phones), I was absolutely not okay with making the trip from Bicester to London for a club night all by myself. I just couldn’t do it. Still a regret.

By 2006, I was still mostly lurking, but I had made a friend — Forest Pines, or Caitlin. And ILX had a band! Shimura Curves! ‘Ver Curves! Kate ‘Masonic Boom’, Anna, Frances, and Miss AMP! Krautrock and girl group harmonies! Zeitgeist songs like Noyfriend and Keep My Name Out of Your Blog (there isn’t another song that encapsulates the pre-Facebook Web 2.0 era as perfectly as the latter). They had an afternoon gig on a Sunday when I was already going to be going to London to see Johnny Boy. Suddenly, I was invited to a pre-gig pub meet up with Kate and Caitlin.

So I ended up drinking strawberry beer with them for several hours before we decided we’d better actually go to the Notting Hill Arts Club, Kate carrying her guitar through town. By that time, we’d also picked up Ed, another ILX regular. The concert itself is mostly lost to my memory, but I remember groupie cards kissed by individual band members, banter about Brown and Sticky, and the Berlin, 1973 ending of Stronger.

Even just all that would have made for a wonderful day. But I still had the second half. Johnny Boy at The Luminaire in Kilburn. I was, of course, back on my own, and resigned to standing sheepishly in the venue waiting for the band to come on stage. But it wasn’t quite to be. I’ll admit my memory does not give me perfect recall, but the way I remember it is this: I noticed a group of people sitting in a booth. One of whom I was damn sure was Kieron Gillen, who’d I known in an online sort of way since joining the Kenickie Mailing List back in 1997. Eventually, I worked past my shyness enough to go over and say hello. This was about two months before the first issue of Phonogram, so we talked about that, and me seeing Miss AMP earlier in the day. The idea of just randomly bumping into somebody I knew at a concert…it had just never happened to me before. And given how the day had gone so far, I was getting beyond giddy at this point.

Towards the end of the gig itself, I found myself dancing with him and Alex De Campi. The elation I had coming out of the venue and rushing to get the last train from Marylebone to Bicester North was something I hadn’t felt since that house party in Chapel Hill way back in 2003, or that time I went into the Carrboro woods to smash up a piñata for a late birthday celebration. On the journey back, I think I was plotting…or more realistically, fantasising about the potential of moving to London. Maybe every weekend could be like that, I thought, as I went past Haddenham & Thame Parkway.

Anyway, twenty years ago this week. I never did move to London; instead I took a rather different direction which sees me typing this in Cincinnati. Which I have to say has worked out very well for me, as I pause to think about Maeryn demanding I play drop dead again on the way home from daycare (Tammy: “With me, it’s Wheels On The Bus. There’s apparently a different vibe in Daddy’s car). And then this September, I’m going to be back in London again, Saint Etienne will likely play Popular, and I’ll smile, recognising all the names.

I felt a little attacked on Wednesday morning when my boss jokingly referred to me as “sometimes a little workaholic.” Later, as I was writing messages on Slack as I was literally having an IV attached…yeah okay, I guess I can sometimes see it. Vaguely.

(nothing too much to worry about - an endoscopy to check on my acid reflux. And I did a reasonable job at staying offline from work on Thursday and Friday. So there.)

Anyway, a truncated week, and I feel like I let the opportunity of two days off mostly slip by without filling them properly, although watching Eno yet again (now that a new version is out every month on the Criterion Channel) did get me to start work on The Assembly Machine, of which I will probably talk a little more about in July when it’s slightly more stable.

It is Father’s Day, and I have the greatest Father’s Day present ever — the best LC! tour t-shirt ever made, and a Lego cement mixer to go along with it, as every Lego city needs the means of production so the city can grow and grow, right? I’m fairly sure that’s correct.

Oh, and I got complimented at the playground today for a t-shirt that is, perhaps, over 25 years old?