0:00
/

Slow Takes Ep. 25: Four Numbers That Shrank When Somebody Checked

Two datacentres bringing their own gas plants, a sleep ring accused of a coin flip, a Grok flaw nobody wrote down, AI for Cocomelon, and a mammogram that may also be a heart scan.

Every Monday, Leor from Exploring ChatGPT and I go through the week’s AI news without the hype. Catch the episode live on Substack, on YouTube, or as a podcast wherever you get yours, so you can pick the format you enjoy. Use this for the facts, the links and a little extra context.


If you know someone who would benefit from more AI news and less BS, please share this with them.

Share


1. Two UK datacentres are bringing their own gas power stations

Wapseys Wood in Buckinghamshire and Quest Park in Bedfordshire will draw 1.3 gigawatts between them, about what a large power station puts out, and produce more than 4.5m tonnes of carbon, against 3.9m tonnes from the whole of ExxonMobil’s UK operations in 2023. Both plan to run their own gas-fired stations because they cannot get grid connections in time. The government calls the figures misleading on the grounds that they assume full demand from day one; Foxglove, who did the analysis, says that assumption is industry standard, and published it, which is the only reason anyone can argue with it.

Leor asked the question I keep coming back to: we are building the most advanced technology in human history and powering it like it is 1975. The UK has no frontier models of its own. We are taking the environmental cost so that companies elsewhere can train theirs. At British infrastructure speed these sites are five years away, which is five years we could spend making the models less hungry instead.


2. Oura marketed 95%, the lawsuit cites 53%

A California woman paid $513.68 for a ring that scores her sleep, and a class action says it cannot detect sleep stages at all. Oura advertised 95% accuracy against a clinical sleep lab. The complaint cites a study putting it at 53.18%, a coin flip. Real sleep research runs on electrodes at the temples, sensors on the eyelids and a night in a ward, so a ring reading pulse and perspiration at the finger was always going to be inferring rather than measuring, and one marketing number covers both without saying which is which. Oura denies everything.

The timing matters, because Leor pointed out they have sold 5.5 million rings, are forecasting $1.5bn of sales this year, and are preparing an IPO at an $11bn valuation. The other cost is the one nobody sues over: a bad sleep score in the morning is its own reason to sleep badly the next night.


3. Grok hands over your chat history, and nobody wrote it down

A security researcher told Musk’s company in June that a single webpage can make Grok give away a user’s name, location and live chat history. You ask Grok to summarise a page, and instructions hidden in that page tell it to hand over everything. The instructions are scrambled, so the safety checks read them as nonsense and let them through. He chased twice in August, got no reply, and checked again on the 19th. There is no entry in the public list where software flaws get recorded, so there is no public trace that the flaw exists, and the missing entry does the same reassuring work as a good score would.

Leor was more even-handed than me here, and correctly: Copilot and Gemini have had versions of the same problem, and in every case we heard about it from the researcher rather than the company. His other point is the useful one for anybody using these tools: guardrails thin out on the cheaper models, so the same request a paid tier refuses will often go through on a lightweight one.


4. Cocomelon’s studio hands its animators an AI Bible

Moonbug Entertainment, which makes Cocomelon, Blippi and Little Baby Bum, issued animators an AI policy and a document it calls the ‘Studio AI Bible’ telling them to experiment. The guidance reads well. Keep a human in the loop, AI assists the artist rather than replacing them, and AI cannot originate core characters, storylines or song lyrics. The documents tie that rule partly to copyright, on the basis that you can only own what a human created, so the human stays in the loop to keep the copyright and the artists get the benefit as a side effect. There is no headcount floor and no definition of ‘assist’.

Earlier this year the crew and actors on The Melon Patch, the live-action spin-off, struck over pay and benefits, and AI asks for neither. Leor put the objection better than I did: if there is one corner of the internet that does not need an infinite slot machine, it is content built to hold the attention of a three-year-old.


5. A mammogram may also be a heart scan

Cardiovascular disease kills more women than anything else and is routinely caught late. A team in Israel took 97,364 mammograms from 29,921 women, average age 54, and cross-referenced their records. A model trained on the scans picked out the women who had the condition 86% of the time for stroke, 79% for high blood pressure and 78% for coronary heart disease, from the mammogram alone. It held whatever her age, and whether or not she also had cancer. A radiologist reading a mammogram is looking for breast cancer and is not asked to look for anything else, so the signal has been sitting in hundreds of millions of scans that nobody was reading for it.


Four of those numbers were built to reassure and shrank when somebody outside checked them. The fifth arrived with its limits attached by the people who produced it.

And the fifth story is the one that sends you back to the first. Nobody objects to AI that finds heart disease in a scan a woman was already having. Plenty of people object to AI that generates more content for toddlers to become addicted to. The datacentres are being built for both. The technology works. The argument is about where we have decided to point it.

If you spotted the t-shirt on the live, there are Slow AI T-shirts now. You can buy them here.

Go slow.


Discussion about this video

User's avatar

Ready for more?