Topic
AGI strategy and foresight
Planning for several futures at once. Which scenarios are worth preparing for, what to write and build and rehearse now for each of them, and how to keep a strategy honest when the thing it is about keeps moving.
This tag is now called AGI strategy and foresight. The old address still works and still lists the same writing.
6 posts.
Hard Truths About Where AI Is Headed
Four LinkedIn posts collected in one place: coding jobs going away, planning as the scarce skill, rogue models arriving, and what the Pentagon story says about red lines.
Quantum Computing, Photonics, and Energy Bottlenecks for AGI
My answer is that quantum probably does not matter and photonics might. The energy numbers, why optical transistors are the wrong thing to judge photonics by, and OpenAI's hire.
AI Insights #1: How Misalignment Could Lead to Takeover & Necessary Safety Properties
A round-up of three: Christiano on takeover thresholds, Shane Legg's necessary properties for any AGI safety plan, and why the danger sits in the agents built on top of models.
Near-Term AI capabilities probably bring low-hanging fruits for global poverty/health
A call to charity entrepreneurs in December 2022. Models of that generation were about to make some interventions viable, and the non-profit sector had not caught up.
Foresight for AGI Safety Strategy
Governance people should be writing the policy document and building the demo for each plausible scenario now, rather than starting the day a minister gives them three months.
Helping organizations survive disasters (and potentially avoid them altogether)
My 2019 primer on strategic foresight, reshared for AGI forecasters. Low-probability high-impact developments get ignored because they are unlikely, and Bostrom's urn is the reason to look anyway.