Topic
AGI strategy and foresight
Planning for several futures at once. Which scenarios are worth preparing for, what to write and build and rehearse now for each of them, and how to keep a strategy honest when the thing it is about keeps moving.
This tag is now called AGI strategy and foresight. The old address still works and still lists the same writing.
8 posts.
Read Dan Selsam on AI Risk
Dan Selsam’s statement on AI risk is my recommended reading this week, alongside some thoughts on the debate about AI safety funding.
How I Came to Take AI Risk Seriously
My path from dismissing AI risk to working on AI safety, and the effort to think clearly amid fear, social pressure, and uncertainty.
Hard Truths About Where AI Is Headed
Four LinkedIn posts collected in one place: coding jobs going away, planning as the scarce skill, rogue models arriving, and what the Pentagon story says about red lines.
Quantum Computing, Photonics, and Energy Bottlenecks for AGI
My answer is that quantum probably does not matter and photonics might. The energy numbers, why optical transistors are the wrong thing to judge photonics by, and OpenAI's hire.
AI Insights #1: How Misalignment Could Lead to Takeover & Necessary Safety Properties
A round-up of three: Christiano on takeover thresholds, Shane Legg's necessary properties for any AGI safety plan, and why the danger sits in the agents built on top of models.
Near-Term AI capabilities probably bring low-hanging fruits for global poverty/health
A call to charity entrepreneurs in December 2022. Models of that generation were about to make some interventions viable, and the non-profit sector had not caught up.
Foresight for AGI Safety Strategy
Governance people should be writing the policy document and building the demo for each plausible scenario now, rather than starting the day a minister gives them three months.
Helping organizations survive disasters (and potentially avoid them altogether)
My 2019 primer on strategic foresight, reshared for AGI forecasters. Low-probability high-impact developments get ignored because they are unlikely, and Bostrom's urn is the reason to look anyway.