AI Watch

Welcome! This is a website to track people and organizations in the AI safety/alignment/AI existential risk communities. A position or organization being on AI Watch does not indicate an assessment that that position or organization is actually making AI safer or that the position or organization is good for the world in any way. It is mostly a sociological indication that the position or organization is associated with these communities, as well as an indication that the position or organization claims to be working on AI safety or alignment. (There are some plans to eventually introduce such assessments on AI Watch, but for now there are none.) See the code repository for the source code and data of this website.

This website is developed by Issa Rice with data contributions from Sebastian Sanchez, Amana Rice, and Vipul Naik, and has been partially funded by Vipul Naik and Mati Roy (who in July 2023 paid for the time Issa had spent answering people’s questions about AI Watch up until that point).

If you like (or want to like) this website and have money: the current funder is mostly only funding data updates to existing organizations as well as adding data for some new effective altruist organizations. As a result, the site is not getting any new features or improvements in design. If you want to bring this site to the next level, contact Issa at riceissa@gmail.com. What you get: site improvements, recognition in the site credits. What the site needs: money.

If you have time and want experience building websites: this website is looking for contributors. If you want to help out, contact Issa at riceissa@gmail.com. What you get: little or no pay (this could change if the site gets funding; see previous paragraph), recognition in the site credits, privilege of working with me, knowledge of the basics of web development (MySQL, PHP, Git). What the site needs: data collection/entry and website code improvements.

Last updated on 2024-04-15; see here for a full list of recent changes.

Table of contents

Agendas

Agenda name Associated people Associated organizations
Iterated amplification Paul Christiano, Buck Shlegeris, Dario Amodei OpenAI
Embedded agency Eliezer Yudkowsky, Scott Garrabrant, Abram Demski Machine Intelligence Research Institute
Comprehensive AI services Eric Drexler Future of Humanity Institute
Ambitious value learning Stuart Armstrong Future of Humanity Institute
Factored cognition Andreas Stuhlmüller Ought
Recursive reward modeling Jan Leike, David Krueger, Tom Everitt, Miljan Martic, Vishal Maini, Shane Legg Google DeepMind
Debate Paul Christiano OpenAI
Interpretability Christopher Olah
Inverse reinforcement learning
Preference learning
Cooperative inverse reinforcement learning
Imitation learning
Alignment for advanced machine learning systems Jessica Taylor, Eliezer Yudkowsky, Patrick LaVictoire, Andrew Critch Machine Intelligence Research Institute
Learning-theoretic AI alignment Vanessa Kosoy
Counterfactual reasoning Jacob Steinhardt

Positions grouped by person

Showing 23 people with positions.

Name Number of organizations List of organizations
Paul Christiano 5 AI Impacts, Machine Intelligence Research Institute, OpenAI, Ought, Theiss Research
Oliver Habryka 3 Center for Applied Rationality, Lightcone Infrastructure, Machine Intelligence Research Institute
Ryan Carey 3 Centre for the Study of Existential Risk, Machine Intelligence Research Institute, Ought
Andrew Critch 2 Berkeley Existential Risk Initiative, Machine Intelligence Research Institute
Ben Weinstein-Raun 2 Machine Intelligence Research Institute, Ought
Beth Barnes 2 Center for Human-Compatible AI, Centre for the Study of Existential Risk
Chris Maddison 2 Google DeepMind, University of Oxford
Connor Flexman 2 AI Impacts, Machine Intelligence Research Institute
Dmitrii Krasheninnikov 2 Center for Human-Compatible AI, University of Amsterdam
Eric Rogstad 2 Berkeley Existential Risk Initiative, Lightcone Infrastructure
Jaan Tallinn 2 Centre for the Study of Existential Risk, Machine Intelligence Research Institute
Jeremy Schlatter 2 Berkeley Existential Risk Initiative, Machine Intelligence Research Institute
Jimmy Rintjema 2 AI Impacts, Machine Intelligence Research Institute
Johannes Heidecke 2 AI Safety Camp, Road to AI Safety Excellence
Kaj Sotala 2 Foundational Research Institute, Machine Intelligence Research Institute
Katja Grace 2 AI Impacts, Machine Intelligence Research Institute
Matthew Graves 2 Lightcone Infrastructure, Machine Intelligence Research Institute
Nick Bostrom 2 Centre for the Study of Existential Risk, Machine Intelligence Research Institute
Remmelt Ellen 2 AI Safety Camp, Road to AI Safety Excellence
Robert Mushkatblat 2 Lightcone Infrastructure, Machine Intelligence Research Institute
Sam Clarke 2 Future of Humanity Institute, The Future Society
Stuart Russell 2 Centre for the Study of Existential Risk, Machine Intelligence Research Institute
Tom McGrath 2 Future of Humanity Institute, Ought

Positions grouped by organization

Showing 37 organizations.

Organization Number of people List of people
Machine Intelligence Research Institute 141 Jimmy Rintjema, James Payor, Edward Kmett, Victoria Krakovna, Christine Peterson, Carson Jones, Linda Linsefors, Evan Hubinger, David Simmons, Daniel Demski, Ben Weinstein-Raun, Alex Zhu, Alex Mennen, Alex Appel, Buck Shlegeris, Blake Borgeson, Nick Tarleton, Kurt Brown, Jesse Liptrap, Benjamin Mann, Sam Eisenstat, Jeremy Schlatter, Jan Leike, Andrew Critch, Matthew Graves, Ryan Carey, Connor Flexman, Colm Ó Riain, Aaron Silverbook, Gary Drescher, Kaya Stechly, Andrew Lapinski-Barker, Robin Hanson, Anna Salamon, Jack Gallagher, Jaan Tallinn, Bart Selman, Stuart Russell, Ramana Kumar, Vanessa Kosoy, Abram Demski, Stuart Armstrong, Nate Thomas, Jessica Taylor, Jed McCaleb, Jake Moskowitz, Scott Garrabrant, Rob Bensinger, Jesse Galef, Tsvi Benson-Tilsen, Matthew Fallshaw, Peter Thiel, Liron Shapira, Elizabeth Morningstar, Antonius Lourenço Kasbergen, Nicolas Gagné, Lila Rieber, Vipul Naik, Nate Soares, Daniel Lewis, Richard Neal, Robert Mushkatblat, Dávid Natingga, Kaj Sotala, Steve Omohundro, Roman Yampolskiy, Nick Bostrom, Nathan Clark, Moshe Looks, Evan Erickson, Sebastian Nickel, Oliver Habryka, Jeremy Miller, Bill Hibbard, Benya Fallenstein, Alex Altair, Paul Christiano, Patrick LaVictoire, Nisan Stiennon, Mihaly Barasz, Stephen Barnes, Louie Helm, Daniel Roth, Patrick Robotham, Pedro Chaves, Topher Brennan, Carl Shulman, Jonathan Wang, Cameron Taylor, Nickolai Leschov, Jake Miller, Gwern Branwen, Erica Edelman, Alex Vermeer, Tomer Kagan, Malo Bourgon, Luke Muehlhauser, Lincoln Quirk, Keefe Roedersheimer, Diego Caleiro, Will Newsome, Nevin Freeman, Minda Myers, Peter Scheyer, Jasen Murray, Peter de Blanc, Daniel Dewey, Abraham Wolk, Thomas Colthurst, Stanislas Sochacki, Luke Grecki, Janos Kramar, Dennis Fan, Ben Hoskin, Jason Levin, Tim Czech, Frank Adamek, Amy Willey, Kevin Fischer, Ray Kurzweil, Ben Goertzel, Harrison Willey, Steve Rayhawk, Michael Anissimov, Kemal Eren, Michael Blume, Andrew Rettek, Katja Grace, Justin Shovelain, Henrik Jonsson, Andriy Brodskyy, Andrew Hay, Vincent Fagot, Thomas McCabe, Steven Kaas, Roko Mijic, Bryan Bishop, Alyssa Vance, Eliezer Yudkowsky, Marcello Herreshoff, Jeff Alexander
GoodAI 32 Karolína H., Ryan Camilleri, Jose Solorzano, Alex Angelini, Dominik Čech, Sarka Krejcova, Stephanie Wendler, Reham Bukhari, Šimon Šicko, Lucia Šicková, Nicholas Guttenberg, Viktorie Knezkova, Steffen Eichler, Isabeau Premont-Schwarz, Filip Hauptfleisch, Petr Sramek, Jan Štafa, Christine Lee, Michal Dvořák, Will Millership, Lucie Krestova, Marek Havrda, Jan Feyereisl, Olga Afanasjeva, Martin Poliak, Marek Rosa, Simon Andersson, Přemek Paška, Jaroslav Vitku, Shantesh Patil, Petr Hlubuček, Joseph Davidson
Centre for the Study of Existential Risk 24 Adrian Weller, Sean Holden, Stephen Hawking, Tim Crane, Max Tegmark, Murray Shanahan, Dana Scott, Stuart Russell, Elon Musk, Alison Gopnik, David Chalmers, Nick Bostrom, Margaret Boden, Ryan Carey, Martina Kunz, Seth Baum, Beth Barnes, Yang Liu, Jaan Tallinn, Martin Rees, Huw Price, Haydn Belfield, Shahar Avin, Seán Ó hÉigeartaigh
Ought 24 Luke Stebbing, Ian McKenzie, Justin Reppert, Eli Lifland, Amanda Ngo, Aparna Ashok, Jungwon Byun, Paul Christiano, Ozzie Gooen, Owain Evans, Neal Jean, Milan Griffes, Girish Sastry, Chris Cundy, Ben Weinstein-Raun, Ben Goldhaber, Andrew Schreiber, Ben Rachbach, Zachary Miller, Zac Kenton, Tom McGrath, Noah Goodman, Ben West, Ryan Carey
AI Impacts 14 Daniel Kokotajlo, Asya Bergal, Ronja Lutz, Richard Korzekwa, Tegan McCaslin, Paul Christiano, Jimmy Rintjema, Ben Hoffman, Justis Mills, Connor Flexman, Finan Adamson, John Salvatier, Katja Grace, Stephanie Zolayvar
Lightcone Infrastructure 11 Robert Mushkatblat, Rafe Kennedy, Jacob Lagerros, Ruben Bloom, Raymond Arnold, Matthew Graves, Harmanas Chopra, Eric Rogstad, Ben Albert Pace, Oliver Habryka, James Babcock
Berkeley Existential Risk Initiative 10 Elizabeth Cooper, Sofia Davis-Fogel, Alex Flint, Josh Jacobson, Sam Bankman-Fried, Colleen Gleason, Jeremy Schlatter, Qiaochu Yuan, Eric Rogstad, Andrew Critch
OpenAI 9 Mor Katz, Christopher Olah, Jeffrey Wu, Ethan Knight, Daniel Ziegler, Joshua Achiam, Geoffrey Irving, Paul Christiano, Dario Amodei
Road to AI Safety Excellence 8 Remmelt Ellen, Trent Fowler, Erik Istre, Rupert McCallum, Robert Miles, Johannes Heidecke, Veerle de Goederen, Toon Alfrink
AI Safety Camp 7 Remmelt Ellen, Jessica Cooper, Kristina Nemcova, Jirí Nadvorník, Anne Wissemann, Jan Kulveit, Johannes Heidecke
Center for Applied Rationality 6 Logan Brienne Strohl, Xavier Prospero, Brienne Strohl, Luke Raskopf, Adom Hartell, Oliver Habryka
Future of Humanity Institute 6 Sam Clarke, Tom McGrath, Sören Mindermann, Tamay Besiroglu, Toby Ord, Anders Sandberg
Foundational Research Institute 5 Brian Tomasik, Kaj Sotala, Caspar Oesterheld, Lukas Gloor, Tobias Baumann
EthicsNet 4 Aleksandra Orchowska, Remco Bloemen, Anish Mohammed, Nell Watson
Center for Human-Compatible AI 3 Christopher Cundy, Beth Barnes, Dmitrii Krasheninnikov
Google DeepMind 3 Vishal Maini, Pedro A. Ortega, Chris Maddison
AIDEUS 2 Sergey Rodionov, Alexey Potapov
Learning Intelligent Distribution Agent 2 Tamas Madl, Stan Franklin
University of Oxford 2 Ruth Fong, Chris Maddison
1 Angela P.
Australian National University 1 Jarryd Martin
Carnegie Mellon University 1 Noam Brown
Centre for Effective Altruism 1 Johannes Treutlein
ETH Zurich 1 Felix Berkenkamp
Institute of Ethics and Emerging Technologies 1 Steven Umbrello
Massachusetts Institute of Technology 1 Jon Gauthier
Oregon State University 1 Thomas Dietterich
Phenomenological AI Safety Research Institute 1 G Gordon Worley III
Sorbonne University 1 Michaël Trazzi
Stanford University 1 Aditi Raghunathan
The Australian National University 1 Michael Cohen
The Consortium on the Landscape of AI Safety 1 Alexis Carlier
The Future Society 1 Sam Clarke
Theiss Research 1 Paul Christiano
University of Amsterdam 1 Dmitrii Krasheninnikov
University of California, Berkeley 1 Michael Janner
University of Toronto 1 Roger Grosse