AI Watch

Welcome! This is a website to track people and organizations in the AI safety/alignment/AI existential risk communities. A position or organization being on AI Watch does not indicate an assessment that that position or organization is actually making AI safer or that the position or organization is good for the world in any way. It is mostly a sociological indication that the position or organization is associated with these communities, as well as an indication that the position or organization claims to be working on AI safety or alignment. (There are some plans to eventually introduce such assessments on AI Watch, but for now there are none.) See the code repository for the source code and data of this website.

This website is developed by Issa Rice with data contributions from Sebastian Sanchez, Amana Rice, and Vipul Naik, and has been partially funded by Vipul Naik and Mati Roy (who in July 2023 paid for the time Issa had spent answering people’s questions about AI Watch up until that point).

Last updated on 2024-11-02; see here for a full list of recent changes.

Table of contents

Agendas

Agenda name Associated people Associated organizations
Iterated amplification Paul Christiano, Buck Shlegeris, Dario Amodei OpenAI
Embedded agency Eliezer Yudkowsky, Scott Garrabrant, Abram Demski Machine Intelligence Research Institute
Comprehensive AI services Eric Drexler Future of Humanity Institute
Ambitious value learning Stuart Armstrong Future of Humanity Institute
Factored cognition Andreas Stuhlmüller Ought
Recursive reward modeling Jan Leike, David Krueger, Tom Everitt, Miljan Martic, Vishal Maini, Shane Legg Google DeepMind
Debate Paul Christiano OpenAI
Interpretability Christopher Olah
Inverse reinforcement learning
Preference learning
Cooperative inverse reinforcement learning
Imitation learning
Alignment for advanced machine learning systems Jessica Taylor, Eliezer Yudkowsky, Patrick LaVictoire, Andrew Critch Machine Intelligence Research Institute
Learning-theoretic AI alignment Vanessa Kosoy
Counterfactual reasoning Jacob Steinhardt

Positions grouped by person

Showing 27 people with positions.

Name Number of organizations List of organizations
Paul Christiano 4 AI Impacts, Machine Intelligence Research Institute, OpenAI, Ought
Oliver Habryka 3 Center for Applied Rationality, Lightcone Infrastructure, Machine Intelligence Research Institute
Ryan Carey 3 Centre for the Study of Existential Risk, Machine Intelligence Research Institute, Ought
Ben Goldhaber 2 FAR.AI, Ought
Ben Weinstein-Raun 2 Machine Intelligence Research Institute, Ought
Beth Barnes 2 Center for Human-Compatible AI, Centre for the Study of Existential Risk
Chris Cundy 2 FAR.AI, Ought
Chris Maddison 2 Google DeepMind, University of Oxford
Connor Flexman 2 AI Impacts, Machine Intelligence Research Institute
Dmitrii Krasheninnikov 2 Center for Human-Compatible AI, University of Amsterdam
Elizabeth Cooper 2 Berkeley Existential Risk Initiative, Centre for the Study of Existential Risk
Eric Rogstad 2 Berkeley Existential Risk Initiative, Lightcone Infrastructure
Ian McKenzie 2 FAR.AI, Ought
Jaan Tallinn 2 Centre for the Study of Existential Risk, Machine Intelligence Research Institute
Jeremy Schlatter 2 Berkeley Existential Risk Initiative, Machine Intelligence Research Institute
Jimmy Rintjema 2 AI Impacts, Machine Intelligence Research Institute
Johannes Heidecke 2 AI Safety Camp, Road to AI Safety Excellence
Josh Jacobson 2 Berkeley Existential Risk Initiative, FAR.AI
Kaj Sotala 2 Foundational Research Institute, Machine Intelligence Research Institute
Katja Grace 2 AI Impacts, Machine Intelligence Research Institute
Matthew Graves 2 Lightcone Infrastructure, Machine Intelligence Research Institute
Nick Bostrom 2 Centre for the Study of Existential Risk, Machine Intelligence Research Institute
Remmelt Ellen 2 AI Safety Camp, Road to AI Safety Excellence
Robert Mushkatblat 2 Lightcone Infrastructure, Machine Intelligence Research Institute
Sam Clarke 2 Centre for the Study of Existential Risk, Future of Humanity Institute
Stuart Russell 2 Centre for the Study of Existential Risk, Machine Intelligence Research Institute
Tom McGrath 2 Future of Humanity Institute, Ought

Positions grouped by organization

Showing 36 organizations.

Organization Number of people List of people
Machine Intelligence Research Institute 143 Jimmy Rintjema, Lisa Thiergart, Gretta Duleba, James Payor, Edward Kmett, Victoria Krakovna, Christine Peterson, Carson Jones, Linda Linsefors, Evan Hubinger, David Simmons, Daniel Demski, Ben Weinstein-Raun, Alex Zhu, Alex Mennen, Alex Appel, Rob Bensinger, Buck Shlegeris, Nick Tarleton, Blake Borgeson, Kurt Brown, Jesse Liptrap, Benjamin Mann, Sam Eisenstat, Jeremy Schlatter, Jan Leike, Andrew Critch, Matthew Graves, Ryan Carey, Connor Flexman, Colm Ó Riain, Aaron Silverbook, Gary Drescher, Kaya Stechly, Andrew Lapinski-Barker, Robin Hanson, Anna Salamon, Ramana Kumar, Jack Gallagher, Jaan Tallinn, Bart Selman, Stuart Russell, Vanessa Kosoy, Nate Thomas, Abram Demski, Stuart Armstrong, Jessica Taylor, Jed McCaleb, Jake Moskowitz, Scott Garrabrant, Jesse Galef, Matthew Fallshaw, Tsvi Benson-Tilsen, Eliezer Yudkowsky, Peter Thiel, Liron Shapira, Nicolas Gagné, Elizabeth Morningstar, Lila Rieber, Vipul Naik, Nate Soares, Daniel Lewis, Richard Neal, Robert Mushkatblat, Dávid Natingga, Nick Bostrom, Nathan Clark, Moshe Looks, Kaj Sotala, Steve Omohundro, Roman Yampolskiy, Evan Erickson, Sebastian Nickel, Oliver Habryka, Paul Christiano, Patrick LaVictoire, Nisan Stiennon, Mihaly Barasz, Jeremy Miller, Bill Hibbard, Benya Fallenstein, Alex Altair, Stephen Barnes, Louie Helm, Patrick Robotham, Daniel Roth, Pedro Chaves, Topher Brennan, Carl Shulman, Nickolai Leschov, Jonathan Wang, Cameron Taylor, Malo Bourgon, Jake Miller, Gwern Branwen, Erica Edelman, Alex Vermeer, Tomer Kagan, Luke Muehlhauser, Lincoln Quirk, Nevin Freeman, Minda Myers, Keefe Roedersheimer, Diego Caleiro, Will Newsome, Peter Scheyer, Peter de Blanc, Jasen Murray, Luke Grecki, Daniel Dewey, Abraham Wolk, Thomas Colthurst, Stanislas Sochacki, Janos Kramar, Dennis Fan, Ben Hoskin, Jason Levin, Tim Czech, Frank Adamek, Amy Willey, Kevin Fischer, Ray Kurzweil, Ben Goertzel, Michael Anissimov, Harrison Willey, Edwin Evans, Steve Rayhawk, Michael Blume, Kemal Eren, Andrew Rettek, Katja Grace, Justin Shovelain, Henrik Jonsson, Andriy Brodskyy, Andrew Hay, Vincent Fagot, Thomas McCabe, Steven Kaas, Roko Mijic, Bryan Bishop, Alyssa Vance, Marcello Herreshoff, Jeff Alexander
Centre for the Study of Existential Risk 100 Pablo Suarez, Reuben Makomere, Aarathi Krishnan, Thomas Moynihan, Elizabeth Cooper, Alexandra Klein, Laura Elmer, Shoshana Dahdi, Clare Arnstein, Kennedy Mbeva, Julian Huppert, Taniel Yusef, Cecil Abungu, Madhulika Srikumar, Zoe Hemsley, Matthew Connelly, Constantin Arnscheidt, Coleman Snell, Clarissa Rios Rojas, Dennis Müller, Sarah Dryhurst, Maurice Chiodo, Sam Clarke, Fazl Barez, Ross Gruetzemacher, Abdullahi Alim, Nathaniel Cooke, Paul Ingram, José Hernández-Orallo, Freya Jephcott, Jessica Bland, Catherine Rhodes, Matthijs M. Maas, Charlotte Christiane Hammer, Tom Hobson, Shin-Shin Hua, Lara Mani, Shahar Avin, Rumtin Sepasspour, S. J. Beard, Carla Zoe Cremer, Adrian Weller, Chris Lowe, Adrian Kent, Sean Holden, Stephen Hawking, Hermann Hauser, Tim Crane, David Cleevely, Jonathan Wiener, Max Tegmark, Peter Singer, Murray Shanahan, Dana Scott, Stuart Russell, Peter Piot, Tim Palmer, Elon Musk, Robert May, David Chalmers, Nick Bostrom, Margaret Boden, Martina Kunz, Beth Barnes, Yang Liu, Simon Goldhill, Jane Heal, Partha Dasgupta, Lalitha Sundaram, Haydn Belfield, Seán Ó hÉigeartaigh, Huw Price, Jaan Tallinn, Martin Rees, Alison Gopnik, Ryan Carey, William Sutherland, Susan Owens, Mami Mizutori, Piers Millett, Thomas Homer-Dixon, Robert Doubleday, Beatrice Crona, Belinda Cleeland, Des Browne, Yuval Noah Harari, Olaf Corry, Seth Baum, Caroline Baylon, Sophie Dannreuther, Charlotte Hammer, Jaime Sevilla, David Krueger, Elizabeth Seger, Di Cooke, Mike Cassidy, James Ginns, Andrew Tanentzap, Allan Dafoe, Rachel Burgess
FAR.AI 51 Jessica Lim, Edward Yee, Lilian Hughes, Chris Cundy, Taylor Boyle, Lindsay Murachver, Jeremy Rich, Saad Siddiqui, Oskar Hollinsworth, Anastasiia Gaidashenko, Philip Quirke, Brendan Murphy, Isabella Duan, Ian McKenzie, Dillon Bowen, Michał Zając, Siao Si Looi, Chris MacLeod, Aaron Tucker, Claudia Shi, Moritz von Knebel, Tony Wang, Fynn Heide, Kellin Pelrine, Ben Goldhaber, Lev McKinney, Adrià Garriga-Alonso, Pablo Moreno, Tomasz Korbak, Pedro Freire, Nora Belrose, Jérémy Scheurer, Juan Rocamonde, Alex Tamkin, Niki Howe, Ethan Perez, ChengCheng Tan, Adam Gleave, Nino Scherrer, Sawyer Bernath, Karl Berzins, Hannah Betts, Edmund Mills, Josh Jacobson, Alyse Spiehler, Tom Tseng, Mohammad Taufeeque, Joseph Miller, Euan McLean, Lawrence Chan, Scott Emmons
GoodAI 32 Karolína H., Ryan Camilleri, Jose Solorzano, Alex Angelini, Dominik Čech, Sarka Krejcova, Stephanie Wendler, Reham Bukhari, Šimon Šicko, Lucia Šicková, Nicholas Guttenberg, Viktorie Knezkova, Steffen Eichler, Isabeau Premont-Schwarz, Filip Hauptfleisch, Petr Sramek, Jan Štafa, Christine Lee, Michal Dvořák, Will Millership, Lucie Krestova, Marek Havrda, Jan Feyereisl, Olga Afanasjeva, Martin Poliak, Marek Rosa, Simon Andersson, Přemek Paška, Jaroslav Vitku, Joseph Davidson, Shantesh Patil, Petr Hlubuček
Ought 24 Luke Stebbing, Ian McKenzie, Justin Reppert, Eli Lifland, Amanda Ngo, Aparna Ashok, Jungwon Byun, Paul Christiano, Ozzie Gooen, Owain Evans, Neal Jean, Milan Griffes, Girish Sastry, Chris Cundy, Ben Weinstein-Raun, Ben Goldhaber, Andrew Schreiber, Ben Rachbach, Zachary Miller, Zac Kenton, Tom McGrath, Noah Goodman, Ben West, Ryan Carey
AI Impacts 15 Jeffrey Heninger, Jimmy Rintjema, Richard Korzekwa, Katja Grace, Daniel Kokotajlo, Asya Bergal, Ronja Lutz, Tegan McCaslin, Paul Christiano, Ben Hoffman, Justis Mills, Connor Flexman, Finan Adamson, John Salvatier, Stephanie Zolayvar
Lightcone Infrastructure 11 Robert Mushkatblat, Rafe Kennedy, Jacob Lagerros, Ruben Bloom, Raymond Arnold, Matthew Graves, Harmanas Chopra, Eric Rogstad, Ben Albert Pace, Oliver Habryka, James Babcock
Berkeley Existential Risk Initiative 9 Elizabeth Cooper, Sofia Davis-Fogel, Alex Flint, Josh Jacobson, Sam Bankman-Fried, Colleen Gleason, Jeremy Schlatter, Qiaochu Yuan, Eric Rogstad
OpenAI 9 Mor Katz, Christopher Olah, Jeffrey Wu, Ethan Knight, Daniel Ziegler, Joshua Achiam, Geoffrey Irving, Paul Christiano, Dario Amodei
Road to AI Safety Excellence 8 Remmelt Ellen, Trent Fowler, Erik Istre, Rupert McCallum, Robert Miles, Johannes Heidecke, Veerle de Goederen, Toon Alfrink
AI Safety Camp 7 Remmelt Ellen, Jessica Cooper, Kristina Nemcova, Jirí Nadvorník, Anne Wissemann, Jan Kulveit, Johannes Heidecke
Center for Applied Rationality 6 Logan Brienne Strohl, Xavier Prospero, Brienne Strohl, Luke Raskopf, Adom Hartell, Oliver Habryka
Foundational Research Institute 5 Brian Tomasik, Kaj Sotala, Caspar Oesterheld, Lukas Gloor, Tobias Baumann
Future of Humanity Institute 5 Sam Clarke, Tom McGrath, Sören Mindermann, Tamay Besiroglu, Anders Sandberg
EthicsNet 4 Aleksandra Orchowska, Remco Bloemen, Anish Mohammed, Nell Watson
Center for Human-Compatible AI 3 Christopher Cundy, Beth Barnes, Dmitrii Krasheninnikov
Google DeepMind 3 Vishal Maini, Pedro A. Ortega, Chris Maddison
AIDEUS 2 Sergey Rodionov, Alexey Potapov
Learning Intelligent Distribution Agent 2 Tamas Madl, Stan Franklin
University of Oxford 2 Ruth Fong, Chris Maddison
1 Angela P.
Australian National University 1 Jarryd Martin
Carnegie Mellon University 1 Noam Brown
Centre for Effective Altruism 1 Johannes Treutlein
ETH Zurich 1 Felix Berkenkamp
Institute of Ethics and Emerging Technologies 1 Steven Umbrello
Massachusetts Institute of Technology 1 Jon Gauthier
Oregon State University 1 Thomas Dietterich
Phenomenological AI Safety Research Institute 1 G Gordon Worley III
Sorbonne University 1 Michaël Trazzi
Stanford University 1 Aditi Raghunathan
The Australian National University 1 Michael Cohen
The Consortium on the Landscape of AI Safety 1 Alexis Carlier
University of Amsterdam 1 Dmitrii Krasheninnikov
University of California, Berkeley 1 Michael Janner
University of Toronto 1 Roger Grosse