AI Watch

Welcome! This is a website to track people and organizations working on AI safety. See the code repository for the source code and data of this website.

This website is developed by Issa Rice with data contributions from Sebastian Sanchez, Amana Rice, and Vipul Naik, and has been partially funded by Vipul Naik and Mati Roy (who in July 2023 paid for the time Issa had spent answering people’s questions about AI Watch up until that point).

If you like (or want to like) this website and have money: the current funder is mostly only funding data updates to existing organizations as well as adding data for some new effective altruist organizations. As a result, the site is not getting any new features or improvements in design. If you want to bring this site to the next level, contact Issa at riceissa@gmail.com. What you get: site improvements, recognition in the site credits. What the site needs: money.

If you have time and want experience building websites: this website is looking for contributors. If you want to help out, contact Issa at riceissa@gmail.com. What you get: little or no pay (this could change if the site gets funding; see previous paragraph), recognition in the site credits, privilege of working with me, knowledge of the basics of web development (MySQL, PHP, Git). What the site needs: data collection/entry and website code improvements.

Last updated on 2023-09-07; see here for a full list of recent changes.

Table of contents

Agendas

Agenda name Associated people Associated organizations
Iterated amplification Paul Christiano, Buck Shlegeris, Dario Amodei OpenAI
Embedded agency Eliezer Yudkowsky, Scott Garrabrant, Abram Demski Machine Intelligence Research Institute
Comprehensive AI services Eric Drexler Future of Humanity Institute
Ambitious value learning Stuart Armstrong Future of Humanity Institute
Factored cognition Andreas Stuhlmüller Ought
Recursive reward modeling Jan Leike, David Krueger, Tom Everitt, Miljan Martic, Vishal Maini, Shane Legg Google DeepMind
Debate Paul Christiano OpenAI
Interpretability Christopher Olah
Inverse reinforcement learning
Preference learning
Cooperative inverse reinforcement learning
Imitation learning
Alignment for advanced machine learning systems Jessica Taylor, Eliezer Yudkowsky, Patrick LaVictoire, Andrew Critch Machine Intelligence Research Institute
Learning-theoretic AI alignment Vanessa Kosoy
Counterfactual reasoning Jacob Steinhardt

AI safety relation by subject

Note: as shown by the large number of “unknown” values, most of the positions haven’t been categorized by relation/subject so this table will only be useful in the future.

Subject UnknownAGI organizationGCR organizationpositionunrelated Total
Unknown 7570 369 60 273 1 8273
background 1 0 0 24 0 25
general 0 0 2 50 0 52
policy 0 0 0 1 0 1
popularization 0 0 0 2 0 2
software engineering 0 2 0 8 0 10
strategy 0 0 0 1 0 1
technical research 9 2 3 34 1 49
Total 7580 373 65 393 2 8413

Positions summary by year

Note: as shown by the large number of “unknown” values, most of the positions haven’t been categorized by start/end dates so this table will only be useful in the future.

Year Start date End date
Unknown 1218 6044
1986 1 0
1993 1 0
1997 3 0
1999 2 1
2000 5 0
2001 5 0
2002 62 1
2003 15 1
2004 32 4
2005 60 5
2006 37 13
2007 48 4
2008 75 7
2009 129 16
2010 174 41
2011 212 53
2012 161 74
2013 192 89
2014 253 65
2015 355 151
2016 631 222
2017 674 264
2018 818 364
2019 847 274
2020 848 257
2021 867 239
2022 559 168
2023 129 56

Positions grouped by person

Showing 173 people with positions.

Name Number of organizations List of organizations
Paul Christiano 9 AI Impacts, Future of Humanity Institute, Machine Intelligence Research Institute, Open Philanthropy, OpenAI, Ought, Redwood Research, Theiss Research, University of California, Berkeley
Nick Bostrom 7 Centre for the Study of Existential Risk, Future of Humanity Institute, Future of Life Institute, Google DeepMind, Leverhulme Centre for the Future of Intelligence, Machine Intelligence Research Institute, University of Oxford
Stuart Russell 7 Berkeley Existential Risk Initiative, Center for Human-Compatible AI, Centre for the Study of Existential Risk, Future of Life Institute, Leverhulme Centre for the Future of Intelligence, Machine Intelligence Research Institute, University of California, Berkeley
Andrew Critch 6 Berkeley Existential Risk Initiative, Center for Applied Rationality, Center for Human-Compatible AI, Encultured AI, Machine Intelligence Research Institute, University of California, Berkeley
Dario Amodei 5 Anthropic, Cooperative AI Foundation, Google Brain, Open Philanthropy, OpenAI
Ryan Carey 5 Centre for the Study of Existential Risk, Future of Humanity Institute, Machine Intelligence Research Institute, OpenAI, Ought
Seán Ó hÉigeartaigh 5 Berkeley Existential Risk Initiative, Centre for the Study of Existential Risk, Future of Humanity Institute, Global Catastrophic Risk Institute, Leverhulme Centre for the Future of Intelligence
Allan Dafoe 4 Cooperative AI Foundation, Future of Humanity Institute, University of Oxford, Yale University
Bas R. Steunebrink 4 IDSIA, NNAISENSE, SUPSI, Università della Svizzera italiana
Heather Roff 4 Arizona State University, Leverhulme Centre for the Future of Intelligence, New America Foundation, University of Oxford
Jaan Tallinn 4 Berkeley Existential Risk Initiative, Centre for the Study of Existential Risk, Future of Life Institute, Machine Intelligence Research Institute
Jan Leike 4 Australian National University, Future of Humanity Institute, Google DeepMind, Machine Intelligence Research Institute
Matthijs Maas 4 Global Catastrophic Risk Institute, Global Politics of Artificial Intelligence Research Group at Yale University and University of Oxford, Hague Centre for Strategic Studies, University of Copenhagen
Miles Brundage 4 Arizona State University, Future of Humanity Institute, General AI Challenge, OpenAI
Roman Yampolskiy 4 General AI Challenge, Global Catastrophic Risk Institute, Machine Intelligence Research Institute, University of Louisville
Seth Baum 4 Centre for the Study of Existential Risk, Global Catastrophic Risk Institute, Machine Intelligence Research Institute, Social & Environmental Entrepreneurs
Adrian Weller 3 Centre for the Study of Existential Risk, Leverhulme Centre for the Future of Intelligence, University of Cambridge
Alison Gopnik 3 Center for Human-Compatible AI, Centre for the Study of Existential Risk, Leverhulme Centre for the Future of Intelligence
Andrew Snyder-Beattie 3 Berkeley Existential Risk Initiative, Future of Humanity Institute, Leverhulme Centre for the Future of Intelligence
Bart Selman 3 Center for Human-Compatible AI, Cornell University, Machine Intelligence Research Institute
Ben Weinstein-Raun 3 Machine Intelligence Research Institute, Ought, Redwood Research
Benjamin Mann 3 Anthropic, Machine Intelligence Research Institute, OpenAI
Christopher Olah 3 Google Brain, Open Philanthropy, OpenAI
Daniel Dewey 3 Future of Humanity Institute, Future of Life Institute, Machine Intelligence Research Institute
Daniela Amodei 3 Anthropic, Epoch, OpenAI
Elon Musk 3 Centre for the Study of Existential Risk, Future of Life Institute, OpenAI
Eric Rogstad 3 Berkeley Existential Risk Initiative, Center for Applied Rationality, Lightcone Infrastructure
Francesca Rossi 3 Future of Life Institute, Leverhulme Centre for the Future of Intelligence, University of Padova
Gillian Hadfield 3 Center for Human-Compatible AI, Cooperative AI Foundation, OpenAI
Girish Sastry 3 Future of Humanity Institute, OpenAI, Ought
Helen Toner 3 Center for Security and Emerging Technology, Future of Humanity Institute, OpenAI
Huw Price 3 Centre for the Study of Existential Risk, Leverhulme Centre for the Future of Intelligence, University of Cambridge
Jack Clark 3 Anthropic, Center for Security and Emerging Technology, OpenAI
Jacob Steinhardt 3 Center for Human-Compatible AI, Open Philanthropy, Stanford University
Janos Kramar 3 Future of Life Institute, Machine Intelligence Research Institute, University of Montreal
Jeremy Schlatter 3 Berkeley Existential Risk Initiative, Machine Intelligence Research Institute, OpenAI
Johannes Treutlein 3 Center for Human-Compatible AI, Centre for Effective Altruism, Effective Altruism Foundation
José Hernández-Orallo 3 General AI Challenge, Leverhulme Centre for the Future of Intelligence, University of Cambridge
Jürgen Schmidhuber 3 IDSIA, SUPSI, Università della Svizzera italiana
Kaj Sotala 3 Foundational Research Institute, Lightcone Infrastructure, Machine Intelligence Research Institute
Kyle Scott 3 Berkeley Existential Risk Initiative, Center for Applied Rationality, Future of Humanity Institute
Laurent Orseau 3 AgroParisTech, Google DeepMind, INRA
Malo Bourgon 3 Berkeley Existential Risk Initiative, Machine Intelligence Research Institute, Redwood Research
Mark Ring 3 IDSIA, SUPSI, Università della Svizzera italiana
Martin Rees 3 Centre for the Study of Existential Risk, Future of Life Institute, Leverhulme Centre for the Future of Intelligence
Matthew Graves 3 Center for Applied Rationality, Lightcone Infrastructure, Machine Intelligence Research Institute
Max Tegmark 3 Centre for the Study of Existential Risk, Future of Life Institute, Machine Intelligence Research Institute
Oliver Habryka 3 Center for Applied Rationality, Lightcone Infrastructure, Machine Intelligence Research Institute
Owain Evans 3 Future of Humanity Institute, Ought, University of Oxford
Patrick LaVictoire 3 Machine Intelligence Research Institute, Quixey, University of Wisconsin–Madison
Pieter Abbeel 3 Center for Human-Compatible AI, OpenAI, University of California, Berkeley
Qiaochu Yuan 3 Berkeley Existential Risk Initiative, Center for Applied Rationality, University of California, Berkeley
Ramana Kumar 3 Data61, Machine Intelligence Research Institute, University of Cambridge
Robin Hanson 3 Future of Humanity Institute, George Mason University, Machine Intelligence Research Institute
Smitha Milli 3 Center for Human-Compatible AI, OpenAI, University of California, Berkeley
Tom Brown 3 Anthropic, Google Brain, OpenAI
Victoria Krakovna 3 Future of Life Institute, Google DeepMind, Machine Intelligence Research Institute
Yang Liu 3 Centre for the Study of Existential Risk, OpenAI, University of Cambridge
Adam Gleave 2 Center for Human-Compatible AI, Fund for Alignment Research
Adam Scholl 2 Center for Applied Rationality, Global Catastrophic Risk Institute
Ales Flidr 2 Centre for Effective Altruism, Future of Life Institute
Alex Turner 2 Center for Human-Compatible AI, Oregon State University
Alex Zhu 2 Machine Intelligence Research Institute, Nonlinear Fund
Alexey Potapov 2 AIDEUS, ITMO University
Amanda Askell 2 Anthropic, OpenAI
Amrit Sidhu-Brar 2 Cooperative AI Foundation, Effective Altruism Foundation
Anca Dragan 2 Center for Human-Compatible AI, University of California, Berkeley
Andreas Stuhlmüller 2 Ought, Stanford University
Anna Salamon 2 Center for Applied Rationality, Machine Intelligence Research Institute
Ben Goertzel 2 CogPrime, Machine Intelligence Research Institute
Ben Goldhaber 2 Center for Applied Rationality, Ought
Benya Fallenstein 2 Machine Intelligence Research Institute, University of Bristol
Beth Barnes 2 Center for Human-Compatible AI, Centre for the Study of Existential Risk
Blake Borgeson 2 Machine Intelligence Research Institute, Redwood Research
Brian Tomasik 2 Effective Altruism Foundation, Foundational Research Institute
Buck Shlegeris 2 Machine Intelligence Research Institute, Redwood Research
Carl Shulman 2 Future of Humanity Institute, Machine Intelligence Research Institute
Carrick Flynn 2 Center for Security and Emerging Technology, Future of Humanity Institute
Catherine Olsson 2 Anthropic, OpenAI
Chris Maddison 2 Google DeepMind, University of Oxford
Christine Peterson 2 Foresight Institute, Machine Intelligence Research Institute
Christopher Cundy 2 Center for Human-Compatible AI, Future of Humanity Institute
Connor Flexman 2 AI Impacts, Machine Intelligence Research Institute
Dan Hendrycks 2 Center for AI Safety, University of California, Berkeley
Daniel Filan 2 Center for Human-Compatible AI, Future of Humanity Institute
Daniel Kokotajlo 2 AI Impacts, Effective Altruism Foundation
Daniel Ziegler 2 OpenAI, Redwood Research
Danny Hernandez 2 Anthropic, OpenAI
David Abel 2 Brown University, Future of Humanity Institute
David Manheim 2 Association for Long Term Existence and Resilience, Future of Humanity Institute
Demis Hassabis 2 Google DeepMind, Leverhulme Centre for the Future of Intelligence
Dmitrii Krasheninnikov 2 Center for Human-Compatible AI, University of Amsterdam
Dylan Hadfield-Menell 2 Center for Human-Compatible AI, University of California, Berkeley
Ethan Perez 2 Fund for Alignment Research, New York University
Gwern Branwen 2 Center for Applied Rationality, Machine Intelligence Research Institute
Holden Karnofsky 2 OpenAI, Redwood Research
Ian McKenzie 2 Fund for Alignment Research, Ought
Jacob Lagerros 2 Future of Humanity Institute, Lightcone Infrastructure
Jade Leung 2 Future of Humanity Institute, OpenAI
Jakob Foerster 2 Center for Human-Compatible AI, OpenAI
James Miller 2 Machine Intelligence Research Institute, Smith College
Jeffrey Ding 2 Center for Security and Emerging Technology, Future of Humanity Institute
Jelena Luketina 2 Aalto University, Université de Montréal
Jesse Clifton 2 Cooperative AI Foundation, Effective Altruism Foundation
Jesse Galef 2 Future of Life Institute, Machine Intelligence Research Institute
Jesse Liptrap 2 Center for Applied Rationality, Machine Intelligence Research Institute
Jia Yuan Loke 2 Anthropic, Effective Altruism Foundation
Joar Skalse 2 Future of Humanity Institute, Oxford University
Johannes Heidecke 2 AI Safety Camp, Road to AI Safety Excellence
John Salvatier 2 AI Impacts, Future of Humanity Institute
Jon Gauthier 2 Massachusetts Institute of Technology, OpenAI
Joseph Halpern 2 Center for Human-Compatible AI, Cornell University
Joshua Fox 2 Association for Long Term Existence and Resilience, Machine Intelligence Research Institute
Joshua Gans 2 National Bureau of Economic Research, University of Toronto
Julia Galef 2 Center for Applied Rationality, OpenAI
Jun Shern Chan 2 Center for AI Safety, Fund for Alignment Research
Justin Shovelain 2 Convergence Analysis, Machine Intelligence Research Institute
Katja Grace 2 Future of Humanity Institute, Machine Intelligence Research Institute
Kenzi Amodei 2 Berkeley Existential Risk Initiative, Center for Applied Rationality
Kristinn R. Thórisson 2 Center for Analysis & Design of Intelligent Agents, Icelandic Institute for Intelligent Machines
Lewis Hammond 2 Cooperative AI Foundation, Future of Humanity Institute
Long Ouyang 2 OpenAI, Theiss Research
Lukas Gloor 2 Effective Altruism Foundation, Foundational Research Institute
Marcello Herreshoff 2 Google, Machine Intelligence Research Institute
Marek Havrda 2 General AI Challenge, GoodAI
Marek Rosa 2 General AI Challenge, GoodAI
Margaret Boden 2 Centre for the Study of Existential Risk, Leverhulme Centre for the Future of Intelligence
Martina Kunz 2 Centre for the Study of Existential Risk, Leverhulme Centre for the Future of Intelligence
Max Daniel 2 Effective Altruism Foundation, Foundational Research Institute
Melody Guan 2 Future of Life Institute, Google Brain
Michael Blume 2 Center for Applied Rationality, Machine Intelligence Research Institute
Michael Cohen 2 Future of Humanity Institute, The Australian National University
Michael Keenan 2 Berkeley Existential Risk Initiative, Center for Applied Rationality
Michael Page 2 Center for Security and Emerging Technology, OpenAI
Michael Wellman 2 Center for Human-Compatible AI, University of Michigan
Mihaly Barasz 2 Machine Intelligence Research Institute, Nilcons
Mrinank Sharma 2 Future of Humanity Institute, University of Oxford
Murray Shanahan 2 Centre for the Study of Existential Risk, Leverhulme Centre for the Future of Intelligence
Neal Jean 2 Future of Humanity Institute, Ought
Olga Afanasjeva 2 General AI Challenge, GoodAI
Owen Cotton-Barratt 2 Centre for Effective Altruism, Redwood Research
Ozzie Gooen 2 Convergence Analysis, Ought
Randall C. O’Reilly 2 eCortex, University of Colorado Boulder
Remco Zwetsloot 2 Center for Security and Emerging Technology, OpenAI
Remmelt Ellen 2 AI Safety Camp, Road to AI Safety Excellence
Reuben Stern 2 Ludwig Maximilian University of Munich, University of Wisconsin–Madison
Richard Ngo 2 OpenAI, University of Cambridge
Robert Miles 2 Nonlinear Fund, Road to AI Safety Excellence
Robert Mushkatblat 2 Lightcone Infrastructure, Machine Intelligence Research Institute
Roger Grosse 2 Future of Humanity Institute, University of Toronto
Rosie Campbell 2 Center for Human-Compatible AI, OpenAI
Roxanne Heston 2 Center for Security and Emerging Technology, Future of Humanity Institute
Sam Clarke 2 Future of Humanity Institute, The Future Society
Sam McCandlish 2 Anthropic, OpenAI
Scott Emmons 2 Center for AI Safety, Fund for Alignment Research
Sergey Rodionov 2 AIDEUS, Aix-Marseille University
Sören Mindermann 2 Center for Human-Compatible AI, Future of Humanity Institute
Stephanie Zolayvar 2 AI Impacts, Center for Applied Rationality
Stephen Hawking 2 Centre for the Study of Existential Risk, Future of Life Institute
Steve Omohundro 2 Machine Intelligence Research Institute, Self-Aware Systems
Steven Umbrello 2 Global Catastrophic Risk Institute, Institute of Ethics and Emerging Technologies
Stuart Armstrong 2 Future of Humanity Institute, Machine Intelligence Research Institute
Tamay Besiroglu 2 Epoch , Future of Humanity Institute
Timothy Telleen-Lawton 2 Anthropic, Center for Applied Rationality
Tobias Baumann 2 Foundational Research Institute, University College London
Tom Everitt 2 Australian National University, Google DeepMind
Tom McGrath 2 Future of Humanity Institute, Ought
Tsvi Benson-Tilsen 2 Center for Applied Rationality, Machine Intelligence Research Institute
Vanessa Kosoy 2 Association for Long Term Existence and Resilience, Machine Intelligence Research Institute
Vincent Conitzer 2 Cooperative AI Foundation, Duke University
Will Millership 2 General AI Challenge, GoodAI
Will Sawin 2 Institute for Theoretical Studies at ETH Zurich, Princeton University
Zac Kenton 2 Montreal Institute for Learning Algorithms, Ought

Positions grouped by organization

Showing 153 organizations.

Organization Number of people List of people
OpenAI 326 Yaniv Markovski, Vishal Kuo, Steven Bills, Daniel Levy, Jessica Shieh, Eugene Wu, Dave Willner, Chester Cho, Adam Nace, Tatiana Zolotova, Sully Chen, Ryan Peterson, Adam Rhodes, Oleg Mürk, Juston Forte, Joanne Jang, Zarina Stanik, Tolly Powell, Chaitra A., Rob Mallery, Austin Wiseman, Preston Tuggle, Adam Goldberg, Matthew Gentzel, Lama Ahmad, Gretchen M. Krueger, Giambattista Parascandolo, Davit Khachatryan, Rajeev Nayak, Sarah Shoker, Carroll Wainwright, Richard Ngo, Anna Makanju, Vitchyr Pong, Victor Benito Garcia Rocha, Elie Georges, Angie Luo, Lukasz Kaiser, Lisa Dethridge, Isabel Alves de Lima, Vlad Ursu, Stefanie Biaggi, Rosie Campbell, Lucas Negritto, Johannes H., Sarthak Agrawal, Kyle Kosic, Karl Whitford Pollard, Jason Kwon, Gualberto Briceño, Emanuele Marchiori, Radhika Mathur, Frances Choi, Tabarak Khan, Natalie Summers, Jonathan Ward, Jesse Han, Jade Leung, Ishant Singh, Hannah Wong, Cullen O"Keefe, Bob Rotsted, Miles Brundage, Lilian Weng, Zack Kass, Steven Adler, Shawn Jain, Che Chang, Jonathan Gordon, Maddie Simens, Julián Santoro, Tyna Eloundou, Bram Adams, Phuong Vu, Philippe Tillet, Mo Bavarian, Fotios Chantzis, Theresa Lopez, Dave Cummings, Joel Lehman, Denny Jin, Joost Huizinga, Fraser Kelton, Raul Puri, Robert B. Brodsky, Red A., Emy Parparita, Kelly Sims, Tim Yanchen Wang, Rachelle F., Arvind Neelakantan, Jeff Clune, Peter Welinder, Christina Hendrickson, Roger Xu Jiang, Aris Konstantinidis, Girish Sastry, Tao Xu, Stanislas Polu, Nikolas Tezak, Mario Saltarelli, Luke Miller, Long Ouyang, Ife Riamah, Diane Yoon, Karson Elmgren, Jerry Tworek, Ilge Akkaya, Alex Paino, Mor Katz, Maxim Sokolov, Jonathan Michaux, Janet Brown, Helen (Mengxin) Ji, Yuhao Wan, Fatma Tarlaci, Elynn Chen, Edgar Barraza, Danny Hernandez, Nancy Otero, Bianca Martin, Ben Chess, Mateusz Litwin, Johannes Otterbach, Sam McCandlish, Qiming Yuan, Janine Korovesis, Clemens Winter, Amanda Askell, Mikhail Pavlov, Lei Zhang, Justin Wang, Jacob Hilton, Jack Clark, Ian Atha, Todor Markov, Christopher Olah, Maddie Hall, Jacob Jackson, Taehoon Kim, Sue Yoon, Michał Staniszewski, Matt Mochary, Jeffrey Wu, Ingmar Kanitscheider, Gillian Hadfield, Christine McLeavey Payne, Brad Lightcap, Arthur Petron, Mira Murati, Karl Cobbe, Josh Meier, Ifu Aniemeka, Holly Grimm, Hannah Davis, Yilun Du, Xingyou (Richard) Song, Will Rice, Ethan Knight, Dolapo Martins, Sophia Arakelyan, Nadja Rhodes, Munashe Shumba, Michael Petrov, Hanjun Dai, Will Grathwohl, Erin Grant, Susan Zhang, Suchir Balaji, Daniel Ziegler, Sadhika Malladi, Aravind Srinivas, Thomas Anthony, Peter Zhokhov, Aleksandar Botev, Manuel Sherbakoff, Eric Sigler, Elena Chatziathanasiadou, Rewon Child, Maran Nelson, Julia Galef, Pieter Abbeel, Kevin Wong, Kaleo Hao, Glenn Powell, Ryan Carey, Parnian Barekatain, Mathew Shrwed, David Farhi, Christy Dennison, Remco Zwetsloot, Ashley C. Pilipiszyn, Adam Smets, David Luan, Maciej Chociej, Jonathan Raiman, Larissa Schiavo, Karthik Narasimhan, Bowen Baker, Phillip Isola, Alex Nichol, Nikhil Mishra, Joshua Achiam, Yuping Luo, Geoffrey Irving, Jiaming Song, Jakob Foerster, Christos Louizos, Cathy Wu, Brooke Chan, AlShaun Baksh, Aditya Grover, Maruan Al-Shedivat, Lerrel Pinto, Kevin Frans, Jason Peng, Han Zhang, Yang Liu, Trapit Bansal, Dustin Tran, Matthias Plappert, David Lansky, Quirin Fischer, Art Chaidarun, Jean Harb, Christopher Berner, Rein Houthooft, Benjamin Mann, Ryan Lowe, Aleks Kamko, Jonathan Gray, Jakub Pachocki, Yaroslav Bulatov, Ankur Handa, Michael Page, Erika Reinhardt, Tim Shi, Danielle Buma, Shariq Hashme, Richard Chen, Bob McGrew, Jeremy Schlatter, Taco Cohen, Szymon Sidor, Meredith Blankenship, Marie La, Louise Cabansay, Josh Tobin, Jonathan Hernandez, Harri Edwards, Desmond Henderson, Rachel Fong, Marika Allely, Ludwig Pettersson, Tambet Matiisen, Craig Quiter, Alexander Skidanov, Alexander Ray, Igor Mordatch, Filip Wolski, Catherine Olsson, Zain Shah, Gavan Woolery, Dario Amodei, Scott Gray, Kate Miltenberger, Jon Gauthier, Tyler Neylon, Rafał Józefowicz, Marcin Andrychowicz, Jie Tang, Prafulla Dhariwal, Paul Christiano, Tim Salimans, Shivon Zilis, Jonas Schneider, Jeff Arnold, Linxi Fan, Jonathan Ho, Yura Burda, Eric Price, Peter Chen, Alec Radford, Ian Goodfellow, Chris Clark, Rocky Duan, Trevor Blackwell, Tom Brown, Andrej Karpathy, John Schulman, Wojciech Zaremba, Elon Musk, Sam Altman, Matt Krisiloff, Vicki Cheung, Durk Kingma, Pamela Vagata, Pavan Sharma, Ilya Sutskever, Holden Karnofsky, Henrique Pondé, Helen Toner, Yuhuai Wu, Yi Wu, Yan Duan, Xue Bin Peng, Greg Brockman, Xi Chen, Vikash Kumar, Elman Mansimov, Tasha McCauley, Tamim Asfour, Daniela Amodei, Smitha Milli, Shun Liao, Shimon Whiteson, Christopher Hesse, Sandy Huang, Bradly Stadie, Reid Hoffman, Aviv Tamar, Przemysław Dębiak, Anish Athalye, Oleg Klimov, Adam D’Angelo, Nicolas Papernot
Machine Intelligence Research Institute 175 James Payor, Edward Kmett, Victoria Krakovna, Carson Jones, Linda Linsefors, Evan Hubinger, David Simmons, Daniel Demski, Ben Weinstein-Raun, Alex Zhu, Alex Mennen, Alex Appel, Andrew Critch, Buck Shlegeris, Nick Tarleton, Blake Borgeson, Kurt Brown, Jesse Liptrap, Sam Eisenstat, Benjamin Mann, Tsvi Benson-Tilsen, Jeremy Schlatter, Jan Leike, Matthew Graves, Ryan Carey, Connor Flexman, Colm Ó Riain, Aaron Silverbook, Malo Bourgon, Gary Drescher, Robin Hanson, Kaya Stechly, Andrew Lapinski-Barker, Stuart Russell, Ramana Kumar, Jed McCaleb, Jack Gallagher, Jaan Tallinn, Bart Selman, Vanessa Kosoy, Stuart Armstrong, Nate Thomas, Abram Demski, Scott Garrabrant, Luke Muehlhauser, Jessica Taylor, Jake Moskowitz, Jesse Galef, Matthew Fallshaw, Nicolas Gagné, Elizabeth Morningstar, Antonius Lourenço Kasbergen, Lila Rieber, Vipul Naik, Nate Soares, Daniel Lewis, Rob Bensinger, Richard Neal, Robert Mushkatblat, Dávid Natingga, Seth Baum, Roman Yampolskiy, Randal Koene, Nathan Clark, Moshe Looks, Kaj Sotala, James Miller, Sebastian Nickel, Evan Erickson, Oliver Habryka, Steve Rayhawk, Paul Christiano, Patrick LaVictoire, Nisan Stiennon, Mihaly Barasz, Joshua Fox, Jeremy Miller, Bill Hibbard, Benya Fallenstein, Anja Heinisch, Alex Altair, Vladimir Nesov, Stephen Barnes, Louie Helm, Ioven Fables, Patrick Robotham, Daniel Roth, Pedro Chaves, Topher Brennan, Liron Shapira, Carl Shulman, Nickolai Leschov, Jonathan Wang, Cameron Taylor, Tomer Kagan, Pejman Makhfi, Kevin Fischer, Jake Miller, Gwern Branwen, Erica Edelman, Alex Vermeer, Robert V. Brazell, Lincoln Quirk, Nevin Freeman, Minda Myers, Keefe Roedersheimer, Diego Caleiro, Will Newsome, Peter Scheyer, Peter de Blanc, Jasen Murray, Thomas Colthurst, Stanislas Sochacki, Luke Grecki, Daniel Dewey, Abraham Wolk, Janos Kramar, Dennis Fan, Ben Hoskin, Ben Goertzel, Tim Czech, Robert Zahra, Jason Levin, Frank Adamek, Aruna Vassar, Amy Willey, Max Tegmark, Henrik Jonsson, Harrison Willey, Edwin Evans, Anna Salamon, Zack M. Davis, Michael Blume, Kemal Eren, Michael Vassar, Andrew Rettek, Thomas McCabe, Steven Kaas, Roko Mijic, Katja Grace, Justin Shovelain, Andriy Brodskyy, Andrew Hay, Vincent Fagot, Bryan Bishop, Alyssa Vance, Peter Cheeseman, Susan Fonseca-Klein, David Hart, C. Colby Thomson, Jonas Lamis, Steve Omohundro, Bruce Klein, Allison Taguchi, Neil Jacobstein, Brian Atkins, Barney Pell, Tyler Emerson, Carolyn L. Burke, Marcello Herreshoff, Peter Thiel, Jeff Medina, Emil Gilliam, Ray Kurzweil, Nick Bostrom, Christine Peterson, Aubrey de Grey, Michael Roy Ames, Jeff Alexander, Michael Wilson, Michael Anissimov, Christian Rovner, Michael Raimondi, Sabine Atkins, Eliezer Yudkowsky
Center for Security and Emerging Technology 171 Brendan Oliss , Jenny Jun, John VerWey, Cole McFaul, Brian Love, Carolina Pachón, Andrea Guerrero, Josh Goldstein, Hanna Dohmen, Katherine Quinn, Christian Schoeberl, Donna Artusy, Ashwin Acharya, Ashton Garriott, Anna Puglisi, Amy Chao, Alexandra Vreeman, Alan Loera, Aditi Joshi, Add Oguike, Zachary Arnold, Tina Huang, Thuy Nguyen, Tarun Chhabra, Tantum Collins, Sue Gordon, Stephanie O"Sullivan, Schuyler Moore, Santiago Mutis, Roxanne Heston, Robert Cardillo, Remco Zwetsloot, Rebecca Kagan, Rafay Ur Rehman Khan, Olivia Albrighton-Vanway, Mina Narayanan, Michael Sulmeyer, Michael Page, Melissa Flagg, Max Langenkamp, Matthew Daniels, Margarita Konaev, Lorand Laskai, Kate Collins, John Bansemer, Jeff Ding, Jason Matheny, Jacob Strieb, Jack Lucas, Ilya Rahkovsky, Eri Phinisee, Emelia Probasco, Emefa Addo Agawu, Elsa Kania, Darrin Gladman, Daniel Cebul, Dalila Scott, Dakota Foster, Dakota Cary, Collins Nji, Claire Perkins, Cindy Martinez, Christopher Back, Christine McNeill, Carrick Flynn, Ben Buchanan, Beba Cibralic, Avonelle Davis, Aurora Johnson, Ronnie Kinoshita, Nii Simmonds, Kevin Wolf, Heather Frase, Walter Haydock, Shuvo Bardhan, Jessica Ji, Owen Daniels, Laissa A., Ella Kay, Caroline Schuerger, Sara Abdulla, Lisa Oguike, Luke Koslosky, Jack Corrigan, Channing Lee, Heeu Millie Kim, Adrienne Thompson, Kayla Goode, Faeez J., Eish Sumra, Arsh S., Ali Crawford, Abelardo Cruz Osorio, Shelton Fitch, Maya Gros, Ingrid Dickinson, Alex Friedland, Alan Omar Loera Martinez, Melissa Deng, Mary Hill Brooks, Lizbeth Lucero, J. Guillermo Mendoza Bazán, Filippo Fagnoni, Cinzia Luna, Will Hunt, Piyush Mishra, Oneeb Ul Haq Khan, Gustavo Mauricio Bastien Olvera, Sean Kucer, George Klein, Emma Caviness, Diana Gehlhaus Carew, Darius Diamond, Andrew Lohn, Andreas Greiler-Basaldúa, Alex Barker, Simon Godfrey Rodriguez, Raveena Kshatriya, Luwei Lei, Jennifer Keltz, Jasmine Ding, Christina Ismailos, Chris Rohlf, Bryce Farabaugh, Zuleirys Santana-Rodriguez, Katerina Sedova, Farid Nemri, Austin Dahmer, Jacob Feldgoise, Wyatt Hoffman, Rebecca Gelles, Emily Weinstein, Autumn Toney, Ngor Luong, Yiming Y., Ines P., Nicolina Demakos, Emily Xue, Ziling Wang, Reginald Brothers, Wenchuan Dong, Charlie Wang, Jack Clark, Daniel Hague, Igor Mikolic-Torreira, Catherine Aiken, Alexander M., Jonathan Murdick, Jennifer Melot, Huey-Meei Chang, Andrew Imbrie, Tim G. J. Rudner, Saif M. Khan, Saif Khan, Ryan Fedasiuk, Husanjot Chahal, Dahlia Peterson, Ben Murphy, Lynne Weil, Jeffrey Ding, Jamie Baker, Daniel Chou, Benjamin Chang, Ziyi Wang, William Hannas, James Dunham, Peggy Evans, Tim Hwang, Tessa Baker, Helen Toner, Dewey Murdick
Center for Human-Compatible AI 82 Rafael Albert, Tom Lenaerts, Pulkit Verma, Yuxi Liu, Lawrence Chan, Jessy Lin, Dan Hendryks, Cassidy Laidlaw, Alyssa Li Dayan, Alex Gunning, Stephen Casper, Sören Mindermann, Sergei Volodin, Pedro Freire, Neel Alex, Matthew Rahtz, Johannes Treutlein, Harry Giles, Eric Michaud, Cynthia Chen, Charlotte Roman, Rediet Abebe, Nika Haghtalab, Moritz Hardt, Jakob Foerster, Brian Christian, Noor Brody, Rachel Freedman, Meir Friedenberg, Shlomi Hod, Jess Reidel, Alex Turner, Micah Carroll, David Lindner, Thomas Krendl Gilbert, Sam Toyer, Michael Dennis, Vincent Corruble, Jacob Steinhardt, Steven Wang, Karthika Mohan, Martin Fukui, Caroline Jeanmaire, Cody Wild, Rohin Shah, Niko Kolodny, Brandon Perry, Vael Gates, Adam Gleave, Michael Littman, Smitha Milli, Monica Gates, Daniel Filan, Jaime Fernandez Fisac, Dylan Hadfield-Menell, Marion Fourcade, Lara Buchak, Ken Goldberg, Juliana Schroeder, John Zysman, Demian Pouzo, Dawn Song, Charis Thompson, Alison Gopnik, Rosie Campbell, Mariano Florentino Cuéllar, Siddharth Srivastava, Gillian Hadfield, Tania Lombrozo, Christopher Cundy, Beth Barnes, Dmitrii Krasheninnikov, Andrew Critch, Mark Nitzberg, Satinder Singh Baveja, Michael Wellman, Joseph Halpern, Bart Selman, Tom Griffiths, Anca Dragan, Pieter Abbeel, Stuart Russell
Future of Humanity Institute 71 Patrick Butlin, Elise Bohan, Matthew van der Merwe, Janvi Ahuja, Peter Wills, Isaac Friend, Maria Violaris, Joar Skalse, Hannah Klim, Tushant Jha, Sam Clarke, Mrinank Sharma, Karolina Milewicz, Duncan Snidal, Michael Osborne, Jacob Lagerros, Michael Cohen, Jan Brauner, Thomas Orton, Roger Grosse, Lewis Hammond, Carla Zoe Cremer, Michael Montague, David Manheim, Ondrej Bajgar, Gregory Lewis, Ryan Carey, Michael Bonsall, Helen Toner, Tom McGrath, Robin Hanson, Paul Christiano, Jan Leike, Ben Garfinkel, Sören Mindermann, William Saunders, Neal Jean, Girish Sastry, Tamay Besiroglu, David Kristoffersson, Piers Millett, John Salvatier, David Krueger, David Abel, Allan Dafoe, Carrick Flynn, Andrew Snyder-Beattie, Daniel Dewey, Toby Ord, Anders Sandberg, Vincent C. Müller, Stuart Armstrong, Sebastian Farquhar, Seán Ó hÉigeartaigh, Roxanne Heston, Owain Evans, Niel Bowerman, Nick Bostrom, Miles Brundage, Kyle Scott, Katja Grace, Jeffrey Ding, Jade Leung, Eric Drexler, Daniel Filan, Clare Lyle, Christopher Cundy, Chelsea Guo, Cecilia Tilli, Carl Shulman, Baobao Zhang
Future of Life Institute 58 Daniel Dewey, Jesse Galef, Zara Yaqoob, Yishuai Du, Xin Wen, Will Jones, Victoria Krakovna, Vera Koroleva, Tucker Davey, Taylor Jones, Stuart Russell, Stephen Hawking, Saul Perlmutter, Risto Uuk, Richard Mallah, Rafael Martinez-Galarza, Peter Haas, Nick Bostrom, Na Li, Morgan Freeman, Melody Guan, Meia Chita-Tegmark, Max Tegmark, Max Kesin, Martin Rees, Mark Brakel, Lucas Perry, Kazue Evans, Janos Kramar, Jacob Trefethen, Jacob Beebe, Jaan Tallinn, Gus Docker, Grzegorz Orwiński, George Church, Frank Wilczek, Francesca Rossi, Erik Brynjolfsson, Eric Gastfriend, Emilia Javorsky, Elon Musk, David Stanley, Daniel R. Miller, Claudia Prettner, Christof Koch, Chase Moores, Carlos Ignacio Gutierrez, Blake Pierson, Ariel Conn, Anthony Aguirre, Anna Yelizarova, Anna Hehir, Andrea Berman, Ales Flidr, Alan Yan, Alan Guth, Alan Alda, Akhil Deo
Effective Altruism Foundation 40 Jia Yuan Loke, Daniel Kokotajlo, Jesse Clifton, Hadrien Pouget, Eric Chen, Emery Cooper, Anthony DiGiovanni, Ali Merali, Paul Knott, Mojmír Stehlík, Michael Aird, Julian Stastny, Alexander Lyzhov, Maxime Riché, Amrit Sidhu-Brar, Linh Chi Nguyen, David Pearce, Daniel Rüthemann, Adrian Hutter, Ulla Wessels, Olle Häggström, Ole Martin Moen, Lucius Caviola, Dina Pomeranz, Brian Tomasik, Anni Leskelä, Stefan Torges, Stefan Klein, Sascha Fink, Sarah Dörpinghaus, Rajshri Jayaraman, Melinda Lohmann, Lukas Gloor, Klaus Wälde, Johannes Treutlein, David Althaus, Thomas Metzinger, Persis Eskander, Ozy Brennan, Max Daniel
Center for Applied Rationality 39 Kyle Scott, Kathryn Schmiedicke, Xavier Prospero, Brienne Strohl, Leah Bookish, Luke Raskopf, Adam Scholl, Adom Hartell, Tsvi Benson-Tilsen, Timothy Telleen-Lawton, Stephanie Zolayvar, Qiaochu Yuan, Matthew Graves, Jordan Tyrrell, Eric Rogstad, Ben Goldhaber, Elizabeth Garrett, Eli Tyre, Cat Lavigne, Michael Blume, Jesse Liptrap, Oliver Habryka, Michael Keenan, Lauren Lee, Davis Kingsley, Jack Carroll, Ben Sancetta, Pete Michaud, Eric Chisholm, Peter McIntyre, Dan Keys, Michael Smith, Andrew Critch, Leah Libresco, Kenzi Amodei, Julia Galef, Gwern Branwen, Duncan Sabien, Anna Salamon
Leverhulme Centre for the Future of Intelligence 37 Kay Firth-Butterfield, Neil Lawrence, Alan Winfield, Michael A. Osborne, Lucy Cheke, Heather Roff, Beth Singler, José Hernández-Orallo, Tameem Adel, Kanta Dihal, Karina Vold, Henry Shevlin, Rune Nyrup, Sarah Dillon, Susan Gowans, Andrew Snyder-Beattie, Adrian Weller, Thomas D. Grant, Seán Ó hÉigeartaigh, David Runciman, Stephen John, Marta Halina, Anna Alexandrova, Murray Shanahan, Stuart Russell, Nick Bostrom, Stephen Cave, Zoubin Ghahramani, Huw Price, Demis Hassabis, Manuela M. Veloso, Francesca Rossi, Martin Rees, Philip Pettit, Alison Gopnik, Margaret Boden, Martina Kunz
Global Catastrophic Risk Institute 35 Dakota Norris, Allan Suresh, Uliana Certan, Kyle L. Evanoff, McKenna Fitzgerald, Oliver Couttolenc, Andrea Owe, Jared Brown, Lena Wang, Jenny Mith, Matthijs Maas, Jessica Cianci, Trevor White, Gary Ackerman, Roman Yampolskiy, Caroline Zaw-Mon, Dave Denkenberger, Robert de Neufville, Arden Rowell, Jianhua Xu, U. Tuncay Alparslan, Steven Umbrello, Jacob Haqq-Misra, Mark Fusco, Kaitlin Butler, Grant Wilson, Tim Maher, Matt Moretto, Kelly Hostetler, Tony Barrett, Seth Baum, Adam Scholl, Marilyn Cotrich, Seán Ó hÉigeartaigh, John Garrick
GoodAI 34 Karolína H., Ryan Camilleri, Jose Solorzano, Alex Angelini, Sarka Krejcova, Stephanie Wendler, Reham Bukhari, Viktorie Knezkova, Steffen Eichler, Petr Šimánek, Isabeau Premont-Schwarz, Petr Šrámek, Michal Dvořák, Christine Lee, Will Millership, Lucie Krestova, Marek Havrda, Jan Feyereisl, Olga Afanasjeva, Martin Poliak, Marek Rosa, Simon Andersson, Přemek Paška, Jaroslav Vitku, Wendelin Boehmer, Šimon Šicko, Shantesh Patil, Petr Hlubuček, Nicholas Guttenberg, Lucia Šicková, Joseph Davidson, Jan Štafa, Filip Hauptfleisch, Dominik Čech
FLOWERS 31 Alvaro Ovalle Castaneda, Florian Golemo, Benjamin Clément, Sébastien Forestier, William Schueller, Cem Karaoguz, Adrien Matricon, Céline Craye, Alexandra Delmas, Gennaro Raiola, Baptiste Busch, Panagiotis Papadakis, Thibaut Munzer, Guillaume Duceux, Louis-Charles Caron, Damien Caselli, Théo Segonds, Stéphanie Noirpoudre, Loïc Dauphin, Matthieu Lapeyre, Nicolas Rabault, Yoan Mollard, Pierre Rouanet, Nicolas Jahier, Didier Roy, Anna-Lisa Vollmer, Freek Stulp, Alexandre Gepperth, David Filliat, Manuel Lopes, Pierre-Yves Oudeyer
Anthropic 28 Avital Balwit, Neel Nanda, Jackson Kernion, Jia Yuan Loke, Liane Lovitt, Timothy Telleen-Lawton, Yuntao Bai, Anna Chen, Amanda Askell, Dario Amodei, Benjamin Mann, Tom Brown, Nicholas Joseph, Jack Clark, Daniela Amodei, Zac Hatfield-Dodds, Tom Henighan, Tom Conerly, Sam McCandlish, Nova DasSarma, Nelson Elhage, Kamal Ndousse, Jared Kaplan, Dawn Drain, Danny Hernandez, Chris Olah, Catherine Olsson, Andy Jones
Berkeley Existential Risk Initiative 26 Elizabeth Cooper, James Paul Gonzales, Jess Riedel, Andrew Critch, Jan Tallinn, Stuart Russell, Sawyer Bernath, Sofia Davis-Fogel, Alex Flint, Josh Jacobson, Sam Bankman-Fried, Matt Fallshaw, Colleen Gleason, Jeremy Schlatter, Jaan Tallinn, Kyle Scott, Rebecca Raible, Kenzi Amodei, Jacob Tsimerman, Qiaochu Yuan, Eric Rogstad, Seán Ó hÉigeartaigh, Malo Bourgon, Andrew Snyder-Beattie, Michael Keenan, Gina Stuessy
Ought 26 Ian McKenzie, Luke Stebbing, Justin Reppert, Aisha Aishwarya, Eli Lifland, Amanda Ngo, Aparna Ashok, Jungwon Byun, Chris Cundy, Girish Sastry, Neal Jean, Ryan Carey, Zac Kenton, Tom McGrath, Ben Weinstein-Raun, Ozzie Gooen, Milan Griffes, Zachary Miller, Ben Goldhaber, Ben West, Ben Rachbach, Noah Goodman, Paul Christiano, Owain Evans, Andreas Stuhlmüller, Andrew Schreiber
Centre for the Study of Existential Risk 25 Adrian Weller, Sean Holden, Stephen Hawking, Tim Crane, Max Tegmark, Murray Shanahan, Dana Scott, Stuart Russell, Elon Musk, Alison Gopnik, David Chalmers, Nick Bostrom, Margaret Boden, Ryan Carey, Martina Kunz, Seth Baum, Beth Barnes, Yang Liu, Jaan Tallinn, Martin Rees, Huw Price, Haydn Belfield, Shahar Avin, Seán Ó hÉigeartaigh, Simon Beard
General AI Challenge 24 Will Millership, Olga Afanasjeva, Virginia Dignum, Tomas Mikolov, Tak Lo, Ryota Kanai, Roman Yampolskiy, Rodolfo Rosini, Pavel Kordik, Miles Brundage, Marek Rosa, Marek Havrda, Ling Ge, Julian Togelius, José Hernández-Orallo, Jan Sekerka, Jan Romportl, Jan Pospíšil, Ivan Zelinka, Irakli Beridze, Frank Dignum, Danit Gal, Ayako Fukui, Alison Lowndes
Center for AI Safety 21 Corin Katzke, Max Kaufmann, Sidney Hough, Scott Emmons, Michael Chen, Mantas Mazeika, Kevin Liu, Jun Shern Chan, Joshua Clymer, Dan Hendrycks, Andy Zou, Nathaniel Li, Anders Edson, Alex Pan, Ronit Singh, Marc Carauleanu, Madhav Malhotra, Steven Basart, Rune Kvist, Oliver Zhang, Thomas Woodside
Conjecture 21 Maris Sala, Daniel Braun, Beren Millidge, Jonathan Low, Janko Prester, Carlos Guevara, Caelum Forder, Lee Sharkey, Katrina Joslin, Andrea Miotti, Rachel Stockton, Adam Shimi, Myriame Honnay, Gabriel Alfour, Sid Black, Chris Scammell, Jacob Merizian, Kip Parker, Laria Reynolds, Kyle Mcdonell, Connor Leahy
Google DeepMind 21 Vishal Maini, Sean Legassick, Verity Harding, Miljan Martic, Pedro A. Ortega, Shane Legg, Mustafa Suleyman, Demis Hassabis, Jeffrey D. Sachs, James Manyika, Christiana Figueres, Edward W. Felten, Diane Coyle, Nick Bostrom, Victoria Krakovna, Laurent Orseau, Chris Maddison, Thore Graepel, Andrew Lefrancq, Tom Everitt, Jan Leike
AI Impacts 19 Jeffrey Heninger, Aysja Johnson, Zach Stein-Perlman, Harlan Stewart, Ronny Fernandez, Daniel Kokotajlo, Asya Bergal, Ronja Lutz, Richard Korzekwa, Tegan McCaslin, Paul Christiano, Jimmy Rintjema, Ben Hoffman, Justis Mills, Connor Flexman, Finan Adamson, Michael Wulfsohn, John Salvatier, Stephanie Zolayvar
Redwood Research 18 Tao Lin, Ben Weinstein-Raun, Seraphina Nix, Paul Christiano, Owen Cotton-Barratt, Noa Nabeshima, Malo Bourgon, Holden Karnofsky, James Bregan, Daniel Ziegler, Claire Zabel, Blake Borgeson, Bill Zito, Ajeya Cotra, Adam Scherlis, Royston Noronha, Buck Shlegeris, Nathaniel Thomas
Stanford University 17 Siddharth Karamcheti, Peter Henderson, Pratyusha Kalluri, Dorsa Sadigh, Stanislav Fort, Cody Coleman, Stefano Ermon, Michael Webb, Percy Liang, Alex Tamkin, Noah D. Goodman, Andreas Stuhlmüller, Aditi Raghunathan, Thomas Icard, Ray Briggs, Alex Aiken, Jacob Steinhardt
Epoch 16 Keith Wynroe, Jenny Xiao, David Atkinson, Daniela Amodei, Ben Cottier, Matthew Barnett, Ege Erdil, Tom Davidson, Tamay Besiroglu, Pablo Villalobos, Neil Thompson, Marius Hobbhahn, Lennart Heim, Jaime Sevilla, Eduardo Infante-Roldán, Anson Ho
University of California, Berkeley 15 Dan Hendrycks, Max Simchowitz, Andrew Critch, Qiaochu Yuan, Frances Ding, Lydia T. Liu, Michael Janner, Tom Kalil, Stuart Russell, Pieter Abbeel, Smitha Milli, Dylan Hadfield-Menell, Anca Dragan, Sergey Levine, Paul Christiano
Lightcone Infrastructure 13 Robert Mushkatblat, Rafe Kennedy, Jacob Lagerros, Ruben Bloom, Raymond Arnold, Kaj Sotala, Matthew Graves, Harmanas Chopra, Eric Rogstad, Ben Albert Pace, Oliver Habryka, James Babcock, Elizabeth Van Nostrand
Nonlinear Fund 12 Daniel del Castillo, Corey Wood, Chris Leong, Peter Barnett, Tristan Cook, David Moss, Robert Miles, Alex Zhu, Spencer Greenberg, Jonas Vollmer, Emerson Spartz, Kat Woods
University of Oxford 10 Charlie Rogers-Smith, Mrinank Sharma, Allan Dafoe, Ruth Fong, Chris Maddison, Heather Roff, Owain Evans, Aidan Gomez, Nick Bostrom, Michael Wooldridge
Cooperative AI Foundation 9 Vincent Conitzer, Lewis Hammond, Ruairi Donnelly, Gillian Hadfield, Eric Horvitz, Dario Amodei, Allan Dafoe, Jesse Clifton, Amrit Sidhu-Brar
Fund for Alignment Research 9 Nora Belrose, Tomasz Korbak, Scott Emmons, Jun Shern Chan, Jérémy Scheurer, Ian McKenzie, Ethan Perez, Claudia Shi, Adam Gleave
Whole Brain Architecture Initiative 9 Koji Morikawa, Hideyuki Nakashima, Hiroyuki Morikawa, Masaru Tomita, Kitano Hiroaki, Kenji Doya, Koichi Takahashi, Yutaka Matsuo, Hiroshi Yamakawa
Road to AI Safety Excellence 8 Remmelt Ellen, Trent Fowler, Erik Istre, Rupert McCallum, Robert Miles, Johannes Heidecke, Veerle de Goederen, Toon Alfrink
7 Joe Collman, Angela P., Anand Srinivasan, Orpheus Lummis, Stag Lynn, Alexander Gietelink Oldenziel, Tegan McCaslin
AI Safety Camp 7 Remmelt Ellen, Jessica Cooper, Kristina Nemcova, Jirí Nadvorník, Anne Wissemann, Jan Kulveit, Johannes Heidecke
Australian National University 7 Jan Leike, Marcus Hutter, Jarryd Martin, Gary Lea, Alan Hájek, Elliot Catt, Tom Everitt
Theiss Research 7 Karina Torres Castro, Rebecca Bone, Rodrigo Duran, Jack Glover, Soraya Bernal, Paul Christiano, Long Ouyang
University of Cambridge 7 Richard Ngo, Adrian Weller, Jose Hernandez-Orallo, Arif Ahmed, Huw Price, Yang Liu, Ramana Kumar
Foundational Research Institute 6 Brian Tomasik, Max Daniel, Caspar Oesterheld, Lukas Gloor, Tobias Baumann, Kaj Sotala
Google Brain 6 Jeremy Nixon, Melody Guan, Tom Brown, Dan Mané, Dario Amodei, Christopher Olah
Association for Long Term Existence and Resilience 5 Vanessa Kosoy, Joshua Fox, Gidon Kadosh, Edo Arad, David Manheim
Carnegie Mellon University 5 Leqi Liu, Noam Brown, David Danks, Manuela Veloso, Andre Platzer
Convergence Analysis 5 Ozzie Gooen, Claire Abu-Assal, Kristian Rönn, Andrew X Stewart, Justin Shovelain
EthicsNet 4 Aleksandra Orchowska, Remco Bloemen, Anish Mohammed, Nell Watson
Massachusetts Institute of Technology 4 Joshua Brett Tenenbaum, Julius Adebayo, Andrew Ilyas, Jon Gauthier
Montreal Institute for Learning Algorithms 4 Zac Kenton, Doina Precup, Joelle Pineau, Yoshua Bengio
Open Philanthropy 4 Paul Christiano, Jacob Steinhardt, Dario Amodei, Christopher Olah
Center for AI Policy 3 Jason Green-Lowe, Thomas Larsen, Jakub Kraus
Centre for Effective Altruism 3 Johannes Treutlein, Ales Flidr, Owen Cotton-Barratt
Cornell University 3 Bart Selman, Joseph Halpern, Jim Babcock
eCortex 3 Randall C. O’Reilly, Seth J. Herd, David J. Jilk
Foresight Institute 3 Christine Peterson, Mark S. Miller, Allison Duettmann
IDSIA 3 Bas R. Steunebrink, Mark Ring, Jürgen Schmidhuber
SUPSI 3 Mark Ring, Jürgen Schmidhuber, Bas R. Steunebrink
Università della Svizzera italiana 3 Mark Ring, Jürgen Schmidhuber, Bas R. Steunebrink
University of Toronto 3 Dami Choi, Roger Grosse, Joshua Gans
Yale University 3 Wendell Wallach, Daniel Eth, Allan Dafoe
AIDEUS 2 Sergey Rodionov, Alexey Potapov
Arizona State University 2 Heather Roff, Miles Brundage
Center for a New American Security 2 Gregory C. Allen, Paul Scharre
Center for Human Success 2 Wyatt Tessari, David Yu
Encultured AI 2 Andrew Critch, Nick Hay
Endgame 2 Hyrum Anderson, Bobby Filar
Google 2 Vladimir Slepnev, Marcello Herreshoff
Learning Intelligent Distribution Agent 2 Tamas Madl, Stan Franklin
Linköping University 2 Mikael Böörs, Tobias Wängberg
London School of Economics 2 Katie Steele, Wlodek Rabinowicz
Ludwig Maximilian University of Munich 2 Stephan Hartmann, Reuben Stern
Oregon State University 2 Alex Turner, Thomas Dietterich
UCLA School of Law 2 Richard Re, Edward Parson
University College London 2 John Shawe-Taylor, Tobias Baumann
University of Michigan 2 Michael Wellman, James M. Joyce
University of Wisconsin–Madison 2 Reuben Stern, Patrick LaVictoire
Aalto University 1 Jelena Luketina
AgroParisTech 1 Laurent Orseau
AI Challenge 1 David Denkenberger
Aix-Marseille University 1 Sergey Rodionov
American University 1 Thomas Zeitzoff
Bar-Ilan University 1 Ram Rachum
Birkbeck, University of London 1 Ulrike Hahn
Broad Institute of MIT and Harvard 1 Gopal Sarma
Brown University 1 David Abel
California Institute of Technology 1 Frederick Eberhardt
Carleton University 1 Andrew MacFie
Center for Analysis & Design of Intelligent Agents 1 Kristinn R. Thórisson
CogPrime 1 Ben Goertzel
Czech Technical University 1 Vojtěch Kovařík
Data61 1 Ramana Kumar
Duke University 1 Vincent Conitzer
Electronic Frontier Foundation 1 Peter Eckersley
ETH Zurich 1 Felix Berkenkamp
George Mason University 1 Robin Hanson
Georgia Institute of Technology 1 Fuxin Li
Global Politics of Artificial Intelligence Research Group at Yale University and University of Oxford 1 Matthijs Maas
Hague Centre for Strategic Studies 1 Matthijs Maas
Harvard University 1 David Parkes
Icelandic Institute for Intelligent Machines 1 Kristinn R. Thórisson
Information Society Project 1 Rebecca Crootof
INRA 1 Laurent Orseau
Institute for Future Studies 1 H. Orri Stefánsson
Institute for Theoretical Studies at ETH Zurich 1 Will Sawin
Institute of Ethics and Emerging Technologies 1 Steven Umbrello
Internet Archive 1 Brewster Kahle
ITMO University 1 Alexey Potapov
Legal Priorities Project 1 Nick Hollman
Lingnan University 1 Jiji Zhang
Moscow Institute of Physics and Technology 1 Vladimir Shakirov
Munich Center for Mathematical Philosophy 1 Catrin Campbell-Moore
Nanyang Technological University 1 Preston Greene
NARS 1 Pei Wang
National Bureau of Economic Research 1 Joshua Gans
New America Foundation 1 Heather Roff
New York University 1 Ethan Perez
Nilcons 1 Mihaly Barasz
NNAISENSE 1 Bas R. Steunebrink
Oxford University 1 Joar Skalse
Phenomenological AI Safety Research Institute 1 G Gordon Worley III
Princeton University 1 Will Sawin
Quebec Artificial Intelligence Institute 1 Vincent Luczkow
Quixey 1 Patrick LaVictoire
Real AI 1 Jonathan Yan
Rice University 1 Moshe Vardi
Self-Aware Systems 1 Steve Omohundro
Smith College 1 James Miller
Social & Environmental Entrepreneurs 1 Seth Baum
Sorbonne University 1 Michaël Trazzi
Susaro 1 Richard Loosemore
Teesside University 1 The Anh Han
Texas A&M University 1 Kenny Easwaran
The Australian National University 1 Michael Cohen
The Consortium on the Landscape of AI Safety 1 Alexis Carlier
The Future Society 1 Sam Clarke
The New School 1 Peter Asaro
Ulm University 1 ‪Daniel Alexander Braun
Université de Montréal 1 Jelena Luketina
University of Alberta 1 Tor Lattimore
University of Amsterdam 1 Dmitrii Krasheninnikov
University of Arizona 1 Jenann Ismael
University of Bath 1 Joanna Bryson
University of Bristol 1 Benya Fallenstein
University of Colorado 1 Seth Herd
University of Colorado Boulder 1 Randall C. O’Reilly
University of Copenhagen 1 Matthijs Maas
University of Edinburgh 1 Angelo Frank De Bellis
University of Illinois at Chicago 1 Brian Ziebart
University of Louisville 1 Roman Yampolskiy
University of Melbourne 1 Benjamin Rubinstein
University of Montreal 1 Janos Kramar
University of New Hampshire 1 Andrew Ware
University of Padova 1 Francesca Rossi
University of Southern California 1 Stephen J. Read
University of Texas 1 Peter Stone
University of Washington 1 Daniel Weld
Washington University in St. Louis 1 Julia Haas

Individuals not affiliated with any organization

Showing 18 people.

Organization Website Source
Wei Dai http://www.weidai.com/ [1], [2]
Iceman [3], [4], [5], [6], [7]
Max Harms http://raelifin.com/ [8], [7]
Jeff Kaufman https://www.jefftk.com [9], [7]
Federico Pistono http://federicopistono.org/ [10]
Chris Pasek [11], [12]
Sune Kristian Jakobsen
Hilary Greaves [13]
Sophie-Charlotte Fischer [14]
Alexey Turchin https://avturchin.livejournal.com/ [15], [16], [17], [18]
Dustin Juliano http://dustinjuliano.com/ [19], [20]
Matteo Turchetta [21]
Angela P. Schoellig [21]
Andreas Krause [21]
Jim O’Neill [22]
Gordon Irlam http://www.gordoni.com/ [23]
Nandi Schoots [12]
John Maxwell [24]

Products

This section lists AI safety-related “products”: interactive tools, websites, flowcharts, datasets, etc. Unlike documents, products tend to be interactive, are updated continually, or require inputs from the consumer.

Showing 33 products.

Name Type Creator Creation date Description
Clarifying some key hypotheses in AI alignment diagram Ben Cottier, Rohin Shah 2019-08-15 A diagram collecting several hypotheses in AI alignment and their relationships to existing research agendas.
AI Alignment Forum blog LessWrong 2.0 2018-07-10 A group blog for discussion of technical aspects of AI alignment. The forum is built using the same software as LessWrong 2.0, and is integrated with LessWrong 2.0. For creation date, see [25].
AI Safety Research Camp workshop Tom McGrath, Remmelt Ellen, Linda Linsefors, Nandi Schoots, David Kristoffersson, Chris Pasek 2018-02-01 A research camp to take place in Gran Canaria in April 2018 and in the United Kingdom in July–August 2018. Facebook group at [26]. The creation date is the date of announcement on LessWrong 2.0.
“Levels of defense” in AI safety flowchart Alexey Turchin 2017-12-12 A flowchart applying multilevel defense to AI safety. There is an accompanying post on LessWrong at [27].
AI Alignment Prize contest Zvi Mowshowitz, Vladimir Slepnev, Paul Christiano 2017-11-03 A prize for work that advances understanding in alignment of smarter-than-human artificial intelligence. Winners for the first round, as well as announcement of the second round, can be found at [24]. Winners for the second round, as well as announcement of the third round, can be found at [28].
AI Watch interactive application Issa Rice 2017-10-23 A website to track people and organizations working on AI safety.
AI Safety Open Discussion discussion group Mati Roy 2017-10-23 A Facebook discussion group about AI safety. This is an open group.
AI safety resources list Victoria Krakovna 2017-10-01 A list of resources for long-term AI safety. Seems to have been first announced at [29].
Map of the AI Safety Community graphic Søren Elverlin 2017-09-26 A pictorial map that lists organizations and individuals in the AI safety community.
Open Philanthropy Project AI Fellows Program fellowship Open Philanthropy 2017-09-12 A fellowship to support PhD students in AI and machine learning. For the creation date, see [30].
LessWrong 2.0 blog LessWrong 2.0 2017-06-18 A community blog about rationality, decision theory, AI, the rationality community, and other topics relevant to AI safety. This is a re-launch/modernization of the original LessWrong. For the launch date, the date of the welcome post [31] is used.
Road to AI Safety Excellence course Toon Alfrink 2017-06-15 A proposed course that is designed to produce AI safety researchers. It used to be called “Accelerating AI Safety Adoption in Academia” and was announced on LessWrong at [32]. The Facebook group was created on 2017-06-30 [33].
Annotated bibliography of recommended materials list Center for Human-Compatible AI 2016-12-01 An annotated and interactive bibliography of AI safety-related course materials, textbooks, videos, papers, etc.
Extinction Risk from Artificial Intelligence blog Michael Cohen 2016-06-01 A series of pages exploring arguments for and against working on AI safety. The creation date is inferred from the URLs of images (example: [34]).
AI Alignment blog Paul Christiano 2016-05-28 Paul Christiano’s blog about AI alignment.
AISafety.com Reading Group discussion group Søren Elverlin, Erik B. Jacobsen, Volkan Erdogan 2016-05-24 A weekly reading group covering topics in AI safety.
Cause prioritization app interactive application Michael Dickens, Buck Shlegeris 2016-05-18 An interactive app for quantitative cause prioritization. The app includes a section [35] on AI safety intervention. The creation date is the date of the first commit in the Git repository [36].
Arbital AI alignment domain wiki Arbital, Eliezer Yudkowsky 2016-03-04 A collection of wiki-like pages on topics in AI alignment. The creation date is the date of the launch announcement for Arbital [37]; it’s unclear when the AI alignment domain itself was created.
Introductory resources on AI safety research list Victoria Krakovna 2016-02-28 A list of readings on long-term AI safety. Mirrored at [38]. There is an updated list at [39].
AI Safety Discussion discussion group Victoria Krakovna 2016-02-21 A Facebook discussion group about AI safety. This is a closed group so one needs to request access to see posts.
Reinforce.js implementation of Stuart Armstrong’s toy control problem interactive application Gwern Branwen, FeepingCreature 2016-02-03 A live demo of Stuart Armstrong’s toy control problem [40]. gwern introduced the demo in a LessWrong comment [41].
AI Policies Wiki wiki Gordon Irlam 2015-12-14 A wiki on AI policy. The wiki creation date can be seen in the revision history of the main page [42].
The Control Problem discussion group CyberPersona 2015-08-29 A subreddit about AI safety and control. For the subreddit creation date, see [43].
AGI Failures Modes and Levels map flowchart Alexey Turchin 2015-01-01 A flowchart about failure modes of artificial general intelligence, grouped by the stage of development. There is an accompanying post on LessWrong at [18].
AGI Safety Solutions Map flowchart Alexey Turchin 2015-01-01 A flowchart on potential solutions to AI safety. There is an accompanying post on LessWrong at [44].
Intelligent Agent Foundations Forum discussion group Machine Intelligence Research Institute 2014-11-04 A forum for technical AI safety research. The source code is hosted on GitHub [45]. The timestamp on the introductory post [46] gives the launch date.
A flowchart of AI safety considerations flowchart Eliezer Yudkowsky 2014-11-02 The flowchart was posted to Eliezer Yudkowsky’s Essays (a Facebook group) and has no title.
Effective Altruism Forum blog Centre for Effective Altruism, Rethink Charity, Ryan Carey 2014-09-10 A community blog about effective altruism which often has posts about AI safety. The forum was announced on LessWrong by Ryan Carey [47].
How to study superintelligence strategy list Luke Muehlhauser 2014-07-03 A list of project ideas in superintelligence strategy.
Ordinary Ideas blog Paul Christiano 2011-12-21 Paul Christiano’s blog about “weird AI stuff” [48].
The Uncertain Future interactive application Machine Intelligence Research Institute 2009-10-01 A tool to model future technology and its effect on civilization. For more about the history of the site, see [49].
LessWrong Wiki wiki Machine Intelligence Research Institute 2009-03-12 A companion wiki to the community blog LessWrong. The wiki has pages about AI safety.
LessWrong blog Machine Intelligence Research Institute 2009-02-01 A community blog about rationality, decision theory, AI, updates to MIRI, among other topics.