Entities & Funding

Updated February 8, 2024

 

There are a lot of people working on AI safety these days, which is a good thing because this is a really big challenge. Some of these entities offer funding for AI safety work. The information on this page is weighted towards the UK (where we are based) and US (which remains the AI technology leader).

 

Government Entities

The US and UK announced the founding of national AI safety institutes at the beginning of November 2023 in connection with the first AI Safety Summit:

• U.S. AI Safety Institute (USAISI) - as preliminary steps towards the USAISI, the US Department of Commerce in February 2024 announced:
  • the leadership of the institute
  • the U.S. AI Safety Institute Consortium comprising over 200 entities that will contribute technical expertise to the work of USAISI
• UK AI Safety Institute.

 

The Global Partnership on AI (GPAI) is a multinational intiative established by the G7 in 2020, now with 29 member countries and operating from OECD offices in Paris. The AI Safety Summit process may affect the role of the GPAI.

 

National regulatory bodies specifically responsible for AI are also starting to emerge, for example:

• China - The Cyberspace Adminstration of China has responsibility for AI legislation (among much broader responsibilities).

• EU - The EU AI Act will establish an AI Office within the European Commission and an AI Board comprising representatives of EU member states.

• US - The Executive Order on AI established a White House Artificial Intelligence Council within the Executive Office of the President.

 

Private Entities

There are many private-sector and university efforts. A few are:

• AI Alliance - a large alliance of companies (with lead roles for IBM and Meta), universities and others "collaborating to advance safe, responsible AI rooted in open innovation"

• Center for AI Safety
• Frontier Model Forum - established by Anthropic, Google, Microsoft, and OpenAI 
• Future of Life Institute, AI Existential Safety Community

• International Association of Algorithmic Auditors - "a community of practice that aims to advance and organize the algorithmic auditing profession, promote AI auditing standards, certify best practices and contribute to the emergence of Responsible AI"

• London Initiative for Safe AI
• Partnership on AI - a charitable organization with a mission of "bringing diverse voices together across global sectors, disciplines, and demographics so developments in AI advance positive outcomes for people and society"
• Safe AI London.

 

AI Safety Fundamentals have published a longer list of local groups, and the aisafety.community site provides a much longer list of "communities working on AI existential safety".

 

There are also a large number of start-ups working on AI safety issues. The Ethical AI Governance Group maintains the Ethical AI Database project (EAIDB), described as "a curated collection of startups that are either actively trying to solve problems that AI and data have created or are building methods to unite AI and society in a safe and responsible manner" -- see EAIDB's latest market map (July 2023).

 

Saihub.info is building our own list of entities addressing AI safety. This currently contains far fewer start-ups than the EAIDB market map. We will expand it over time.

 

Funding

Funding for safe and responsible AI initiatives is emerging fairly quickly, and we expect this to continue. A few UK and US funding sources are:

• Frontier Model Forum - AI Safety Fund
• Responsible AI UK - including Keystone Projects
• UK Advanced Research and Innovation Agency (ARIA) - including an ambitious initiative for designing provably safe AI

• US Defense Advanced Research Projects Agency (DARPA) - including AI Forward program.