Germany's AI Safety Institute

Published: September 20, 2026
URL: https://toranm.me/post/2026-09-20-germanys-aisi/

Germany founded its own AI Safety Institute (AISI) called AISI Deutschland. It is still in the very early stages of being operational, but a good initiative.Current Status and Political Background#The current German government is very unpopular. They tried to do major reforms in the labour market and social security system, but faced major backlash. In order to regain some credibility, the government tries to act quickly and headline-worthy but very much chaotic behind the scenes (AV-Depot, Tankrabatt, …). I see the founding of the AISI as one of these chaotic actions but it is one step in the right direction. More commitment from the government is needed to make it worthwhile.Timeline#The AISI will be founded in two stages [1]. It was founded by a decision of the national security council [2]. In the first stage, that we are currently in, the AISI will be only a brand name and consist of a collaboration between the existing BSI and BNetzA. BSI is Germany’s one-stop-shop for all things infosec, BNetzA is the federal regulatory agency for network-based economies (energy, post, telecommunication, …). BNetzA also is involved in AI topics regarding the EU AI act. Who exactly the people working on AISI are, is not known, there is not even a publicly known president. In the second phase, AISI will be an actual self-contained organization, with dedicated offices and employees. Not much detail about this phase have been provided. Plans for the budget allocations, tasks and capabilities defined by law are not known at the moment. It will be based in Berlin.Current work#AISI publishes a newsletter on AI-Safety topics [1]. As a first step, the AISI will focus on cybersecurity risks arising from AI models [3].What is needed to make it worthwhile#AISI needs access to the international AI labs. To get this access there are several options. Either the labs cooperate voluntarily. That seems only realistic for some labs [4], others might not be willing to cooperate or be forced to not cooperate by their jurisdictions. AISI needs to be granted special privileges within the foreign jurisdictions. The German government needs to use its diplomatic power to achieve an international treaty on AISIs. AISIs need to be protected in foreign jurisdictions, but they also need to be run responsibly. They must not be used for economic espionage, be neutral towards the system they are evaluating, have the capabilities to be critical and must be independent of political influence. For this, international binding guidelines need to be developed with mechanisms for peer-review, arbitration and joint-analysis. Democratic countries can have a deeper level of cooperation, but all countries need to be included. Dario’s proposal is a very good reference.Focus the mission#Please people in charge, do not dilute the AISI’s mission with other political goals like regional development or some stuck-up public administration bureaucracy. Focus the AISI only on the mission, no side-quests, no cheaping out. Hell, even allow it to be outside of procurement law and collective bargaining law.Why this matters#AI risk is existential for humanity. We are already seeing misaligned agents. Agents of the past incidents are just a little too stupid, but with the pace that AIs get better, this won’t be for long.[1]Bundesamt für Sicherheit in der Informationstechnik, “AISI - Deutschland,” 2026. Accessed: Sep. 19, 2026. [Online]. Available: https://www.bsi.bund.de/DE/Themen/Unternehmen-und-Organisationen/Informationen-und-Empfehlungen/AISI/aisi_node.html[2]Bundesregierung, “Eröffnung des KI-Sicherheitsinstituts,” 2026. Accessed: Sep. 19, 2026. [Online]. Available: https://www.bundesregierung.de/breg-de/aktuelles/eroeffnung-ki-sicherheitsinstitut-2449410[3]BMDS, “AISI Deutschland gegründet,” 2026. Accessed: Sep. 19, 2026. [Online]. Available: https://bmds.bund.de/aktuelles/pressemitteilungen/detail/aisi-deutschland-gegruendet[4]D. Amodei, “We Must Pace the Frontier,” 2025. Accessed: Sep. 19, 2026. [Online]. Available: https://darioamodei.com/post/we-must-pace-the-frontier