Yoshua Bengio on the catastrophic risk from uncontrolled AI agency

On Thursday, I had the opportunity to attend the AI lecture „Avoiding Catastrophic Risk from Uncontrolled AI Agency“ at the Bavarian Academy of Sciences together with Gunnar Stöcker. The talk was held by Yoshua Bengio, one of the leading voices in AI, and it was both impressive and eye-opening.

Beyond his compelling call for a realistic assessment of the risks emerging from current AI developments, two points from his somewhat dystopian call to action stuck with me:

• Most companies seem to be out of the race for frontier AI. For me, this highlights the danger of European markets becoming dependent on American AI solutions. Yoshua identified two core risks here: First, most efficiency gains will benefit foreign companies, and second, these firms may outcompete local businesses due to access to proprietary frontier AI.
• The second risk is even more troubling: beyond the loss of human control, we see an excessive concentration of economic, political, and military power—something already becoming visible today.

It was a truly eye-opening evening. Thanks to Baiosphere for the excellent organization!