The chaos inside OpenAI – Sam Altman, Elon Musk, and existential risk explained | Karen Hao

• OpenAI was founded in 2015 as a non-profit research organization aimed at developing AGI for the benefit of humanity, in response to the rise of AI developments by those companies. This is why Elon Musk joined and donated $10M.
• AGI means AI reaching and overtaking human capabilities in every area of life, capable of improving itself autonomously and recursively. In the extreme (hyperintelligence), it's a God. It can know and do everything.
• OpenAI aimed to mitigate these existential risks facing humanity: AI safety and ethics must be secured, and all discoveries should follow open standards!

Co-founder Ilya Sutskever, a successful scientist, has groundbreaking papers published as a researcher. He's well aware of the threats of the coming AI wave and wants to ensure every improvement is safe and useful for humanity.

Ilya's vision was that creating AGI is possible by scaling the tech stack (with more data and larger infrastructure), but this requires a substantial amount of money for hardware and to attract talent.

Consequently, securing investment became a necessity.

This was where non-profit to for-profit transformation happened (2019) and Sam Altman joins as CEO. They altered the corporate structure.

Although they shifted towards a Silicon Valley startup model, they still maintained their strict rules to prevent possible misleading and drift from the initial objectives. Measures were implemented to avert investor involvement in management and to limit investor profits. Additionally, founders were not allowed to hold any equity in the company.

Sam Altman is an expert in Silicon Valley startup culture, which emphasizes moving fast and growing infinitely.

Co-founder Greg Brockman, sharing an entrepreneurial mindset and background with Sam Altman, is also a hustler. That's the reason for their deep mutual solidarity.

Sam Altman secured significant funding from Microsoft, but his entrepreneurial approach began to pose risks and inconsistencies with OpenAI's foundational principles.

On November 17th 2023, the board fired Sam for straying from their main altruistic mission. Greg immediately resigned in support of him. This was followed by the resignation of three senior employees.

OpenAI was successful to bring skilled people into the company by emphasizing altruistic reasons — working for the benefit of humanity. 

Employees care about humankind but also their state of mind. They became distraught after the chaos began, as they never received any explanation about what was happening, leading to significant anger and a feeling of being left in the dark.

The other thing they care is their pockets. Their financial expectations were slipping into dangerous territory. Nobody wanted to see the company's skyrocketing valuation take a downturn due to a fiasco.

Simultaneously, Satya Nadella (CEO of Microsoft), offered a guaranteed position to all OpenAI staff at Microsoft. It was a reasonable move because Microsoft put billions in OpenAI and didn't want to lose it all of a sudden.

Over 700 out of 770 employees signed a letter that threatens to quit unless Sam Altman was reinstated.

Without employees there is no company. This made the board reverse their decision and despairingly call Sam Altman back to his position.

This is the end of drama.

Do you recall the part where I mentioned key people implementing safety measures to protect their lofty goals by isolating the company from external influences like superpower investors, various political situations, and even the founders themselves?

When AI governance is concerned, self-regulation does not work!

OpenAI has faced similar challenges in the past. In early 2018, a disagreement between Sam Altman and Elon Musk led to Musk completely withdrawing his support and exiting the company.

Furthermore, a dispute over the commercialization of GPT-3 prompted the departure of senior scientist Dario Amodei, who took along a significant portion of the team to found Anthropic, now a major competitor to OpenAI.

Companies like Meta are open-sourcing (rather than commercialize) their AI models, claiming to democratize AI. However, they never share the data used to train these models. This attitude creates a significant barrier for other researchers who wish to analyze, understand and, shepherd these AI models.

AI safety does not only mean as extreme as preventing existential risks or national security threats. When you think of AI safety, remember that it also relates to avoiding discrimination and bias in any form. Ethics and misinformation are critical components that require proactive management to mitigate AI's potential adverse effects on society.

I dropped out of my PhD study while working on my thesis in AI. Those were the days AI was not a buzzword and my SVM models were outperforming NN models. I quit to start my own business, immersing myself in both academic and startup cultures. Wearing both hats in the past has been invaluable.

From this perspective, I believe AI needs to be managed with great caution. Because it's a huge challenge (in fact impossible) to predict the complete outcomes of an autonomous and self-improving technology.

AI community is broadly divided into two ideological camps:

1. Techno-optimists: Most entrepreneurs & business people support unbounded development of AI. They have a commercial vision. Move fast, get more money in, quickly build and release products for public use.
2. Existential-risk Advocates: Most scientists & researchers support AI containment. AI safety is the utmost priority for them. Move cautiously, make sure it's completely tested against potential risks and do not release products until they're resistant to misuse and misbehave.

We'll see who prevail and what the consequences will be. Soon.