How will artificial intelligence (AI) shape our lives? Who is developing AI, and what motivates them? What flaws currently exist in AI systems, and how might they be improved or exploited? Quantitative futurist Amy Webb explores these questions in her 2019 book The Big Nine: How the Tech Titans and Their Thinking Machines Could Warp Humanity, which ProSPER members read for our Spring 2022 book club.
Book summary
Webb begins with a history of computers and AI. She emphasizes how the early developers of AI were mostly white men performing research at a small number of American and Canadian universities. Starting in 2006, several major research advances revolutionized AI, and within 6 years, tech companies were using these new AI algorithms in commercial products. Power concentrated to a small number of companies such as Google, Microsoft, Amazon, Facebook, IBM, and Apple, which Webb collectively refers to as the G-MAFIA. At the same time, the Chinese government rapidly invested in AI research, education, and tech companies Baidu, Alibaba, and Tencent, or the BAT. Together, the G-MAFIA and the BAT form the Big Nine. Webb claims that the G-MAFIA is primarily driven by profit margins and thus are pressured to consistently deliver new products and features; meanwhile, the BAT is primarily driven by its partnership with the Chinese Communist Party, which has a long-term strategic plan to make AI integral to all parts of society. Webb uses the contrast between the G-MAFIA and the BAT to establish the main premise of her book: that AI is shaped by a small number of powers with two different sets of priorities and values.
Webb goes on to argue that on its current trajectory, AI will not help all of humanity. Currently, AI automates specific tasks—like setting airfare prices or detecting credit fraud—but eventually they will be capable of performing medical diagnosis, writing books and music, and streamlining crop growth and production. One advance that will make this possible is the development of personal data records (PDRs): universal records incorporating our purchases, interests, locations, social interactions, grades, health, and legal records. Proto-PDRs already exist in the form of email addresses, which we use to log into the Big Nine’s apps and gadgets. In China, PDRs are also being developed to build social credit scores, which uses someone’s behavior to control their ability to access public goods and services. However, AI currently emphasizes automation and efficiency, not human values and emotion, so it is not clear how PDRs will shape society as they are adopted. Using predictions from strategic forecasting, Webb describes three potential scenarios for how AI might develop over the next 50 years:
- In the “optimistic” scenario, the US creates a national strategy for AI—encouraging the G-MAFIA to make their AIs interoperable and publicly accountable—and forms the Global Alliance on Intelligence Augmentation (GAIA). AI is used to build “smart cities” and safely relocate populations displaced by climate change; eventually, China’s economy is destabilized, forcing it to give up its authoritarian agenda in order to join GAIA. As AI nears the ability to become smarter than humans, GAIA and the Big Nine implement strategies to limit the rate at which AI can self-improve so humans are not harmed.
- In the “pragmatic” scenario, the G-MAFIA emphasizes speed over safety and competes with each other for government funding. Digital ecosystems do not interoperate, making it difficult to access services (such as healthcare, finance, or travel) that are not in the same ecosystem as your PDR. Meanwhile, China colonizes Southern Africa and Asia, becomes economically independent from the West, and enhances their military with human-augmenting brain-machine interfaces. The G-MAFIA does not protect its AIs from attack, which China exploits to hack Western devices and digitally conquer the US.
- In the “catastrophic” scenario, the G-MAFIA builds its own form of a social credit score, leading to a digital caste system in the US based on punishment and reward. The US government and military do not form long-term strategic partnerships with the G-MAFIA, allowing China to colonize Latin America and become the major AI superpower, locking the West out of the Chinese AI ecosystem. American healthcare becomes reliant on injectable, AI-controlled nanorobots that extend our lifespan, leading to overpopulation as climate change causes land and resources to grow scarce. China addresses this overpopulation problem by building an AI that exploits vulnerabilities in nanorobots to exterminate the population of the US and its allies.
Group discussion
While the group agreed that AI is revolutionizing society and is currently not poised to help all of humanity, we felt that Webb was trying to push a specific worldview that caused her three scenarios to boil down to a cold war between the US and China. Webb seemed extremely forgiving to the G-MAFIA and their capitalistic motives, and overly critical of the BAT. While there are valid concerns surrounding how China (and by association, the BAT) handles privacy, security, and choice, many of these concerns still exist in US under the optimistic scenario. For example, Webb proposed that individuals, not companies, can own PDRs. Individuals can revoke a company’s access at any time, but Webb suggested this would not be common because we learn to trust the Big Nine. The group thought a society where virtually everyone turns over all their personal information to the Big Nine resembles China’s social credit system—which Webb previously argued was not a good thing—and that “plac[ing] our unwavering affection and trust” in the Big Nine was a dystopian outlook. Webb also argued against government regulations because AI changes too quickly, but she also compared AI to public goods such as roads and clean air that are themselves commonly regulated. The group thought that these contradictions suggest that Webb’s goal in writing this book is not to discuss the good and bad ways AI can shape humanity, as the title suggests, but instead to argue that the US should curb China and become the world leader of AI.
Webb spent a lot of time in the beginning of her book discussing how current AI developers lack diversity and how this introduces implicit and explicit bias in AI systems. We agreed that increasing the diversity of the AI workforce is important and will make AI better for more of humanity, but we felt Webb undermined this argument throughout her book. For example, in the optimistic scenario Webb envisioned a global alliance led by the US, Canada, Western Europe, and Japan to create policy frameworks that leads AI in the direction that will best benefit humanity—notably absent from the leadership was the Global South or any non-Western country. Furthermore, Webb expresses shocking nativist sentiments by suggesting American universities could put pressure on the Chinese Communist Party by banning Chinese students—in the “optimistic” scenario. These points caused the group to question who Webb had in mind when she called for increased diversity.
One of the reasons Webb argued for diversity is so AI can build consensus across many perspectives. AI can already come up with game strategies humans have not thought of before, but it is hard to understand how AI reaches these decisions. While the group agreed with Webb’s argument that the utility of AI should be judged based on its ability to contribute to society, we questioned whether AI ought to be trusted to make new decisions in high-stakes situations involving millions of lives. Individuals in the group noted that a majority opinion does not necessarily make an idea ethically correct, and it was not clear that trusting an AI to resolve moral ambiguities would be beneficial.
Although Webb outlined possible ways AI can be improved in the future, we felt that these improvements would not address some of the underlying problems in society. For example, Webb argued that mergers and acquisitions among the G-MAFIA in the pragmatic and catastrophic scenarios will create a lack of choice. In the optimistic scenario, Webb paints collaboration between the G-MAFIA and government as a way to create more choice. The group did not feel that a small number of companies would create more choice simply because we trust them with all of our information. This weakened Webb’s attempt to contrast the optimistic scenario with the other two. Furthermore, Webb argued that technologies in the optimistic scenario would replace jobs traditionally performed by humans, but that those people would be re-trained for new jobs that do not exist today. The group felt that this concept would only perpetuate inequality that already exists; some individuals proposed an alternative argument where new technologies allow humans to work less and have more time for leisure.
It was unclear what audience Webb had in mind for her book. The group felt that the language was intended for a relatively broad audience, but many of Webb’s calls to action were not practical for the everyday person. She urged readers to stop pressuring the G-MAFIA for higher quarterly profits so they have the time to develop quality AI, but that kind of pressure comes from big investors and hedge funds. Thus, it was unclear what everyday people can do to affect change towards one of Webb’s scenarios, or something else entirely.
Although we generally disagreed with Webb’s worldview, we had many fruitful discussions about the general utility of AI, the way it influences our society, the meaning of intelligence and consciousness, and human nature. For example, one reason AI can beat humans in games like chess and Go is because humans get frustrated, tired, and emotional, while AIs do not have emotions. Webb argued that AI should incorporate and account for a diverse set of human values and emotions, which made us wonder whether AIs could develop emotions or if they will remain emotionless. If AI remains emotionless, is removing human emotions from decision-making a good thing? If AI develops emotions, should we trust its emotions over human emotions? And if AI can write music, safely resettle refugees, and meaningfully contribute to society, what makes it different from a human? Would the AI be conscious? These questions are highly philosophical and difficult to answer concretely, but we noted that they could be explored with dystopian fiction stories set in Webb’s three difference scenarios. This would have the added benefit of exploring potential future technologies driven by AI and their impact on society, which may help Webb—and AI creators—think about scenarios that do not end in a digital cold war between the US and China.