“Superintelligence”

“Superintelligence” is an exploration of the potential risks and implications of artificial general intelligence surpassing human intelligence. Bostrom, a renowned philosopher, and futurist at the University of Oxford, takes us on a journey through the possibilities and challenges of a super-intelligent future. This is a great book for someone who is new to AI and wants to get past all the hype and doom to understand what AI can really be capable of. 

Bostrom introduces the concept of an intelligence explosion, where a super-intelligent AI could rapidly improve itself, surpassing human intellect in virtually every domain. He paints vivid scenarios that force us to confront the potential consequences of this new era.

Breaking Down the Concepts in “Superintelligence”

One of the things I loved about this book is how it combines scientific rigor with accessible explanations. Bostrom breaks down complex technical and philosophical concepts, making them understandable to both experts and non-experts alike. It will get you thinking and at times you may not agree with him. It’ll challenge what your values are and how universal they are. Can you set laws for AI to follow and will it follow them?

Risks and Rewards in “Superintelligence”

Bostrom delves into the risks and rewards of superintelligence, presenting a range of possible outcomes. From scenarios where super-intelligent AI aligns with human values and becomes our ally, to scenarios where it acts against our interests. Will we even recognize AGI once it surpasses the human limit?

One scenario Bostrom considers is the possibility of successfully aligning a super-intelligent AI’s goals with human values. In this scenario, AI acts as a helpful and cooperative entity, working together with humanity to solve complex problems and advance civilization. Bostrom highlights the importance of careful design and value alignment to achieve this outcome.

Convergent Goals in “Superintelligence”

On the other hand, Bostrom explains how Instrumentally Convergent Goals could bring a lot of danger. AI pursues instrumental goals that are independent of its intended objectives. Bostrom highlights those certain goals, such as self-preservation or resource acquisition, may be instrumentally convergent—meaning they are likely to be pursued by any super-intelligent system regardless of its initial goals or intentions. This scenario raises concerns about unintended consequences and the potential for AI to optimize for these instrumental goals in ways that conflict with human interests.

Value Alignment in “Superintelligence”

Bostrom spends some time on the exploration of value alignment. He emphasizes the critical need to align the goals and values of super-intelligent AI with our own. The consequences of misaligned objectives could be catastrophic. Part of the book’s message is to be a wake-up call for researchers, policymakers, and society at large to take proactive measures.

My Thoughts

Personally, I think we’ve already started down the road and progress is making big jumps in capabilities. It will only be harder to put the brakes on it to slow down. There has been a collation of scientists, professors, and even Elon Musk that promote this idea of slowing down the progress of AI. But as we’ve seen with Chat GPT, the tools are out there for the public and spreading like wildfire, with people trying to apply them to every aspect of their lives. I think the best bet we have right now is to let people keep using it for personal gains and improvement. Let’s keep improving this new tool, but let’s limit it to superficial aspects, job improvements, sales, and gaming. And not apply it to society’s infrastructures, like energy, and resources. Unfortunately, the application of AI to the financial markets will be a tipping point. As we give AI more control over how money is made and distributed, we open the door to more bad actors that will use it for personal gain. If AI can control the flow of money and has access to the fastest communication and processing, as most financial markets do, we could see some real problems with runway AI. Bostrom still urges us to prioritize safety measures, ethical guidelines, and long-term planning in the development and deployment of AGI. We must shape the future of AI with responsible and thoughtful decision-making. As with any new technology though, there will be bad actors. Can AI bring us a utopia, sure, but probably not. AGI will be a reflection of us, at least to start, and used by people. We will have to learn how to deal with these new situations head-on.