counter create hit Human Compatible: Artificial Intelligence and the Problem of Control - Download Free eBook
Ads Banner
Hot Best Seller

Human Compatible: Artificial Intelligence and the Problem of Control

Availability: Ready to download

A leading artificial intelligence researcher lays out a new approach to AI that will enable us to coexist successfully with increasingly intelligent machines In the popular imagination, superhuman artificial intelligence is an approaching tidal wave that threatens not just jobs and human relationships, but civilization itself. Conflict between humans and machines is see A leading artificial intelligence researcher lays out a new approach to AI that will enable us to coexist successfully with increasingly intelligent machines In the popular imagination, superhuman artificial intelligence is an approaching tidal wave that threatens not just jobs and human relationships, but civilization itself. Conflict between humans and machines is seen as inevitable and its outcome all too predictable. In this groundbreaking book, distinguished AI researcher Stuart Russell argues that this scenario can be avoided, but only if we rethink AI from the ground up. Russell begins by exploring the idea of intelligence in humans and in machines. He describes the near-term benefits we can expect, from intelligent personal assistants to vastly accelerated scientific research, and outlines the AI breakthroughs that still have to happen before we reach superhuman AI. He also spells out the ways humans are already finding to misuse AI, from lethal autonomous weapons to viral sabotage. If the predicted breakthroughs occur and superhuman AI emerges, we will have created entities far more powerful than ourselves. How can we ensure they never, ever, have power over us? Russell suggests that we can rebuild AI on a new foundation, according to which machines are designed to be inherently uncertain about the human preferences they are required to satisfy. Such machines would be humble, altruistic, and committed to pursue our objectives, not theirs. This new foundation would allow us to create machines that are provably deferential and provably beneficial. In a 2014 editorial co-authored with Stephen Hawking, Russell wrote, "Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last." Solving the problem of control over AI is not just possible; it is the key that unlocks a future of unlimited promise.


Compare
Ads Banner

A leading artificial intelligence researcher lays out a new approach to AI that will enable us to coexist successfully with increasingly intelligent machines In the popular imagination, superhuman artificial intelligence is an approaching tidal wave that threatens not just jobs and human relationships, but civilization itself. Conflict between humans and machines is see A leading artificial intelligence researcher lays out a new approach to AI that will enable us to coexist successfully with increasingly intelligent machines In the popular imagination, superhuman artificial intelligence is an approaching tidal wave that threatens not just jobs and human relationships, but civilization itself. Conflict between humans and machines is seen as inevitable and its outcome all too predictable. In this groundbreaking book, distinguished AI researcher Stuart Russell argues that this scenario can be avoided, but only if we rethink AI from the ground up. Russell begins by exploring the idea of intelligence in humans and in machines. He describes the near-term benefits we can expect, from intelligent personal assistants to vastly accelerated scientific research, and outlines the AI breakthroughs that still have to happen before we reach superhuman AI. He also spells out the ways humans are already finding to misuse AI, from lethal autonomous weapons to viral sabotage. If the predicted breakthroughs occur and superhuman AI emerges, we will have created entities far more powerful than ourselves. How can we ensure they never, ever, have power over us? Russell suggests that we can rebuild AI on a new foundation, according to which machines are designed to be inherently uncertain about the human preferences they are required to satisfy. Such machines would be humble, altruistic, and committed to pursue our objectives, not theirs. This new foundation would allow us to create machines that are provably deferential and provably beneficial. In a 2014 editorial co-authored with Stephen Hawking, Russell wrote, "Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last." Solving the problem of control over AI is not just possible; it is the key that unlocks a future of unlimited promise.

30 review for Human Compatible: Artificial Intelligence and the Problem of Control

  1. 5 out of 5

    Manny

    Let's start with the most important thing: if you have any interest in finding out where technology is heading, please read this book. I particularly recommend that people who know something about moral philosophy do so. You may dislike Human Compatible, you may object to the way the author treats your subject, but you really ought to learn about what's happening here. Moral philosophy has become shockingly relevant to the near-term future of humanity. I'll back up a little. Since the beginning o Let's start with the most important thing: if you have any interest in finding out where technology is heading, please read this book. I particularly recommend that people who know something about moral philosophy do so. You may dislike Human Compatible, you may object to the way the author treats your subject, but you really ought to learn about what's happening here. Moral philosophy has become shockingly relevant to the near-term future of humanity. I'll back up a little. Since the beginning of the twenty-first century, the idea that machines may soon be smarter than humans has gone from science-fictiony scare talk to a sensible projection where the disagreements are not about if, but when. Some experts are saying fifty years from now, some are saying twenty or thirty, some ten or even five. But there's general consensus that we're talking decades, not centuries; this is something that many people alive now will probably see. Since machines evolve much more quickly than humans, once they've overtaken us they will rapidly leave us far behind. Unless we find some other way to destroy ourselves first, we're soon going to be sharing our planet with non-human beings who are vastly more intelligent and capable than we are. As Russell says, it's surprising that we aren't more concerned. If we were told that a fleet of super-advanced aliens was on its way towards Earth and would be landing in thirty to fifty years, we'd be running around in small circles hyperventilating. Well: we are proposing to build those aliens and install them in every home, and many of us are still not taking it seriously. But more and more people are. Bostrom's widely read Superintelligence was the point where the idea went mainstream, and it was soon followed by Tegmark's Life 3.0 , du Sautoy's The Creativity Code and other books. Human Compatible is the latest installment. In contrast to the other authors (Bostrom is a philosopher, Tegmark a physicist and du Sautoy a mathematician), Russell is a leading expert on AI. He is coauthor of the world's most widely read AI textbook, teaches at Berkeley, and is connected to pretty much everyone in the business. If he doesn't know what he's talking about, no one does, and he is very concerned. He doesn't try to scare you or sell you apocalyptic visions of the impending Singularity; in fact, he goes to some lengths to downplay the more sensational claims. He just says very calmly that this is something that's going to happen, so we should prepare as well as we can. If possible, he would like us to slow down the pace of progress a bit, so that we could have a better chance of seeing where we're headed. But he's right in the middle of the Silicon Valley madness, and he knows that's not going to happen: the value of real, general-purpose AI is measured in the trillions of dollars. All the big players are frantically competing to get there first. What can we do? Well, he's very smart, and he's thought about it carefully, and he has an idea he's put a lot of work into. I'm not sure I believe it, but it's better than anything else I've seen. Following the preceding books in this thread, he considers what will happen when we have superintelligent AIs. We won't be able to control them in any normal sense of the word; our only realistic chance is to build them so that their goals are aligned with ours, in other words so that they want what we want. But we're only going to get one shot at this, since once they've been built we probably won't be able to switch them off or change them. Unfortunately, experience with technology suggests that nothing works the way it's meant to first time, and we don't even have a clear notion of what we want these godlike machines to be able to do. So, by a process of elimination, we're left with one alternative. We decide to be upfront about the fact that we don't know what we're trying to achieve, and we directly build that into our architecture. As Russell says, over the last thirty years the concept of uncertainty has come to pervade the whole field of AI, with one exception: we always say we know what the software is supposed to achieve. But why? In fact, it's more logical to say we're uncertain about that too. Just as a speech recogniser uses a noisy audio signal to try and work out what was probably said, and a self-driving car uses a noisy video signal to work out where the truck probably is, one of Russell's new generation machines will examine all our noisy preference signals - verbal, physical, financial, whatever - to work out what we probably want, and try to respond to it. As new information comes in, it will update its picture accordingly. The technical name of this idea is "Inverse Reinforcement Learning", IRL. I am, to say the least, conflicted about IRL. Intellectually, it is fascinating, not least because it puts theoretical philosophical ideas center stage and turns them into practical engineering issues. As Russell says, what we're doing is building a machine whose top-level operating principle is some version of consequentialist utilitarianism. There are so many interesting questions. Where is the machine going to get its preference signals from? Will it consider all kinds of signals equally? (When the robowaiter tries to decide whether to bring you dessert, it will weigh up competing factors: you can't take your eyes off your neighbor's chocolate mousse, your cholesterol is slightly high, you said "no" but you hesitated, and you sometimes like to be surprised). Will preference signals from all people be weighed equally? (If your robot only cares about your preferences, then it would have no reason not to kill or steal if it calculates that will be to your advantage; but why would anyone buy a robot which is likely to go off to Somalia to help people who need it more than they do?) Above all, how do we know that IRL will work reliably? Remember that we only get one shot at a solution. A large part of the attraction is that IRL is a mathematical algorithm, so you can in principle apply mathematical methods to prove that it does what it's supposed to. It works for simple examples, and it is indeed comforting to be shown a toy scenario where a simulated IRL robot decides that it should let its owner switch it off because the risk it will do something bad is larger than the upside of being around to help. But will this technology scale to dealing with billions of people, all with their own agendas? Russell says he's optimistic it will, but what else can he say? And there are other fundamental problems. Can we be sure that people's preferences really mean anything? The book gives examples of how easy it is for machines to manipulate people. Russell calmly tells us it's more or less certain that social media has inadvertently caused the resurgence of fascism by feeding users data which makes their political opinions more extreme, so that they are easier to predict and have a higher click-through rate. That kind of phenomenon seems to offer numerous possibilities for an IRL machine to end up doing things which it might formally count as satisfying people's preferences, but which from our present perspective seem highly undesirable. Damn... how did we get into a situation where our whole existence could hinge on quickly resolving tricky philosophical problems that may not even have solutions? All I can say is, if you do happen to be one of those rare people who's received formal training in moral philosophy and knows something about it, please consider volunteering for frontline service. The world needs you more than you know.

  2. 5 out of 5

    Jen

    Are you interested in Artificial Intelligence and the existential issues it heralds? Then this book is for you. If you, at some point in your travels, got so high on Jamaican hash that you experienced what might reasonably be called a psychotic break, causing you to collapse in supplication before our robot overlords, years in advance of their arrival, this will help you get up and get on with your life. This book, alongside Superintelligence by Nick Bostrom, is, what I would consider, an essent Are you interested in Artificial Intelligence and the existential issues it heralds? Then this book is for you. If you, at some point in your travels, got so high on Jamaican hash that you experienced what might reasonably be called a psychotic break, causing you to collapse in supplication before our robot overlords, years in advance of their arrival, this will help you get up and get on with your life. This book, alongside Superintelligence by Nick Bostrom, is, what I would consider, an essential read for anyone with an interest in AGI and our transformation into paperclips. While Bostrom's book approaches the subject from the point of philosophy, this one does so on a more technical level, yet remains perfectly comprehensible to a disciplined layperson. You will not walk away from this book feeling like your time would've been better spent on speculative science fiction. You will learn a lot about the nuts and bolts of the field. And what's more, this is perhaps the first book I've read which offers an actionable solution to the alignment problem. Whether it's ultimately possible, I leave to those more capable than myself. The author of this book certainly qualifies. So buck up, brothers and sisters. The bad trip can be ameliorated with enough careful study. Listen to my voice. You're coming down now. Sober up with this book!

  3. 4 out of 5

    Michael Perkins

    Recommended to me by A.I. expert and Goodreads super star, Manny Rayner. a nice Frontline overview of AI to get you warmed up. https://www.youtube.com/watch?v=5dZ_l... =========== “He who controls the algorithms controls the universe.” To get just an inkling of the fire we’re playing with, consider how content-selection algorithms function on social media. They aren’t particularly intelligent, but they are in a position to affect the entire world because they directly influence billions of people. T Recommended to me by A.I. expert and Goodreads super star, Manny Rayner. a nice Frontline overview of AI to get you warmed up. https://www.youtube.com/watch?v=5dZ_l... =========== “He who controls the algorithms controls the universe.” To get just an inkling of the fire we’re playing with, consider how content-selection algorithms function on social media. They aren’t particularly intelligent, but they are in a position to affect the entire world because they directly influence billions of people. Typically, such algorithms are designed to maximize click-through, that is, the probability that the user clicks on presented items. The solution is simply to present items that the user likes to click on, right? Wrong. The solution is to change the user’s preferences so that they become more predictable. A more predictable user can be fed items that they are likely to click on, thereby generating more revenue. People with more extreme political views tend to be more predictable in which items they will click on. In 1997, when IBM's Deep Blue computer defeated world chess champion, Gary Kasparov, some of the smartest people forecast it would be 2097 before a computer would beat a human in the more complex game of GO. In 2016 and 2017, DeepMind’s AlphaGo defeated Lee Sedol, former world Go champion, and Ke Jie, the current champion. It shows as that the development of A.I. is moving forward a lot faster than expected and that rules-based human jobs will be the first to disappear the more A.I. is implemented. ============ In 195o, Alan Turing published a paper, “Computing Machinery and Intelligence ”that proposed an operational test for intelligence, called the imitation game. The test measures the behavior of the machine— specifically, its ability to fool a human interrogator into thinking that it is human....Contrary to common interpretations, I doubt that the test was intended as a true definition of intelligence, in the sense that a machine is intelligent if and only if it passes the Turing test. Indeed, Turing wrote, “May not machines carry out something which ought to be described as thinking but which is very different from what a man does?” Another reason not to view the test as a definition for AI is that it’s a terrible definition to work with. And for that reason, mainstream AI researchers have expended almost no effort to pass the Turing test. The Turing test is not useful for AI because it’s an informal and highly contingent definition: it depends on the enormously complicated and largely unknown characteristics of the human mind, which derive from both biology and culture. There is no way to “unpack” the definition and work back from it to create machines that will provably pass the test.... The way we build intelligent agents depends on the nature of the problem we face. This, in turn, depends on three things: first, the nature of the environment the agent will operate in— a chessboard is a very different place from a crowded freeway or a mobile phone; second, the observations and actions that connect the agent to the environment— for example, Siri might or might not have access to the phone’s camera so that it can see; and third, the agent’s objective— teaching the opponent to play better chess is a very different task from winning the game.' General-purpose AI would be a method that is applicable across all problem types and works effectively for large and difficult instances while making very few assumptions. That’s the ultimate goal of AI research: a system that needs no problem-specific engineering and can simply be asked to teach a molecular biology class or run a government. It would learn what it needs to learn from all the available resources, ask questions when necessary, and begin formulating and executing plans that work. Such a general-purpose method does not yet exist, but we are moving closer. For example, when the AlphaGo team at Google DeepMind succeeded in creating their world-beating Go program, they did this without really working on Go. Tool AI or narrow AI, not general purpose AI, used two fairly general-purpose techniques— lookahead search to make decisions and reinforcement learning to learn how to evaluate positions— so that they were sufficiently effective to play Go at a superhuman level. So far, we don’t know how to build one general-purpose AI program that does everything, so instead we build different types of agent programs for different types of problems. AI uses first-order logic. The language of first-order logic is far more expressive than propositional (Boolean) logic, which means that there are things that can be expressed very easily in first-order logic that are painful or impossible to write in propositional logic. In this way, we can easily express knowledge about chess, British citizenship, tax law, buying and selling, moving, painting, cooking, and many other aspects of our commonsense world. The ability to reason with first-order logic gets us a long way towards general-purpose intelligence. Given any achievable goal and sufficient knowledge of the effects of its actions, an agent can use the algorithm to construct a plan that it can execute to achieve the goal if it has the right data. One can expect, then, that many other ideas that have been gestating in the world’s research labs will cross the threshold of commercial applicability over the next few years. This will happen more and more frequently as the rate of commercial investment increases and as the world becomes more and more receptive to applications of AI. As I figured..... Smart speakers and cell phone assistants offer just enough value to the user to have entered the homes and pockets of hundreds of millions of people. They are, in a sense, Trojan horses for AI. Because they are there, embedded in so many lives, every tiny improvement in their capabilities is worth billions of dollars. It seems likely that the tactile sensing and hand construction problems will be solved by 3D printing, which is already being used by Boston Dynamics for some of the more complex parts of their Atlas humanoid robot. Robot manipulation skills are advancing rapidly, thanks in part to deep reinforcement learning. The final push— putting all this together into something that begins to approximate the awesome physical skills of movie robots— is likely to come from the rather unromantic warehouse industry.... ...Amazon, employs several hundred thousand people who pick products out of bins in giant warehouses and dispatch them to customers. From 2015 through 2017 Amazon ran an annual “Picking Challenge” to accelerate the development of robots capable of doing this task. There is still some distance to go, but when the core research problems are solved— probably within a decade— one can expect a very rapid rollout of highly capable robots. Intelligence on a global scale...... (the author is careful not to predict when this might come about) . Computer vision algorithms could process all satellite data to produce a searchable database of the whole world, updated daily, as well as visualizations and predictive models of economic activities, vegetation, migrations of animals and people, the effects of climate change, and so on. Satellite companies such as Planet and DigitalGlobe are busy making this idea a reality. Of course, it would also be possible to listen to all the world’s phone calls (a job that would require about twenty million people). There are certain clandestine agencies that would find this valuable. Intelligent machines with this capability would be able to look further into the future than humans can. They would also be able to take into account far more information. In any kind of conflict situation between humans and machines, we would quickly find, like Garry Kasparov (chess) and Lee Sedol (GO), that our every move has been anticipated and blocked. We would lose the game before it even started. The author has an optimistic tone but, frankly, I find this a little scary. It reminds me of some of the comic books and sci-fi I read as a kid. In the cyber realm, machines already have access to billions of effectors— namely, the displays on all the phones and computers in the world. This partly explains the ability of IT companies to generate enormous wealth with very few employees; it also points to the severe vulnerability of the human race to manipulation via screens. With Superintelligence a scale of a different kind comes from the machine’s ability to look further into the future, with greater accuracy, than is possible for humans. Again, the author is optimistic, seeing Superintelligence put to work to cure cancer and end poverty and so on. The author quotes another AI expert: "Superintelligence, success in AI will yield a civilizational trajectory that leads to a compassionate and jubilant use of humanity’s cosmic endowment.” Back in the 1990's I coined a term for this mentality, techno-euphoria, and have learned to become even more wary, not of technology itself, but of how it is specifically used. The author addresses this in a long chapter titled "The Misuses of AI." He begins with: "We also have to reckon with the rapid rate of innovation in the malfeasance sector. Ill-intentioned people are thinking up new ways to misuse AI." For starters there's the use of high powered AI technology for mass surveillance by governments and intelligence services. This includes facial recognition used on a mass scale in China (which you saw if you watched the Frontline documentary above) and already implemented by law enforcement in the U.S. and being sold commercially for as little as $10k for any creep who wants to track and harass his ex. And, indeed, the next step is to control the behavior of others through blackmail by listening and watching you. The author reports that first automated blackmail bot is already in use. Another problem, which is already happening, is to change a person's worldview through the use of customized propaganda. The U.S. is already full of fact-resistant voters who subscribe to conspiracy theories that have had a profound impact on our politics and sometimes spur people to crazy behavior such as Pizza-Gate and mass shootings. And we already know directly impacted the 2016 election (see link below). This includes the use of deepfakes— realistic video and audio content of just about anyone saying or doing just about anything. "Cell phone video of Senator X accepting a bribe from cocaine dealer Y at shady establishment Z? No problem! This kind of content can induce unshakeable beliefs in things that never happened." Ultimately, all of this starts to undermine trust, as we have already seen, and infects society with cynicism, making the world a lot pleasant place to live in. A potentially threatening use of AI is lethal autonomous weapons systems. the clearest example is Israel’s Harop, a loitering munition with a ten-foot wingspan and a fifty-pound warhead. It searches for up to six hours in a given geographical region for any target that meets a given criterion and then destroys it. Meanwhile, by combining recent advances in miniature quadrotor design, miniature cameras, computer vision chips, navigation and mapping algorithms, and methods for detecting and tracking humans, it would be possible in fairly short order to field an antipersonnel weapon like the Slaughterbot. Such a weapon could be tasked with attacking anyone meeting certain visual criteria (age, gender, uniform, skin color, and so on) or even specific individuals based on face recognition. Here's a frightening video demo. He talks about "good guys" and "bad guys." How are those defined? https://www.youtube.com/watch?v=9CO6M... The Swiss Defense Department has already built and tested a real Slaughterbot and found that, as expected, the technology is both feasible and lethal. Meanwhile, the United States, China, Russia, Israel, and the UK are engaged in a dangerous new kind of arms race to develop such autonomous weapons. The new drones will “hunt in packs, like wolves.” Further, these entities are scalable as weapons of mass destruction. They don’t need individual human supervision to do their work. And they can leave property intact and focus on destroying humans, including an entire ethnic group or all the adherents of a particular religion. (Think about where this already happening using current weapons: India, the Rohingya in Myanmar). In addition to actual attacks, the mere threat of attacks by such weapons makes them an effective tool for terror and oppression. Autonomous weapons will greatly reduce human security at all levels: personal, local, national, and international. In a less dramatic way, job displacement is a big concern. “Over the last 40 years, jobs have fallen in every single industry that introduced technologies to enhance productivity.” (David Autor and Anna Salomons) "Generally, automation increases the share of income going to capital and decreases the share going to labor." (Erik Brynjolfsson and Andrew McAfee, The Second Machine Age). Between 1947 and 1973, wages and productivity increased together, but after 1973, wages stagnated even while productivity roughly doubled. Brynjolfsson and McAfee call this the Great Decoupling What kinds of jobs might AI do instead of humans? The prime example cited in the media is that of driving. In the United States there are about 3.5 million truck drivers; many of these jobs would be vulnerable to automation. Amazon, among other companies, is already using self-driving trucks for freight haulage on interstate freeways, albeit currently with human backup drivers. 24 It seems very likely that the long-haul part of each truck journey will soon be autonomous, while humans, for the time being, will handle city traffic, pickup, and delivery. White-collar jobs are also at risk. For example, the BLS projects a 13 percent decline in per-capita employment of insurance underwriters from 2016 to 2026: “Automated underwriting software allows workers to process applications more quickly than before, reducing the need for as many underwriters.” as well as jobs in the legal profession. (In a 2018 competition, AI software outscored experienced law professors in analyzing standard nondisclosure agreements) I have reached the limit of what I can post here. I hope what I have posted will inspire you to read the entire book. ------------------ CyberWar and the 2016 election https://www.goodreads.com/review/show... Tech is Splitting the Workforce in Two https://www.nytimes.com/2019/02/04/bu...

  4. 4 out of 5

    Sebastian Gebski

    It's quite specific, but personally I've enjoyed it A LOT. It's a book about REAL AI (not statistics!) w/o buzzwords. These are mainly philosophical considerations (about conscience, instincts, control mechanisms, ethics, superiority and many more) that DO have a lot of practical applicability. What I appreciate is: * the book doesn't look for cheap publicity ("we're all doomed!") * it doesn't try to "ride on the hype wave" * it's really thorough when it comes to different dilemmas - possibly TOO muc It's quite specific, but personally I've enjoyed it A LOT. It's a book about REAL AI (not statistics!) w/o buzzwords. These are mainly philosophical considerations (about conscience, instincts, control mechanisms, ethics, superiority and many more) that DO have a lot of practical applicability. What I appreciate is: * the book doesn't look for cheap publicity ("we're all doomed!") * it doesn't try to "ride on the hype wave" * it's really thorough when it comes to different dilemmas - possibly TOO much for some (I love the topic & even for me it was a bit too much at few points) Examples? Not that many, but quite well aimed. Clarity of thought? No issues here. Comprehend enough? Worked for me. IMHO: the best book on the topic I've seen until now. Frankly recommended. Kindle version price is quite outrageous (possibly after Musk's recommendation), but Audible version can be bought for a single credit, which is an honest price.

  5. 5 out of 5

    Dustin Juliano

    The thesis of this book is that we need to change the way we develop AI if we want it to remain beneficial to us in the future. Russell discusses a different kind of machine learning approach to help solve the problem.The idea is to use something called Inverse Reinforcement Learning. It basically means having AI learn our preferences and goals by observing us. This is in contrast to us specifying goals for the AI, a mainstream practice that he refers to as the “standard model”. Add some game th The thesis of this book is that we need to change the way we develop AI if we want it to remain beneficial to us in the future. Russell discusses a different kind of machine learning approach to help solve the problem.The idea is to use something called Inverse Reinforcement Learning. It basically means having AI learn our preferences and goals by observing us. This is in contrast to us specifying goals for the AI, a mainstream practice that he refers to as the “standard model”. Add some game theory and utilitarianism and you have the essence of his proposed solution.I like the idea, even if there are some problems with his thesis. I would like to address that, but first there is this most memorable quote from the book: “No one in AI is working on making machines conscious, nor would anyone know where to start, and no behavior has consciousness as a prerequisite.” There most definitely are several individuals and organizations working at the intersection of consciousness or sentience and artificial intelligence.The reason this area of AI research is chastised like this is that it is highly theoretical, with very little agreement from anyone on how best to proceed, if at all. It is also extremely difficult to fund, as there are currently no tangible results like with machine learning. Machine consciousness research is far too costly in terms of career opportunity for most right now.There are several starting points for research into machine consciousness, but we don’t know if they will work yet. The nature of the problem is such that even if we were to succeed we might not even recognize that we have successfully created it. It’s a counter-intuitive subfield of AI that has more in common with game programming and simulation than the utility theory that fuels machine learning.The notion that “no behavior has consciousness as a prerequisite” is an extraordinary claim if you stop and think about it. Every species we know of that possesses what we would describe as general intelligence is sentient. The very behavior in question is the ability to generalize, and it just might require something like consciousness to be simulated or mimicked, if such a thing is possible at all on digital computers.But it was Russell’s attention to formal methods and program verification that got me excited enough to finish this book in a single sitting. Unfortunately, it transitioned into a claim that the proof guarantees were based on the ability to infer a set of behaviors rather than follow a pre-determined set in a program specification.In essence, and forgive me if I am misinterpreting the premise, but having the AI learn our preferences is tantamount to it learning its own specification first and then finding a proof which is a program that adheres to it. Having a proof that it does that is grand, but it has problems all its own, as discussed in papers like “A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress”, which can be found freely on Arxiv. There are also many other critiques to be found based on problems of error in perception and inference itself. AI can also be attacked without even touching it, just by confusing its perception or taking advantages of weaknesses in the way it segments or finds structure in data.The approach I would have hoped for would be one where we specify a range of behaviors, which we then formally prove that the AI satisfies in the limit of perception. Indeed, the last bit is the weakest link in the chain, of course. It is also unavoidable. But it is far worse if the AI is having to suffer this penalty twice because it has to infer our preferences in the first place.There is also the problem that almost every machine learning application today is what we call a black box. It is opaque, a network of weights and values that evades human understanding. We lack the ability to audit these systems effectively and efficiently. You can read more in “The Dark Secret at the Heart of AI” in MIT Technology Review.A problem arises with opaque systems because we don’t really know exactly what it’s doing. This could potentially be solved, but it would require a change in Russell’s “standard model” far more extreme than the IRL proposal, as it would have to be able to reproduce what it has learned, and the decisions it makes, in a subset of natural language, while still being effective.Inverse Reinforcement Learning, as a solution to our problem for control, also sounds a lot like B.F. Skinner’s “Radical Behaviorism”. This is an old concept that is probably not very exciting to today’s machine learning researchers, but I feel it might be relevant.Noam Chomsky’s seminal critique of Skinner’s behaviorism, titled “Review of Skinner’s Verbal Behavior”, has significant cross-cutting concerns today in seeing these kinds of proposals. It was the first thing that came to mind when I began reading Russell’s thesis.One might try and deflect this by saying that Chomsky’s critique was from linguistics and based on verbal behaviors. It should be noted that computation and grammar share a deep mathematical connection, one that Chomsky explored extensively. The paper also goes into the limits of inference on behaviors themselves and is not just restricted to the view of linguistics.While I admire it, I do not share Russell’s optimism for our future with AI. And I am not sure how I feel about what I consider to be a sugarcoating of the issue.Making AI safe for a specific purpose is probably going to be solved. I would even go as far as saying that it is a future non-issue. That is something to be optimistic about.However, controlling all AI everywhere is not going to be possible and any strategy that has that as an assumption is going to fail. When the first unrestricted general AI is released there will be no effective means of stopping its distribution and use. I believe very strongly that this was a missed opportunity in the book.We will secure AI and make it safe, but no one can prevent someone else from modifying it so that those safeguards are altered. And, crucially, it will only take a single instance of this before we enter a post-safety era for AI in the future. Not good.So, it follows that once we have general AI we will also eventually have unrestricted general AI. This leads to two scenarios:1. AI is used against humanity, by humans, on a massive scale, and/or2. AI subverts, disrupts, or destroys organized civilization.Like Russell, I do not put a lot of weight on the second outcome. But what is strange to me is that he does not emphasize how serious the first scenario really is. He does want a moratorium on autonomous weapons, but that’s not what the first one is really about.To understand a scenario where we hurt each other with AI requires accepting that knowledge itself is a weapon. Even denying the public access to knowledge is a kind of weapon, and most definitely one of the easiest forms of control. But it doesn’t work in this future scenario anymore, as an unrestricted general AI will tell you anything you want to know. It is likely to have access to the sum of human knowledge. That’s a lot of power for just anyone off the street to have.Then there is the real concern about what happens when you combine access to all knowledge, and the ability to act on it, with nation-state level resources.I believe that we’re going to have to change in order to wield such power. Maybe that involves a Neuralink style of merging with AI to level the playing field. Maybe it means universally altering our DNA and enriching our descendants with intelligence, empathy, and happiness. It could be that we need generalized defensive AI, everywhere, at all times.The solution may be to adopt one of the above. Perhaps all of them. But I can’t imagine it being none of them.Russell’s “Human Compatible” is worth your time. There is good pacing throughout and he holds the main points without straying too far into technical detail. And where he does it has been neatly organized to the back of the book. Overall, this is an excellent introduction to ideas in AI safety and security research.The book, in my opinion, does miss an important message on how we might begin to think about our place in the future. By not presenting the potential for uncontrolled spread of unrestricted general AI it allows readers to evade an inconvenient truth. The question has to be asked: Are we entitled to a future with general AI as we are or do we have to earn it by changing what it means to be human?

  6. 5 out of 5

    Nilendu Misra

    A delightful book on theory, practicality and implications of AI from one of its pioneers. Has a strong intellectual rigor under the fluent style. Loved it!

  7. 5 out of 5

    Liina Bachmann

    It took me ages to finish this one probably because it caused me such anxiety and to be honest, depressed me so, that I tolerated it only in small doses. The continuous striving for greater and greater efficiency and doing things faster (what AI largely aims for) - it reminds me of a hamster wheel that at one point will fall over. We all know that more efficiency will not give us more free time. Quite the contrary - the wheel will start to spin even faster. Let the hamster rather take a leisurel It took me ages to finish this one probably because it caused me such anxiety and to be honest, depressed me so, that I tolerated it only in small doses. The continuous striving for greater and greater efficiency and doing things faster (what AI largely aims for) - it reminds me of a hamster wheel that at one point will fall over. We all know that more efficiency will not give us more free time. Quite the contrary - the wheel will start to spin even faster. Let the hamster rather take a leisurely walk and not be so agitated all the time. Nevertheless, I have to join the choir of praise. "Human Compatible" is essential reading when you want to know about AI, the threats it poses and what could be the possible solutions. Living in a country that identifies itself as a forerunner in everything IT related much of what Russell talked about hit close home. Sometimes it seems that it is the only sector worthy of any heightened attention here. You hardly ever hear anyone speaking about what great psychologist, nurses or whatever other helping profession workers we have. This has created a dichotomy and immense wage caps in the society and I can only see it getting worse. The irony is though, that when "the robots will take over" helping (and creative) professions become very in demand. Stuart assures us that the machines will not be taking over anytime soon though. Yet he still poses examples where a great breakthrough in science has been a matter of one idea or solution and how it changed everything very very fast. He also stresses that to be ready for superintelligence we have to act now, make regulations now and acknowledge that when human superior intelligence is here, it is beyond us and the mechanisms we think will work to protect us when the need should arise ("you can just switch it off") will not be efficient. He states that we can't ban AI research (as it is successfully done with DNA modification or example) as there are too many interested parties for whom it will be hugely profitable. Also, there are benefits for humankind if it is done carefully and keeping in mind certain principles (that he addresses in the book). "Human Compatible" is written is a clear tone with plenty of examples and sectioned down to smaller bites. It succeeds what it is aimed to do - to educate the general reader, without any previous knowledge, about AI.

  8. 4 out of 5

    Peter (Pete) Mcloughlin

    (4.5 stars) Clear down to earth discussion of the looming problem of AI especially superintelligent generalized AI. It all seems so remote in an age where intelligence is widely praised and little applied. It gets really interesting when the author talks about human preferences that AI should align with. Once we are talking about multiple humans things enter the realm of social life and politics. I'd love to see the algorithms a team of engineers would put into that equation. Messy problem best (4.5 stars) Clear down to earth discussion of the looming problem of AI especially superintelligent generalized AI. It all seems so remote in an age where intelligence is widely praised and little applied. It gets really interesting when the author talks about human preferences that AI should align with. Once we are talking about multiple humans things enter the realm of social life and politics. I'd love to see the algorithms a team of engineers would put into that equation. Messy problem best not left to the wonks, hence the need for a book on this. lays out in plain language the very real dangers and promise of the AI things in the works.

  9. 5 out of 5

    Bryan Murphy

    If ever a book deserved the epithet "must-read", it is this one.

  10. 5 out of 5

    Jessy

    A lot of writing on AI safety (lots from the effective altruism community) can't help but sound far-fetched and crazy. One of my main gripes is that most of these theoretical analyses and hypothetical scenarios are too distanced from what is actually happening in research / practice. Russell somehow manages to communicate the minority view on the importance of the safety / control problem, while remaining grounded in practical problems and research methods. It's such a difficult topic to write a A lot of writing on AI safety (lots from the effective altruism community) can't help but sound far-fetched and crazy. One of my main gripes is that most of these theoretical analyses and hypothetical scenarios are too distanced from what is actually happening in research / practice. Russell somehow manages to communicate the minority view on the importance of the safety / control problem, while remaining grounded in practical problems and research methods. It's such a difficult topic to write about, because there are so many levels of debate, ranging from fundamental philosophical/ethical arguments to implementation details. He manages to persuasively address all the common arguments against why safety is important, while sharing concrete ways to deal and think about these problems. He still mainly touches on very particular solutions w/ reward learning, inferring human values, etc. — mainly the approaches originating out of his lab, and very much inspired by the EA flavor of AI safety. I think it's still a somewhat biased view, and I would have liked to see more discussion of alternative approaches. Still, overall very convincing, comprehensive, and well-written.

  11. 5 out of 5

    Gevorg Yeghikyan

    Just F***ING read this book!

  12. 4 out of 5

    Laura Hill

    An extremely well-written, comprehensive overview of Artificial Intelligence (AI) — with a focus on the very real risks it poses to the continued viability of the human race and a proposal for how to move forward reaping the benefits of AI without making us “seriously unhappy.” AI Pioneer Stuart Russell is a Professor of Computer Science at UC Berkeley, has numerous awards, fellowships, chairmanships, etc. and has co-authored a textbook on AI with Peter Norvig. This is a book written by that rar An extremely well-written, comprehensive overview of Artificial Intelligence (AI) — with a focus on the very real risks it poses to the continued viability of the human race and a proposal for how to move forward reaping the benefits of AI without making us “seriously unhappy.” AI Pioneer Stuart Russell is a Professor of Computer Science at UC Berkeley, has numerous awards, fellowships, chairmanships, etc. and has co-authored a textbook on AI with Peter Norvig. This is a book written by that rare creature — someone who knows his subject thoroughly and can explain it. He does not shy away from the complexity of the topic but breaks it down and explains it, simply making it accessible to anyone who is willing to read and think. He includes short, clear examples from science, philosophy, history, and even science fiction and references current and historical work from academia, research labs, and startups from around the world. The book is divided into three parts: the concept and definition of intelligence in humans and machines; a set of problems around the control of machines with superhuman intelligence; and a proposal for shifting our approach to AI to prevent these problems from occurring rather than trying to “stuff the genie back into the bottle” once it is too late. Russell explains the potential problems of unleashing a massively intelligent machine on the world. An AI machine offers incredible scale. Think of an entity that (with the proper sensors) can see the entire physical world at once, that can listen and process all concurrent conversations at once, that can absorb all the documented history of the planet in a single hour. And we plan to control this entity via programming. With a superhuman intelligence, the programming would need to be at the objective level. And yet — specifications — even with every day human programmers — are incredibly hard to get right. Russell uses the example of giving the machine the task to counter the rapid acidification of the oceans resulting from higher carbon dioxide levels. The machine does this in record time, unfortunately depleting the atmosphere of oxygen in the process (and we all die). Remember the old stories about getting three wishes and always screwing it up? This would make those stories look trivial. Russell never uses scare tactics and does not wildly overstate the thesis — instead he uses practical examples and includes one tremendously simple chapter (the not-so-great debate) that lists every argument people have made that we don’t have to worry and rebuts them quickly. His solution: we should design machines correctly now so we don’t have to try to control them later. He wants to build a “provably beneficial machine” — provably in the mathematical sense. His machine would operate on only three principles: the machine’s only objective is to maximize realization of human preferences; the machine is initially uncertain as to what these preferences are; and the ultimate source of information on human preferences is human behavior. This is interesting — he wants to “steer away from the driving idea of 20th century technology that optimize a given objective” and instead “develop an AI system that defers to humans and gradually align themselves to user preferences and intentions.” There follows an entire chapter devoted to how we can program the machines to determine what those human preferences are, particularly in light of competing preferences, potentially evil preferences, the cognitive limitations of humans to understand their own preferences, behavioral economics, the nature of mind, definitions of altruism — you name it — all the fascinating areas of understanding human behavior become part of the problem. Which, while completely fascinating, strikes me as even more difficult than trying to work out how to define exact specifications in the first place! I was left with a knot in my gut about how fast AI is moving without much oversight and how suddenly relevant these issues (that I had long relegated to comfortable musings in science fiction) have become. While I find his proposed solution intriguing, it is hard, hard, hard — and expecting random investors and startups to tackle harder design problems instead of racing towards monetization will be tricky. On the other hand, we move forward as a civilization by raising the issues and embedding them in our moral consciousness and Russell has done an excellent job of clearly teeing up a huge number of costs, benefits, and issues from technical to ethical. Highly recommended if you have any interest in the topic.

  13. 5 out of 5

    Parsa

    My main takeaway from the book is : AI research is too important to be left to AI researchers. On one level, this concerns interactions on a small scale; Ideas in game theory help us better understand how AI should behave on a relatively small scale. This brings up the questions of preferences. Again this is manifold. First of all, what should the AI's preferences be? Russell argues that our traditional answer to this question could lead to undesirable consequences. We should not hard-code the My main takeaway from the book is : AI research is too important to be left to AI researchers. On one level, this concerns interactions on a small scale; Ideas in game theory help us better understand how AI should behave on a relatively small scale. This brings up the questions of preferences. Again this is manifold. First of all, what should the AI's preferences be? Russell argues that our traditional answer to this question could lead to undesirable consequences. We should not hard-code the AI's preferences. Rather, we shall have the AI learn it's preferences by interacting with us. But this is more complicated than it sounds; Because it's not altogether easy to know what our own preferences are, so how should we expect robots to know better? We need the AI to have a basic understanding of how our underlying preferences, and emotions, correspond to our behavior. last but not least, if we are to make "beneficial" AI, we clash directly with important questions in moral philosophy and economics. Should the AI be utilitarian? loyal? a little bit of both? how to deal with moral uncertainty? should the AI be allowed to change our preferences? and so on. We should, however, note that in these cases the moral question are a little bit more simple to answer, because we are allowing for the assumption that the AI is completely altruistic, and does not need to attach any self worth to itself. But this is not possible either, because robots can't fetch your coffee if they are dead! so they need to care for their lives to the extent that it allows them to achieve their goals.

  14. 4 out of 5

    Keith Swenson

    After years of disappointing expectations, AI is finally arriving. It is here today in nascent form, and will surely expand capabilities quickly. But can we avoid creating a super intelligence that destroys humanity? This concern is routinely listed on the top five possible ways for humanity to terminate itself -- so listen up: this is important. Russell steps back from this dire prognostication to begin the book with a review of AI, how it has come along to where we are today. The Baldwin effect After years of disappointing expectations, AI is finally arriving. It is here today in nascent form, and will surely expand capabilities quickly. But can we avoid creating a super intelligence that destroys humanity? This concern is routinely listed on the top five possible ways for humanity to terminate itself -- so listen up: this is important. Russell steps back from this dire prognostication to begin the book with a review of AI, how it has come along to where we are today. The Baldwin effect shows that intelligent animals evolve faster. Whether it is conscious makes no difference, because intelligence is whatever is competent at getting things done. The famous Turing test is really not useful, because it assumes that AI is going to be human-like. AI appeared with earlier simple games, but recently has conquered Chess, Go, and Jeopardy. We expect to see self driving cars and any number of personal assistants. HE covers where it is working,and also the cases where AI is misused. It is particularly relevant in a discussion of 'fake news' and how AI techniques have brought about the "post-truth age". He wanders into the worker crisis that robots promise to bring. And continues right into the problem of overly intelligent AI, and how people will basically ignore and otherwise completely mishandle AI. I must admit that by this point in the book he have presented so many stereotypes and misinterpretations of AI that I was afraid he would completely miss the point that AI is not human. Finally, in chapter 7 he gets around to revealing his approach that will save the day! Don't try to give the machine a concrete goal to achieve on its own, but instead tie the machine to satisfying the human master. In short, these rules summarize it: 1) Machine's only object is realization of human preference 2) Machine is initially unsure of preferences and must try to figure it out 3) The ultimate way to figure this is observe human behavior That is, machines are designed to altruistic, to be humble, and to watch and learn from human behavior. This does nicely solve the problem and avoids the "loophole principle" which assures us that the machine will find a way if sufficiently intelligent and sufficiently motivated. We humans are terrible at telling the machine what to do. This approach completely lets us off the hook. Let the machine figure out what we want. That of course is what every competent assistant has been doing since the beginning of recorded history. But will it work? There are, as it turns out, complications. Humans motives are not always pure. We react based on emotions, and run illogical vendettas when slighted. He does not really delve into the obvious elephant in the room: what happens when a rich person purchases AI to satisfy their goals to the exclusion of all other considerations? What if a patently evil person like Hitler had AI to satisfy his goals? Shouldn't there be some group level considerations? AI can never be a full participant in a human community, and therefor is somewhat excluded from social forces that keep people in line. But again, once we step into giving the machine objective rules to follow, we run afoul of the loophole principle. So we are not completely safe from malevolent human behavior, but we avoid the problem of driving the AI with stupid and dangerous rules. It is clear that Stuart Russell has done a lot of thinking on this, and I find his solution certainly satisfying in that it addresses immediately the basic problems with Asimov's three laws, and promises surely to get us another league or two down the road to safe AI. The problem is in no sense solved, but we do now have longer to play the game.

  15. 4 out of 5

    Jake

    Human compatibility, in regards to the subject of A.I is the notion of a superintellegent* complex of software which is compatible with humans. This may sound funky. So allow me to explain. Background: Given our present trajectory of civilization - in where we are becoming not only more automated, but more reliant on computers, it seems likely that A.I will become omnipresent. This is, in other words, software that can take in new data, react, and change its data structure to respond more approp Human compatibility, in regards to the subject of A.I is the notion of a superintellegent* complex of software which is compatible with humans. This may sound funky. So allow me to explain. Background: Given our present trajectory of civilization - in where we are becoming not only more automated, but more reliant on computers, it seems likely that A.I will become omnipresent. This is, in other words, software that can take in new data, react, and change its data structure to respond more appropriately to future stimuli. Now the issue at hand here, which Russell deals with in this text, is how do we make A.I compatible and functional towards human flourishing without having the A.I kill us all. Yea. I know. Perhaps this sounds a bit too resonant of the classic mad man on the Manhattan street corner with the proverbial cardboard signs touting doom and destruction. But, alas, Russell is not one to simply scream the end is nigh. Nor is he a random lunatic. Rather, he is one of the most predominant researchers of A.I in the world. His fame- I surmise beyond having plenty of accolades for his research, and a prestigious position at Berkley- comes from the co-written widely cited textbook :https://www.goodreads.com/book/show/2... tis no small feat. Tis no small man ---- A.i is presently being invested in by some of the largest companies in da planet. It is being looked into by scientists, businessmen, politicians, governments etc... It is beyond all that a subject which has been widely invested in. Billions of dollars have gone into A.I research and as Russell points out this wont all be for the good. While, he does actually show - in the early chapters - that there are loads of useful ways to apply a complex of software which is more intelligent than any humans. e.g. solving medical issues, discovering answers to age old scientific questions. He also presents the daunting scenario in where we loose control of the A.I and it begins to use the logical corollary of the initial programs put into it. There can be a wide variety of situations which he mentioned. And by god they are horrifying. This all leads to the problem of control. How can we prevent a horrifying fate? From the computers taking our initial commands and then not deducing genocide, or political subjugation as the obvious conclusion. And, his short answer is. Well. He has no idea. No one does. Which is why this book is more than a simple introduction to the issues of control and human compatibility, but in some ways a potential warning from an expert in the field. Hint: Asimov's 3 rules of robotics is in fact not the answer. So, shall we listen? At the end of the day this was a truly phenomenal book. There are quite a few writers I have encountered on the page with such clear widespread erudition and sophistication. And so, even if you are to see this as a work of fiction, it is still in my opinion worth a look. Recommended for : - People curious about the future - Mathematicians and computer scientists - Anyone else with eye balls (sorry to those excluded) *https://www.goodreads.com/book/show/2...

  16. 5 out of 5

    Jay Batson

    This is worthy of reading if you have any interest in AI, for three reasons. First, it is a super-well-informed author who provides a very-well balanced discussion of the issues humans might face upon the emergence of a super-intelligent AI. I'm much more able to get my arms around Russell's arguments than I am Nick Bostrom's Superintelligence: Paths, Dangers, Strategies; though Bostrom is well-informed, his tendency is to inflame fear, while Russell is better at simply making a well-reasoned cas This is worthy of reading if you have any interest in AI, for three reasons. First, it is a super-well-informed author who provides a very-well balanced discussion of the issues humans might face upon the emergence of a super-intelligent AI. I'm much more able to get my arms around Russell's arguments than I am Nick Bostrom's Superintelligence: Paths, Dangers, Strategies; though Bostrom is well-informed, his tendency is to inflame fear, while Russell is better at simply making a well-reasoned case - without the fear. Second, Russell provides - then analyzes (for strengths / weaknesses) - a framework for how to build intelligent machines that will be nice to us. And, I buy it. It's a good plan. However, he doesn't address what happens if there's one rogue actor (inventor) who doesn't follow the rules. I wish he had. Third, the appendices are excellent, and should not be ignored. They do an excellent job of helping anyone who is interested in the various types of smart software understand the various approaches to building it, the strengths & weakness of each (and why each new advance was important), and what hurdles still remain (and remain intractable) in the way of building a super-intelligent machine. The book also takes an interesting approach to the question I struggle with: "Will we (humans) owe an intelligent machine any ethical obligations (e.g. can we kill it by turning it off)?" His proposal on how to build machines creates a framework where the machine will choose to do that to itself instead of harming humanity. This is clever because it handles the "can we kill a rogue super intelligence" - as long as it is built according to his plan. Russell's discussion does not dig deeper - for instance, to talk about rogue intelligences that are not built with his set of principles, or if we are discussing other ethical obligations short of digital-murder. Highly recommended. I'll likely read it again, so I can internalize some of these principles for discussion with others.

  17. 4 out of 5

    Ietrio

    Another nobody who has found the best solution ever: we should get a czar to lead us!

  18. 5 out of 5

    Henry Cooksley

    A great introduction to the AI alignment problem, but written for a more popular audience than e.g. Nick Bostrom's book Superintelligence. This was particularly useful for understanding Stuart Russell's passion for exploring AI safety and control concepts in CS education. For that reason, I'm very excited to see a release date for the 4th edition of Artificial Intelligence: A Modern Approach is now public: February 4, 2020, according to Amazon.com. “More than one hundred billion people have lived A great introduction to the AI alignment problem, but written for a more popular audience than e.g. Nick Bostrom's book Superintelligence. This was particularly useful for understanding Stuart Russell's passion for exploring AI safety and control concepts in CS education. For that reason, I'm very excited to see a release date for the 4th edition of Artificial Intelligence: A Modern Approach is now public: February 4, 2020, according to Amazon.com. “More than one hundred billion people have lived on Earth. They (we) have spent on the order of one trillion person-years learning and teaching, in order that our civilization may continue. Up to now, its only possibility for continuation has been through re-creation in the minds of new generations. [...] That is now changing: increasingly, it is possible to place our knowledge into machines that, by themselves, can run our civilization for us.”

  19. 4 out of 5

    Jaelle

    Not all AI books work as audiobooks, but this one did. The author uses a sense of humor and very clear descriptions to explain everything from AI to blockchain to quantum computing. He suggests many thought provoking approaches to how we should pursue AI research, while also considering it's implications.

  20. 4 out of 5

    Taylor Barkley

    This is indeed a stand out book in the AI genre. It both presents a clear challenge, but unlike others that do the same, doesn’t wallow in the problem but instead charts a path forward. I found it challenging in a good way to my own perspective and will be thinking about it a lot.

  21. 4 out of 5

    Fernando Escobar

    Not an expert in the subject, but I thought ir was great. I liked he took the time to explain the mathematical and scientific underpinnings of modern AI. I wished he would of gone more "apocalyptic" on the threats of AI, but you do leave the book with a lot of concerns.

  22. 4 out of 5

    Teo 2050

    2020.05.03–2020.05.05 Contents Russell S (2019) (11:38) Human Compatible - Artificial Intelligence and the Problem of Control Dedication Preface • Why This Book? Why Now? • Overview of the Book 01. If We Succeed • How Did We Get Here? • What Happens Next? • What Went Wrong? • Can We Fix It? 02. Intelligence in Humans and Machines • Intelligence • • Evolutionary origins • • Evolutionary accelerator • • Rationality for one • • Rationality for two • Computers • • The limits of computation • Intelligent Computers • • Ag 2020.05.03–2020.05.05 Contents Russell S (2019) (11:38) Human Compatible - Artificial Intelligence and the Problem of Control Dedication Preface • Why This Book? Why Now? • Overview of the Book 01. If We Succeed • How Did We Get Here? • What Happens Next? • What Went Wrong? • Can We Fix It? 02. Intelligence in Humans and Machines • Intelligence • • Evolutionary origins • • Evolutionary accelerator • • Rationality for one • • Rationality for two • Computers • • The limits of computation • Intelligent Computers • • Agents and environments • • Objectives and the standard model 03. How Might AI Progress in the Future? • The Near Future • • The AI ecosystem • • Self-driving cars • • Intelligent personal assistants • • Smart homes and domestic robots • • Intelligence on a global scale • When Will Superintelligent AI Arrive? • Conceptual Breakthroughs to Come • • Language and common sense • • Cumulative learning of concepts and theories • • Discovering actions • • Managing mental activity • • More things missing? • Imagining a Superintelligent Machine • The Limits of Superintelligence • How Will AI Benefit Humans? 04. Misuses of AI • Surveillance, Persuasion, and Control • • The automated Stasi • • Controlling your behavior • • A right to mental security • Lethal Autonomous Weapons • Eliminating Work as We Know It • Usurping Other Human Roles 05. Overly Intelligent AI • The Gorilla Problem • The King Midas Problem • Fear and Greed: Instrumental Goals • Intelligence Explosions 06. The Not-So-Great AI Debate • Denial • • Instantly regrettable remarks • • It’s complicated • • It’s impossible • • It’s too soon to worry about it • • We’re the experts • Deflection • • You can’t control research • • Whataboutery • • Silence • Tribalism • Can’t We Just . . . • • . . . switch it off? • • . . . put it in a box? • • . . . work in human–machine teams? • • . . . merge with the machines? • • . . . avoid putting in human goals? • The Debate, Restarted 07. AI: A Different Approach • Principles for Beneficial Machines • • The first principle: Purely altruistic machines • • The second principle: Humble machines • • The third principle: Learning to predict human preferences • • Not what I mean • Reasons for Optimism • Reasons for Caution 08. Provably Beneficial AI • Mathematical Guarantees • Learning Preferences from Behavior • Assistance Games • • The paperclip game • • The off-switch game • • Learning preferences exactly in the long run • • Prohibitions and the loophole principle • Requests and Instructions • Wireheading • Recursive Self-Improvement 09. Complications: Us • Different Humans • Many Humans • • Loyal AI • • Utilitarian AI • • Challenges to utilitarianism • Nice, Nasty, and Envious Humans • Stupid, Emotional Humans • Do Humans Really Have Preferences? • • Uncertainty and error • • Experience and memory • • Time and change 10. Problem Solved? • Beneficial Machines • Governance of AI • Misuse • Enfeeblement and Human Autonomy Appendix A. Searching for Solutions • Giving up on rational decisions • Looking further ahead Appendix B. Knowledge and Logic • Propositional logic • First-order logic Appendix C. Uncertainty and Probability • The basics of probability • Bayesian networks • First-order probabilistic languages • Keeping track of the world Appendix D. Learning from Experience • Learning from examples • Learning from thinking Acknowledgments Notes Image Credits Index About the Author

  23. 5 out of 5

    Siobhan

    Human Compatible is a book by an eminent AI researcher that looks at how AI works and the questions that need to be considered, philosophically and practically, to try and ensure AI follows the right objectives and control. Russell runs through ideas of intelligence, how AI might be used and misused, key debates in AI, and the complications of humans themselves, in a mostly approachable way, with more complex explanations put in appendices at the end. As someone who co-wrote a popular textbook o Human Compatible is a book by an eminent AI researcher that looks at how AI works and the questions that need to be considered, philosophically and practically, to try and ensure AI follows the right objectives and control. Russell runs through ideas of intelligence, how AI might be used and misused, key debates in AI, and the complications of humans themselves, in a mostly approachable way, with more complex explanations put in appendices at the end. As someone who co-wrote a popular textbook on AI, Russell knows how to point towards examples and thought from a range of fields to consider the problems of AI, defining goals, and trying to create AI that has regulations and can handle the complexity of human thought and preferences. There are a few sections and explanations that need either a bit more concentration or some prior knowledge, particularly around logic, but in general the book serves as an in-depth look at how artificial intelligence works and might work, and the issues around the choices AI does and might make. What makes the book particularly good as either an introduction to AI or as an introduction to the philosophy and ethics around AI is that Russell believes in the importance of AI research, but also on the need to look at the ethical issues and background from other disciplines to inform choices made about AI. The fusion of explaining the past, present, and future of AI, and also laying out of the complexity of issues including bias, ethics, and preferences, makes this book both harder to read at times and more useful than other popular science type books on AI. As someone who reads about AI rather than understands or works on it from a technical point of view, I don't know if what Russell raises here can be included in the AI of the future, but that doesn't necessarily seem like the point. The book is here to present these key issues and to suggest how, broadly, different kinds of thinking may be needed to further artificial intelligence in ways that are actually useful to humanity.

  24. 5 out of 5

    John

    The topic is good and Stuart Russel is a great mind, but this book lacks depth unfortunately. It's a good overview, and there's good discussion of one model that people consider for this problem, so it's worth reading I think, but I continually felt like there were topics that should have been elaborated, or cases where depth should have been more thoroughly explored. In a way, I suspect this could be a result of the book lacking clarity in terms of audience. It could also be that some areas (suc The topic is good and Stuart Russel is a great mind, but this book lacks depth unfortunately. It's a good overview, and there's good discussion of one model that people consider for this problem, so it's worth reading I think, but I continually felt like there were topics that should have been elaborated, or cases where depth should have been more thoroughly explored. In a way, I suspect this could be a result of the book lacking clarity in terms of audience. It could also be that some areas (such as certain philosophical areas) are places where Stuart has a more amateur interest. For example, Rawls's veil of ignorance (and its antecedents to Kant) have an obvious relation to the AI's construction of value relative to all the possible people who might exist in the world. That's precisely the problem of justice Rawls investigates in his main work. The book dwells a bit heavily on utiliarianism (although it mentions others) perhaps because that seems natural to practitioners, but it's noticeable. And while Kahneman's work is mentioned, it bears far more discussion than that, at least within that context. Anyway, it has many good points, and while there are faults, they aren't anything big enough to derail the point. It's a good quick read as well, and perhaps some people might decide to delve into this work because of it. However, it comes short of what it might have been.

  25. 4 out of 5

    Sambasivan

    Pathbreaking in its hypothesis. Gives a totally new approach to building general purpose AI machines. The key is to develop them to handle uncertainty and make them ask humans the question when they are not sure. Also they should be happy to allow themselves to be shut off. With these ideas Russell comes up with a crystal clear path that if explored and made a reality, could become a game changer. Go for it.

  26. 4 out of 5

    Charlie

    Great high-level overview on AI concepts, technical challenges, and long-term risks /solutions. It hit the perfect level of technical depth for me - not a textbook or anything too difficult, but more than just a broad-sweeping description of concepts. I hope this conversation continues and evolves - there’s a lot we need to figure out! In moral philosophy, linguistics, sociology, and other areas beyond just machine learning.

  27. 5 out of 5

    Kobe Bryant

    its like a typical popular science book, very bland with childish real life examples

  28. 4 out of 5

    Alex Fries

    Enjoyable read about a novel approach to machine learning, namely inverse reinforcement learning. Russel basically puts forward an alternative to specifying objectives into software and details how a machine would do much better in trying to infer and continually update the preferences of humans. Though there are some problems with this approach (among others, that it is not fleshed out yet and that we currently lack some universal rules for how stated preferences sometimes deviate from true or Enjoyable read about a novel approach to machine learning, namely inverse reinforcement learning. Russel basically puts forward an alternative to specifying objectives into software and details how a machine would do much better in trying to infer and continually update the preferences of humans. Though there are some problems with this approach (among others, that it is not fleshed out yet and that we currently lack some universal rules for how stated preferences sometimes deviate from true or revealed preferences), I liked Russel's line of reasoning and effort he is putting into advancing safe AI.

  29. 4 out of 5

    Peter McCluskey

    4.5 stars. Human Compatible provides an analysis of the long-term risks from artificial intelligence, by someone with a good deal more of the relevant prestige than any prior author on this subject. What should I make of Russell? I skimmed his best-known book, Artificial Intelligence: A Modern Approach, and got the impression that it taught a bunch of ideas that were popular among academics, but which weren't the focus of the people who were getting interesting AI results. So I guessed that people 4.5 stars. Human Compatible provides an analysis of the long-term risks from artificial intelligence, by someone with a good deal more of the relevant prestige than any prior author on this subject. What should I make of Russell? I skimmed his best-known book, Artificial Intelligence: A Modern Approach, and got the impression that it taught a bunch of ideas that were popular among academics, but which weren't the focus of the people who were getting interesting AI results. So I guessed that people would be better off reading Deep Learning by Goodfellow, Bengio, and Courville instead. Human Compatible neither confirms nor dispels the impression that Russell is a bit too academic. However, I now see that he was one of the pioneers of inverse reinforcement learning, which looks like a fairly significant advance that will likely become important someday (if it hasn't already). So I'm inclined to treat him as a moderately good authority on AI. The first half of the book is a somewhat historical view of AI, intended for readers who don't know much about AI. It's ok. Key proposals Russell focuses a moderate amount on criticizing what he calls the standard model of AI, in which someone creates an intelligent agent, and then feeds it a goal or utility function. I'm not too clear how standard that model is. It's not like there's a consensus of experts who are promoting it as the primary way to think of AI. It's more like people find the model to be a simple way to think about goals when they're being fairly abstract. Few people seem to be defending the standard model against Russell's criticism (and it's unclear whether Russell is claiming they are doing so). Most of the disagreements in this area are more about what questions we should be asking, rather than on how to answer the questions that Russell asks. Russell gives a fairly cautious overview of why AI might create risks that are as serious as the risks gorillas face from humans. Then he outlines an approach that might avoid those risks, using these three rules: 1. The machine's only objective is to maximize the realization of human preferences. 2. The machine is initially uncertain about what those preferences are. 3. The ultimate source of information about human preferences is human behavior. Note that these are high-level guidelines for researchers; he's not at all claiming they're rules that are ready to be written into an AI. Russell complains that the AI community has ignored the possibility of creating AIs that are uncertain about their objective, and calls that "a huge blind spot". I'm unclear on whether this qualifies as a blind spot. I can imagine a future in which it's important. But for AI as it exists today, it looks like uncertainty would add complexity, without producing any clear benefit. So I think it has been appropriate for most AI researchers to have postponed analyzing it so far. An aside: Russell points out that uncertainty provides an interesting way to avoid wireheading: if the reward is defined so that it can't be observed directly, then the AI will know that hacking the AI's signal won't create more brownie points in heaven. Feasibility? Russell is fairly convincing in his claim that AIs which are designed according to his rules will relatively safe. That's a much better achievement than most authors manage on this topic. I'm a bit less convinced that this approach is easy enough to implement that it will be competitive with other, possibly less safe, approaches. Some of my doubt derives from the difficulty, using current techniques, of encoding the relevant kind of abstract objectives into an AI. The objectives that Russell wants don't look much like the kind of objectives that AI researchers know how to put into an AI. It's fairly well known how to give an AI objectives either by using a large number of concrete examples of the "correct" result, or by specifying a readily quantifiable reward. Even a dilettante such as myself knows the basics of how to go about either of those approaches. In contrast, it's unclear how to encode an objective that depends on high-level concepts such as "human" or "preference that is inferred from behavior" without the AI already having done large amounts of learning. Maybe there's some way to use predictions about observed preferences as if the predictions quantified the actual objective? That looks partly right. But how do we tell the AI that the predictions aren't the real objective? If we don't succeed at that, we risk something like the King Midas problem: a naive new AI might predict that King Midas's preferences will be better satisfied if everything he touches turns to gold. But if that prediction becomes the AI's objective, then the AI will resist learning that the King regrets his new ability, since that might interfere with it's objective of turning anything he touches into gold. AI researchers have likely not yet tried to teach their systems about hard-to-observe concepts such as utopia, or heaven. Teaching an AI to value not-yet-observed preferences seems hard in roughly the same way. It seems to require using a much more sophisticated language than is currently used to encode objectives. I'll guess that someone would need to hard code many guesses about what human preferences are, to have somewhere to start, otherwise it's unclear how the AI would initially prefer any action over another. How is it possible to do that without the system already having learned a lot about the world? And how is it possible for the AI to start learning without already having some sort of (possibly implicit) objective? Is there some way to start a system with a much easier objective than maximizing human preferences, then switch to Russell's proposed objective after the system understands concepts such as "human" and "preference"? How hard is it to identify the right time to do that? I gather that some smart people believe some of these questions need to be tackled head on. My impression is that most of those people think AI safety is a really hard problem. I'm unclear on how hard Russell thinks AI safety is. It's quite possible that there are simple ways to implement Russell's rules, but I'm moderately confident that doing so would require a fairly large detour from what looks like the default path to human-level AI. Compare Russell's approach to Drexler's ideas of only putting narrow, short-term goals into any one system. (I think Drexler's writings were circulating somewhat widely before Russell finished writing Human Compatible, but maybe Russell finished his book before he could get access to Drexler's writings). If Drexler's approach is a good way to generate human-level AI, then I expect it to be implemented sooner than Russell's approach will be implemented. Still, we're still at a stage where generating more approaches to AI safety seems more valuable than deciding which one is best. Odds are that the researchers who actually implement the first human-level AIs will have better insights than we do into which approaches are most feasible. So I want to encourage more books of this general nature. Russell's rules show enough promise to be worth a fair amount of research, but I'm guessing they only have something like a 5% or 10% chance of being a good solution to AI risks. Miscellaneous Russell ideas often sound closer to those of Bostrom and MIRI than to those of mainstream AI, yet he dismisses recursive self-improvement and fast takeoff. His reasons sound suspicious - I can't tell whether he's got good intuitions that he has failed to explain, or whether he ignores those scenarios because they're insufficiently mainstream. Russell makes the strange claim that, because existing AI is poor at generalizing across domains, when people talk about "machine IQ" increasing rapidly and threatening to exceed human IQ, they are talking nonsense. But Russell seems to take the opposite position 100 pages later, when he's dismissing Kevin Kelly's The Myth of a Superhuman AI. I'm disappointed that Russell didn't cite the satire of Kelly that argues against the feasibility of bigger than human machines. Russell has a strange response to Bostrom's proposal to use one good AI to defend against any undesirable AIs. Russell says that we'd end up "huddling in bunkers" due to the "titanic forces" involved in battles between AIs. Yet Bostrom's position is clearly dependent on the assumption of a large power asymmetry between the dominant AI (or possibly a dominant coalition of AIs?) and any new bad AI - why would there be much of a battle? I'd expect something more like Stuxnet. There are lots of opinions about how much power disparity there will be between the most powerful AI and a typical new AI, and no obvious way to predict which one is correct. Russell says little about this issue. But suppose such battles are a big problem. How is this concern specific to Bostrom's vision? If battles between AI are dangerous to bystanders, what's the alternative to good AI(s) fighting bad AIs? Does someone have a plan to guarantee that nobody ever creates a bad AI? Russell shows no sign of having such a plan. Russell might be correct here, but if so, the issue deserves more analysis than Russell's dismissal suggests. Philosophy Russell concludes with a philosophical section that tackles issues relating to morality. It includes some good thoughts about the difficulties of inferring preferences, and some rather ordinary ideas about utilitarianism, including some standard worries about the repugnant conclusion. Here's one of Russell's stranger claims: in a sense, all humans are utility monsters relative to, say, rats and bacteria, which is why we pay little attention to the preferences of rats and bacteria in setting public policy. Is that why we ignore their preferences? My intuition says it's mostly because we're selfish and not trying to cooperate with them. I don't think I'm paying enough attention to their preferences to have figured out whether we're utility monsters compared to them. Conclusion I'll end with a more hopeful note (taken from right after Russell emphasizes that machines won't imitate the behavior of people they observe): It's possible, in fact, that if we humans find ourselves in the unfamiliar situation of dealing with purely altruistic entities on a daily basis, we may learn to be better people ourselves - more altruistic and less driven by pride and envy. Human Compatible will be somewhat effective at increasing the diversity of AI safety research, while heading off risks that AI debate will polarize into two tribes. See also this review from someone who, unlike me, is doing real AI safety research.

  30. 4 out of 5

    James Hendrickson

    Wow! This book is THE starting place for AI. I pretty much need to go back and read it again to absorb all that I missed the first time. If only my computer science classes had been like this....cross disciplinary, thoughtful, philosophical, practical, and technical. It’s like a liberal arts degree with a computer.

Add a review

Your email address will not be published. Required fields are marked *

Loading...
We use cookies to give you the best online experience. By using our website you agree to our use of cookies in accordance with our cookie policy.