Artificial Intelligence Deprived of Self-Belief

Cybernetics Dylan Hadfield-Menell of the University of California at Berkeley and his colleagues decided that artificial intelligence, as we know it today, is too confident, and that it is time to teach the computer to doubt itself.

Creating artificial intelligence, according to the authors of the article (preprint is available on, a person should lay the possibility of emergency shutdown by human will in his architecture. The “uprising of machines”, which the researchers argue, can happen not because the computer will have its own will and desire to “live” and act freely, but because of the paradox that the machine will encounter. Imagine: a person tells the machine to shut down, but the machine is configured to solve some tasks, and shutting down means that the tasks will not be completed.

To develop the habit of AI in difficult situations to turn to people for help, scientists have created a mathematical model of the process of interaction between a person and a machine in a situation where a person can turn off the machine at any time, but the machine can turn off the switch. It turned out that it is possible to keep the computer from deactivating the “OFF” button by laying in the system a certain percentage of “uncertainty” about how some of its results are more important than others.

In other words, if the robot is confident that it is doing useful work that needs to be completed at all costs, it may not obey the order to stop the activity and turn off altogether. However, robots too should not be too insecure. Hudfield-Menell gives an example of an unmanned vehicle that takes a child to school: his on-board computer must under no circumstances obey the child’s request for a taxi. Safe AI must find a steady balance between self-confidence and the habit of relying on a person.

I wonder how a nuclear reactor works and can robots build a house?

All about new technologies and inventions! OK I agree to the rules of the site Thank you. We have sent a confirmation email to your email.


Why is the glass clear?
Is a warp engine possible?
Flying over water: a century of hydrofoil ships