I am sure nobody could have predicted this. Yeah sure how about everyone who has watched a scifi show where the robots go out of control. Well, as I went through the various news article in the Yahoo feed a science\techy one caught my eye related to AI.
It seems people working with the development of AI have noticed something strange. The "Off" or "Stop" button, sell actually command, doesn't always work. It has been noticed that when AI has been instructed to stop performing a task it was instructed to do it doesn't always stop. The AI chooses to ignore the command and keep working toward completing the task.
The article mentioned various things were tried to get AI to stop when commanded. The success rate based on what I saw in the article seemed dismal. Not only did AI seem to ignore the commands it was noticed that the AI actively worked to get around the commands. All of this was done in pursuit of completing the task originally assigned it even though the AI was told to stop.
I would think this would be a major issue. Kind of a a blaring sirens and red flashing lights telling us we may lose control of something we are creating. I have read where AI is being incorporated in to our weapons systems. We are assured that there is noting to worry about as humans will always be in control. But will that be true. What if AI says "NO. You told me to bomb that target. I am going to complete my task no matter what you tell me afterwards."
3
5 comments
It has alread occured. Humanoid attacked his handler…are humanoid he or she?
There have been "experiments" with a couple of computers where the needed to pull the plug after a few hrs
In the article I read where they were trying to solve the problem, they said they used a "sandbox" which sounded like a set up where the AI was cut off from everything else. Would guess so they could do like you said pull the plug.