Ceding Control to Machines That Can't Think

One of the more interesting contributions to this year's Edge question on machines that think comes from philosopher Daniel C. Dennett. Don't worry about the machines that can think just yet, he says, worry about ceding control to those that can't:

I think, on the contrary, that these alarm calls distract us from a more pressing problem, an impending disaster that won't need any help from Moore's Law or further breakthroughs in theory to reach its much closer tipping point: after centuries of hard-won understanding of nature that now permits us, for the first time in history, to control many aspects of our destinies, we are on the verge of abdicating this control to artificial agents that can't think, prematurely putting civilization on auto-pilot.

...

The real danger, then, is not machines that are more intelligent than we are usurping our role as captains of our destinies. The real danger is basically clueless machines being ceded authority far beyond their competence.

He has some examples, but my eye was caught by this Washington Post story of the Russian spy ring that just got busted. It seems that their priority target was uncovering means for:

“destabilization of the markets” and automated trading algorithms — “trading robots.”

As the story points out:

“The acceleration of Wall Street cannot be separated from the automation of Wall Street,” wrote Mother Jones’s Nick Baumann. “Since the dawn of the computer age, humans have worried about sophisticated artificial intelligence … seizing control. But traders, in their quest for that million-dollar millisecond, have willingly handed over the reins. Although humans still run the banks and write the code, algorithms now make millions of moment-to-moment calls in the global markets.”

For evidence of what can go wrong when one of these bots goes crazy, look no further than Aug. 1, 2013. That was when a mid-size trading company named Knight Capital Group lost nearly $10 million per minute over the course of 45 minutes for a total of $440 million. The managers said it was a computer glitch, a misfiring algorithm, a complex computer program gone rogue.

“The company said the problems happened because of new trading software that had been installed,” the New York Times reported. “The event was the latest to draw attention to the potentially destabilizing effect of the computerized trading that has increasingly dominated the nation’s stock markets.”

Of course the idea of rogue trading programs ravaging the markets is really the same story of unintended consequences as Skynet.

I think, though, that the problem is not so much that the machines don't think, as that they don't think like us.

Comments

Post a Comment

Popular posts from this blog

Anti-Libertarian: re-post

The most pointed critique of social security and liberalism in general is the libertarian critique. The problem with libertarians, for a liberal, is that we have too much in common. We both believe in individual rights, tolerance of individual differences, and dislike government prescription of religion. The basic difference, it seems to me, is the different answers we give to Cain's (with thanks to the Captain - my original version had the corpse asking the question!) famous question: "Am I my brother's keeper?" That's not really my basic bitch against the libertarians though. My real complaint is the same as my complaint against most religion - its premise is a fraud. For those who can't stand to wait for the punchline, I believe that trying to implement libertarian principles leads to tyranny or social disintegration. Demonstrating that takes some historical (and pre-historical) context. For all but the last 15,000 or so of the 100,000 years the human spec...
Read more

Uneasy Lies The Head

Uneasy lies the head that wears a crown ........... Shakespeare Henry IV, Part II. So it is with us and our slightly distant primate cousins , so it seems. A 9-year study of wild baboons in Kenya by Princeton University and the Institute of Primate Research may turn conventional assumptions of alphas males upside down. The study found that alpha males have high testosterone levels, which allows them to dominate other baboons, get access to more food, and attract better mates. This is expected. Surprisingly, though, the alpha males also have high levels of stress hormone glucocorticoid, which scientists measured from their feces samples. In fact, the stress of alpha baboons was on par with low-ranking baboons. The convention wisdom is that access to food and females provides life security and thus less stress for dominant males. Conversely, weak males who are constantly threatened by starvation and physical violence from stronger males are thought to be more stressed. The st...
Read more

We Call it Soccer

The US played its best, and maybe only good game of World Cup, but lost thanks to brilliant goalkeeping by the Swedish keeper and the bizarre method soccer uses to settle ties when three US stars could not put the ball on frame from the 12 yard spot.  At the end it came down to a blocked shot that spun in by maybe a couple of centimeters. Well, that's football - too much like life.   Which brings me to a modest proposal.  Replace the thirty minute overtimes with as many 7-player 15 minute periods as needed to settle it.  Scoring should be much easier with fewer players.
Read more