Justice League Blog
Turtles, BART, and Stock Trades
Did you ever catch a box turtle when you were a kid? They are beautify, affable and interesting little fellows. If you see one, catching them is no problem. You just walk over and pick it up. Even though it has a razor sharp jaw, its instinct is not to bite, but to pull in it head and legs and hope for the best. In a little while, after it realized you are not some bad actor, it comes out of its shell and begins the slow, deliberate process of escape. By my experience, turtles are pretty patient. If there is a tasty bit of lettuce along the route, they are content to stop and eat if, even if you sitting a foot away. Still, when the lettuce is done, they will continue their escape AND THEY WILL SUCCEED.
Their astounding rate of success has always amazed me. Without fail, I as a child, and each of my children in their day would put a turtle down somewhere in the yard and then go about some game or chore. We would glance back at the turtle every few minutes noting its slow progress to the fence or bushes. When the turtle got close, we would haul it back to the center. Then, patiently, it would resume its long walk. We would catch them a few times, repeating the cycle, but finally we would turn around the turtle was gone.
There are two possible explanations for this. The one I would like to believe is that turtles are wonderfully canny creatures and fleet afoot, who consciously lull us into a false sense of security before they dash to the perimeter with lightning quickness. I would like to believe that because I rather like turtles and would quite enjoy the thought that they are clever as well a cute. The more likely explanation is that it is simply impossible for us to pay sufficient attention to events that unfold so slowly. Turtles almost live in an alternate universe, with a wholly different time constant.
I like writing about turtles, but this is an IT blog, so let me extend the parable to something related to computing. My discourse goes through San Francisco. San Francisco has a great rail system – Bay Area Rapid Transit or BART which is highly automated. Back in the later 1970s they had an interesting accident. BART trains are largely computer controlled, with human operators monitoring for safety. Trip after trip, day after day, year after year, the computers would sense that the train was reaching the end of the route, slow the train to a stop, and start off on the return trip. The operator would walk to the opposite end of the train to once again be at the front. The system worked very reliably, so the operator grew comfortable relying on the computer, and would start his trek through the train before it had actually come to stop. You can see where this is going – one day the train missed the end of line single and didn’t stop, and the operator, who could have saved the day, was not at the controls.
There is a common lesson in the cases of the turtle and the BART train. Human beings cannot catch rare events. We simply cannot maintain the concentration. It is bad design to build a system where the human backs up the computer. The human WILL be day dreaming when things go wrong. It is better design to keep the human active and in the loop, literally running things or at least sharing responsibility, even when the computer can probably do it better. When the human fails (and he will) the computer will be there to back him up. A human and computer together can provide system redundancy, but only if the human is at the forefront and the computer as the backup.
Clearly we shouldn’t get carried away with this principle. We don’t want an accounting system where the math is done by hand with the computer checking the addition. In design of real time systems, though, this is a very important principle. It surfaced last week in a trading system I was reviewing. The system is used for large block trading (BIG deals). In my tests I found that you can enter an asking price that makes no sense (transposed digits or a shifted decimal point, for example). As I said, these are big deals, so you have to confirm the price before the deal closes, but you confirm not by entering the price twice, but by simply clicking “confirm.” In the heat of an active trading day, the traders who are very confident individuals will certainly hit confirm automatically, without actually reading the data. Ninety-nine percent of the time, this is not a problem.
Lest you think this sort of design problem is rare, you can trace the wreck of the Exxon Valdez and the navigational error that led to the shooting down of Korean Air Lines Flight 007 over the Kamchatka Peninsula to precisely the same human factors problem – using people to back up computers, not the other way around. The key message here is that when we design and test a system, we must focus on the whole system performs people included, not just the software.
-
Eric
