It's been nearly three weeks since Terry Childs was arrested on four counts of computer tampering and sent to jail on US$5 million bail. In those three weeks, this event has taken turns to the strange, and wound up firmly in the land of the absurd. From bombastic claims in the press to midnight visits by San Francisco Mayor Gavin Newsom to pages of functional usernames and passwords entered into the public record, this case has certainly proven engaging.

Lost in all the drama is what actually happened. How could a city government apparently lose control of its network, and how could its own characterizations of the system be so questionable?

I've been covering this case in my blog almost since day one, and have been trying to figure out exactly what happened, reading between the lines of published articles, and reading court documents until the wee hours of the morning. Here's what seems to be true, what is clearly open for question, and what lessons business IT should draw from this saga.

First, despite the many news reports claiming that Childs had shut down all or part of the city and county of San Francisco's network, what actually happened was that Childs refused to provide his superiors the passwords to the city's core FiberWAN network, effectively preventing them from administering the network. The network continued to function, and no city applications, data, or resources were lost or inaccessible.

Just who is Terry Childs, and why was he so powerful?

Terry Childs, a Cisco Certified Internetworking Engineer (certification number 14018), was a member of the San Francisco DTIS, the city's IT department, for the past five years. As a CCIE, Childs shares this distinction with only 16,000 or so others across the globe. He was part of the group that built and managed the city's networks, and in the past several years had been tasked with bringing together the many disparate networks that ran the city. As the city's most experienced and advanced network administrator, he essentially single-handedly designed and built the FiberWAN, a city-wide network built on fiber interconnects and MPLS. This network is complex, and forms the core of all city services.

Following the completion of the FiberWAN, Childs looked upon his creation as art -- so much so that he applied and was granted a copyright for the network design as technical artistry. Skeptical of his colleagues' abilities, Childs became the sole administrator of the FiberWAN, and the only person with the passwords to the routers and switches that comprised the network. This state of affairs was widely known throughout DTIS, and Childs was the only point of contact for changes, troubleshooting, and overall management of this network.

Sources have stated that not only was Childs the only admin, he was always on call, 24 hours a day, 7 days a week, 365 days a year. As the only admin with the knowledge and access to the FiberWAN, he had no help. During the past few years, the DTIS staff has been significantly reduced due to budget cuts, keeping the city dependent on a sole admin for its core network.

The confrontation that started the standoff

On Friday, June 20, there was an altercation between Childs and Jeana Pieralde, the new DTIS security manager at the 1 Market Street datacenter in San Francisco. The city's court filings claimed that Childs harassed Pieralde, confronted her, and took photos of her with his mobile phone. Fearing for her safety, Pieralde retreated to a room in the building, locked herself in, and called the DTIS CIO for help. The DTIS CIO then called Childs and the two had words. Childs subsequently left the premises. Why was Childs so upset? According to the city, no one had told him that Pieralde was auditing his network, and he perceived it as a threat or intrusion.