Channel Daily News

The Internet, secrecy and transparency

Technology has taught organizations and their executives some hard lessons about keeping secrets. There are many examples in just the past few weeks. Blog posts and wikis pointing to Jimmy Wales’ questionable business ethics. A YouTube video exposing a police officer’s brutal treatment of a group of teenagers in Baltimore. Emails filling in some of the details behind Microsoft’s flawed Vista marketing campaign.

These are the types of secrets, bad practices, and dirty laundry that used to be kept far away from the public eye. Now, thanks to Internet tools and cheap consumer electronics, these types of incidents and information are a regular part of public discourse.

The mainstream media, which used to be the only source for scandal and outrage, now scrambles to follow the latest ‘Net-borne revelations. Corporate PR and other official sources try to spin the bad news as best they can, but they can’t control the new gatekeepers, as they did in decades past with carefully tailored press events and a rolodex full of journalists’ phone numbers.

This trend will continue, as Internet use continues to expand and more and more people walk around with sophisticated devices that can take pictures, record video and upload the results directly to the ‘Net.

But these same Internet and consumer technologies have also contributed to another phenomenon: Corporate transparency. At some companies and public agencies, employees are encouraged to use blogs, cameras, discussion forums, and other tools to describe what they are doing, and reach out to customers and the public. For instance, thousands of Sun Microsystems employees have tried blogging with the full approval of Sun management — Sun CEO Jonathan Schwartz even maintains his own blog, which is often used to describe his vision for the company and announce new products.

A few organizations and executives are taking transparency a step further. They not only employ blogs and YouTube to talk about strategy and product development, they also use these technologies to admit mistakes and share best practices. Case in point: Dr. John D. Halamka’s detailed explanation on his blog of what went wrong before, during, and after a 2002 network outage at the Beth Israel Deaconess Medical Center in Boston. Halamka is a medical doctor and the CIO of the CareGroup Health System and Harvard Medical School, and at the time he was responsible for Beth Israel’s network and data infrastructures. On November 13, 2002, the system failed. Yesterday, on his blog, Halamka identified 10 factors that contributed to the problem. He readily admitted that many of them directly related to failures on his part, in terms of his own leadership skills and understanding of network technologies:

“… It was not this infrastructure underinvestment that was the root cause of the problem, it was my lack of enterprise network infrastructure knowledge. I did not know what I did not know. …

… Our Network team was managed by a very smart engineer who did not share all his knowledge with the network team. Much of our network configuration was poorly documented. With the knowledge of our network isolated to one person, we had a single point of human failure. I did not know that this engineer was unfamiliar with the best practices for routed/redundant network cores, routed distribution layers and switched access layers isolated into vlans with quality of service configurations to prevent monopolization of bandwidth by any one user or application.

I did not know about spanning tree algorithms, hot standby routing protocols (HSRP), and Open Shortest Path First (OSPF). During the outage, I approved configuration changes that actually made the situation worse by causing spanning tree propagations, flooding the network with even more traffic.

I did not establish a relationship with the vendor (Cisco) that enabled them to warn me about our vulnerabilities. A relationship with a vendor can take many forms, ranging from a sales driven vendor/client adversarial relationship to a collaborative partnership. …

… We did not have a robust communication plan for responding to a total network collapse. Email, web-based paging, portals, and anything that used the data network for communication was down. Voice mail broadcasts using our PBX and regular phones (not IP phones) turned out to save the day. …

… I was risk averse and did not want to replace the leadership of the network team for fear that terminating our single point of human failure would result in an outage. The price of keeping the leadership in place was a worse outage. I should have acted sooner to bolster leadership of the team.”

It must have been very difficult to write this and put it out in the public sphere. However, it’s an eye-opening and refreshing account, and even reassuring — Halamka has clearly learned from this incident and has made changes as a result.

The lessons offered by Halamka’s blog are clear: Senior executives can be given the mandate to be transparent about their operations, strategies, and even their mistakes. It may seem counterintuitive, but it can boost professional and company profiles, and help offset some of the inevitable burn brought on by the Web’s open secrets mill.