Data center leaks

Leaking data center

We had a problem in our data center: water, where it does not belong!

Our data storage systems had water coming out of them.  Data isn’t wet; this was not a leak of private information, although it put data at risk in other ways.

The problem is that our roof is old, and it was raining.  Not particularly hard; our data center remained dry after Tropical Storm Irene (which brought widespread flooding after >10 inches of rain in the area).  But there is a seam in the roof just in behind these systems, and this was the time it started to leak.  The water you see actually comes from the other side of the rack.

Of course we shut down our systems — electronic systems can often survive water as long as they don’t have electricity flowing through them while they are wet.  We covered the areas where water seemed to be coming, and set up fans to help dry out wet electronics.  The normal intense air flow from the air handlers also helped to dry the systems, as well as the wet areas forming under the floor.

By the next morning, everything had dried sufficiently to bring the systems back on-line.  There were a few places where ceiling tiles had collapsed; these were soon repaired, and the data center once again in good condition.  No data were lost!wpid-IMAG1252


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Create a free website or blog at

Up ↑

%d bloggers like this: