---
title: Flooded
categories:
- personal
content-tags:
- work
---

So, yesterday [we][1] discovered that our server room, built out quite nicely by a previous tenant, is located directly under an HVAC chiller.  The [landlord][2] didn't seem to know this, either.

This kind of equipment involves a cooling water line direct from the City main.  It broke.  And dumped a few thousand gallons of water onto running, servers.  At least a half million dollars of fragile electronics got doused with dirty (filtered by wood, insulation, ceiling tiles) water; even more was exposed to splash or sauna-like humidity.

My team rocks, several of us worked from lunch yesterday through lunch today drying out hard drives, setting up an emergency colocation (props to [Xiolink][3]) and confiscating a conference room for a makeshift server room.  It looks like we're going to be able to limp along for a bit on equipment that has silt in crevices and copper oxide bubbles on logic boards.  Hopefully the insurance companies won't cause delay while they argue over replacement.

   [1]: http://www.infuz.com/
   [2]: http://www.gmbcommercial.com/
   [3]: http://www.xiolink.com/
