The Data Center Diaries

A little over a week ago, I sent out one of the most difficult emails that Squarespace has ever delivered to our customers.

Peer1, our data center in downtown Manhattan, was so severely impacted by Hurricane Sandy that it suffered a total loss of power despite multiple levels of redundant systems. At the time, there was no resolution in sight. Our backup fuel reserves and building infrastructure had been destroyed by Sandy's storm surge, which flooded many buildings downtown. As you may be aware, this was a historic and unprecedented storm for the entire tri-state region, bringing about the largest storm-related power outage in Con Edison’s history.

I am proud to announce that throughout this event, Squarespace customers experienced absolutely no downtime related to the power outage. This is an amazing outcome considering the extraordinary circumstances we faced last week. What remains is an incredible story.

For those of you that haven’t been following our updates, employees from Squarespace, Fog Creek, and Peer1 manually carried fuel up 17 flights of stairs for three days to save our generator while an interim fuel supply and pump could be installed. These efforts to provide uninterrupted service for our customers were chronicled by numerous publications including All Things D, Fast Company, The New York Times, PandoDaily, and The Verge.

We now have a working pump system delivering fuel to the roof generator, more than enough fuel on site, and a redundant street-level generator connected and tested as of last night. These systems will remain in place for the foreseeable future. Our building has still not been able to connect to Manhattan’s power grid, as the building’s two sub-basements were submerged in 30 feet of water that took four days to pump out. We will continue to post updates on status.squarespace.com as we resume normal operations.

Of course, such heroics should not be necessary to keep operations running smoothly. We initiated a plan to build a geographically redundant operation this past summer and expect to have it online in early 2013. This gives us the ability to route around areas affected by natural disasters much more effectively.

We take the responsibility of running the hundreds of thousands of sites on Squarespace very seriously. Our homepage states that our scalable, reliable cloud infrastructure eliminates downtime, and our customers all over the world count on us to keep their websites online no matter what. Wanting to keep that promise is what propelled us forward and helped us persevere during this most challenging of times. Thank you all for being Squarespace customers – it is with your continued support that we can continue to fight for great design, amazing products, and exceptional service.

We know that there are many in our area that were impacted far beyond what we experienced – if you can, please take a moment and contribute to hurricane relief efforts. A little goes a long way.

Thank you,

Anthony