Yesterday afternoon, during planned maintenance that was not intended to interrupt service, an issue arose that took down a critical database cluster. This brought down our entire network while our engineers worked feverishly to restore these databases and bring your blogs back online.
While you might feel like you’ve gotten used to seeing errors on Tumblr recently, know that this is absolutely unacceptable to our team, and unacceptable for a platform determined to be the best place in the world for your creative expression.
Frankly, keeping up with growth has presented more work than our small team was prepared for — with traffic now climbing more than 500M pageviews each month. But we are determined and focused on bringing our infrastructure well ahead of capacity as quickly as possible. We’ve nearly quadrupled our engineering team this month alone, and continue to distribute and enhance our architecture to be more resilient to failures like today’s.
We can’t apologize enough, nor can we thank you enough for putting up with these growing pains. We know how impossibly frustrating it is to see your work offline. But please always know that we truly care about your work as much as you do, and we have an incredibly capable team working incredibly hard to take good care of it.
Sorry we let you down today.
P.S. If our staff blog is ever unreachable, you can check our Twitter stream for updates.
P.P.S. Anyone that had a paid Directory Feature running through the last two days has had it automatically extended 36 hours.