The Anatomy of a Double Digg

So I happened to be talking to a friend who works for Apple, who was also a bit drunk AND mad at his boss who had been giving him some shit. It turns out he was willing to spill some info about upcoming products and didn’t seem to mind too much who found out (managers of employees with privileged info, remember not to be dicks). So while he (yes, Steve Jobs I’ve narrowed your employee search in half) was willing to give me some juicy stuff on some upcoming events, little did I know my ISP wasn’t up for it. 

I’d like to add my thoughts on Apple’s secrecy at this time: I understand their need for secrecy when it comes to proprietary hardware and software designs, however a majority of their secrecy is to intentionally drum up buzz about their products. Rumor sites are simply a byproduct of that. 9to5mac is just a hobby for us and we like to cover more grounded stuff usually. I don’t think we released anything that is going to give Apple’s competitors any extra time in “Copying” their intelectual material – either hardware or software.

So we wrote up some quick prose on the subject and a few friends posted them to Digg. The first one went main page pretty quickly- as in only an hour or two – I guess Sunday night has a slow news releases to high user volume ratio.  Perhaps to our luck, Apple also ran its first iPhone commercials on 60 minutes right after we posted the 1st story to Digg.  I am assuming people saw the ads, ran to their computers and some of them, looking for info on the iPhone, hit my story.  It made the front page of Digg.  I then linked to the second story and put a digg button on it.

We have made the digg front page before and didn’t really notice much traffic – perhaps because it was a less interesting story.

I was getting about 500 users at the same time. This was late in the night in the US on a Sunday so not too bad on the server which I share with some other (unfortunate) users. Within a few hours BOTH of the stories had gone front page. I was seeing over 1000 concurrent users. 

Suddenly instead of getting the story, users were getting the Drupal DB error page.

Excellent, So I tried calling my service providor, Site5 and wasn’t able to get through.  I decided to email them.  They rebooted the server within a few minutes and things were going well again until morning.  I had enabled caching on the Drupal server and except for a small but noticable amount of sluggishnes, it seemed to be handling itself well.

That all changed however, when it turned 9am on the East Coast. My concurrent user count went to 1500 and the sight REALLY started to slow down.  Way to hit digg first thing when you get to work slackers (oh wait, I am writing this from work!)

I went into the admin control panel to try to turn on a more aggressive caching and throttle some lessor needed modules but it was too late.  Someone else had, rightfuly I might add, complained that the server was toasting and one of the Admins had diagnosed the problem as my account and turned it off.  I immediately emailed them saying if they let me back in, I could trim the traffic and CPU cycles dramatically.   Then I got an email.

I immediately responded on how I could reduce traffic.  No response for over 6 hours from the guy who informed me that my account had been deactivated.  Had I lost some traffic?  Just a little.

I figure I lost about 50,000-100,000 or so pageviews but I am on a pretty inexpensive service that had, until I got double-dugg, worked flawlessly.  We only use poorly placed Google ads so that would have probably amounted to like $10.

Anyway, I kept emailing throughout the day and finally, as you can see above, at 4 EST they turned my account back online.  They had additionally put some forwarders in my .htaccess files that would point the diggs to an error page.  I went in and switched them back.  Traffic was at a 100 concurrent user pace for the rest of the evening and continues to slowly drop off until this writing.

Conclusion:
Don’t get double-dugg.  More specifically, if you don’t have a serious dedicated bandwidth and server capacity, don’t test out your setup this way – that’s why Dugg Mirror exists.  Most typical ISPs on typical plans would have done the same thing I am sure.  Grid Servers like Media Temple (which I think are slower but have more capacity from another website I work on) would have probably have handled the load but ended up having a serious fee attached.    Also the “fame” brings some nastieness not just from the hordes of Digg Trolls (and man there are a lot of useless Diggers – mostly the front page variety) but also from some more famous bloggers (who happen to be right – I did pull an image from Google image which was theirs – I didn’t recognize it because it was hosted on Blogsmith their Blogging Service Provider.)

(Don’t worry, its linked)