Please create an account to participate in the Slashdot moderation system

 


Forgot your password?
Close
typodupeerror
×
Cloud Networking Social Networks Technology

How MySpace Generates Enough Load To Test Itself 65

Posted by timothy from the advertises-free-riding-lessons-and-crack dept.
An anonymous reader points out this article about "...how a big site like MySpace uses thousands of cloud computing cores to do performance testing on its live site. There are some really great numbers in there from the performance tests, like generating 16GB/second of bandwidth and 77,000 hits/second during testing (not including the live traffic on the site at the time)."
This discussion has been archived. No new comments can be posted.

How MySpace Generates Enough Load To Test Itself More

How MySpace Generates Enough Load To Test Itself

Comments Filter:

  • Wait, what? (Score:1, Insightful)

    by girlintraining ( 1395911 )

    The real news here is that people still use myspace. -_-

    • Re: (Score:2)

      by Pojut ( 1027544 )

      Agreed.

      Is there ANYONE on here that still uses Myspace?

    • Re: (Score:2, Interesting)

      by ircmaxell ( 1117387 )
      The real news (to me at least) is that they use 77k hits per second to TEST. Normally, when I test at least, I try to go for at least 150% to 200% expected peak load. Assuming that they aren't doing this during peak hours, that puts their expected PEAK load somewhere around 100k hits per second or so... Large, yes. Large for that size of site? No way... Using RMS, I'd say that they are averaging around 25k to 35k hits per second IF they are testing for a 100k peak... That puts MYSPACE (once one of th

      • Re: (Score:2, Informative)

        by DIplomatic ( 1759914 )
        I'm not sure where you're getting 77k hits per second. FTFA: "The goal was to test an additional 1 million concurrent users on their live site stressing the new video features. The key word here is ‘concurrent’. Not over the course of an hour or day 1 million users concurrently active on the site."
        • FTS:

          77,000 hits/second during testing

          (77000 hits/second) / 1 million users = 4.6 hits/minute per user (That's a little bit higher than the figures from my experience, but still well within reason)...

          • Re:Wait, what? (Score:4, Informative)

            by Volante3192 ( 953645 ) on Thursday March 04, 2010 @05:11PM (#31363578)

            They were adding 77k hits/sec to their live traffic, not testing against 77k hits/sec.

            I.e., if 4 mil live users were hitting MySpace during the test, MySpace's servers were actually feeling the impact of 5 mil.

            • Correct. Based on my assumption previously that the site was at less than peak load during testing (let's call that initial load: x) here's how I calculated my figures...

              x + 77k = testpercentage * peakload

              Two things become apparent here. First is the assumption that x 1 (or in otherwords x + 77k > peakload). Why this assumption? Now, I assumed that they were doing capacity tests (they could be doing functionality tests, but I got the impression from the summary and TFA that it was capacity testing

              • This is testing just one part of the site (streaming video in NZ no less!), so you can't make wild generalizations based on those numbers if you expect any accuracy at all. You're not even going to get a 'rough' estimation.

                For example, viewmorepics.myspace.com might do X req/s duting peak and home.myspace.com, www.myspace.com or music.myspace.com might do something wildly different because they have completely different traffic patterns.

              • From my viewpoint, which is likely more amateur armchair as it were, it just sounded like they were testing the extra load, not so much peak usage. And even then they were having to shuffle load around.

                One could question the logic of trying to stress test your live servers when you're the size of MySpace as well.

                Of course it could have been a third thing they were trying to accomplish as TFA was pretty weak.

      • Re: (Score:2)

        by cgenman ( 325138 )

        They wanted to supplement the live traffic with test traffic to get an idea of the overall performance impact of the new launch on the entire infrastructure.

        So the 77k hits per second wasn't their expected peak load, but their expected delta in peak load after opening online streaming in... (looks up) New Zealand. 77k hits per second for streaming in New Zealand doesn't sound that far off.

      • The real news (to me at least) is that they use 77k hits per second to TEST.

        According to TFA - the 'test' users were in addition to the normal users, simulating the additional load of the new services. So actual load on the production servers was something well north of 77khits/sec.

    • Re:Wait, what? (Score:5, Funny)

      by Arthur Grumbine ( 1086397 ) on Thursday March 04, 2010 @05:45PM (#31364010) Journal

      I'm pretty sure this test was just for nostalgic purposes:

        MySpace Admin #1: Remember when we used to have millions of hits per hour?

        MySpace Admin #2: Yeah... those were the days. I was, like, a rockstar to my friends.

        MySpace Admin #1: Yeah, my friends thought I had the coolest job in the world, working for MySpace.

        MySpace Admin #1 and #2: *sigh*

        MySpace Admin #2:...hey - I know! We could pay some company to run a load test, it'll be just like the good ol' days!

        MySpace Admin #1: Yeah!!

    • Re: (Score:1)

      by s2theg ( 1185203 )
      Not impressed. I normally generate enough load to finish before the next page even loads.

    • The funny part is that I read the summary and the article, then came and read your comment and it was the first time I realized they were talking about MySpace and not Facebook.

      Funny that they are one and the same to me now.

    • Letting laymen edit HTML always worked out for the best.

      Although the Myspace Worm [namb.la] has to be one of the most hilarious things I've ever read.

    • MySpace still exists?

  • They use to mask the reality (Score:1, Funny)

    by Anonymous Coward

    That the site only has one actual visitor anymore.

  • Anyone else think from the title that MySpace has enough load (how, I don't know because who even uses it anymore) to consistently test its capacity?
  • /. people should know better, especially anonymous readers

    • Re: (Score:1)

      by xOneca ( 1271886 )
      16 gigabits per second... 16 gigabytes per second... Doesn't matter: my server would be DoSed at only 1mb/s...

  • who generates enough loa....

    oh geeze. nevermind.

  • They did it (Score:5, Informative)

    by OverlordQ ( 264228 ) on Thursday March 04, 2010 @04:55PM (#31363396) Journal

    by outsourcing to This Company. In additon, This Company used Stuff to do Things. After initial tests, This Company did Other Things. This Company is a leader in stuff, especially utilizing their software This Stuff. Try This Stuff Today!

  • 16 Gigabits/Second (Score:3, Informative)

    by TheNinjaroach ( 878876 ) on Thursday March 04, 2010 @04:58PM (#31363422)
    I thought that 16GB/sec seemed a little high so I checked the article. The actual network load they generated is 16 gigabits per second using 800 instances of Amazon's EC2.

    • Re: (Score:1, Informative)

      by Anonymous Coward

      I'm sorry.. WHAT!?

      16Gbit/second isn't much. It's hardly more than we handled at my previous company. A relatively small national hosting company in northern europe.

      Not for testing, mind you, but 16Gbit/sec is not a lot of bandwidth. You can easily handle it with say, ~50 boxes. Considering the million of requests myspace, not even amazon, gets per second, plus all the computation they have to do in the background, it hardly seems like a .. big deal.

      We're talking about 800 machines, with 3.2Kcores or so.

  • Gotta be prepared (Score:5, Funny)

    by Tobor the Eighth Man ( 13061 ) on Thursday March 04, 2010 @05:02PM (#31363470)

    MySpace is wise to do this kind of testing and load balancing. You never know when a twelfth person might attempt to connect to the site, throwing their carefully laid plans into total chaos.

    • MySpace is wise to do this kind of testing and load balancing. You never know when a person might attempt to connect to the site, throwing their carefully laid plans into total chaos.

      FTFY

  • Pretty stupid. They could "crowdsource" by simply challenging slashdot to a duel. We'll turn your servers into smoking rubble facespace BRING IT!

  • Add revenue (Score:5, Interesting)

    by KevMar ( 471257 ) on Thursday March 04, 2010 @05:14PM (#31363632) Homepage Journal

    I wonder how much add revenue they generated from all those impressions?

    does testing on production increase your page views?

    • does testing on production increase your page views?

      "Oops, our 2009 Quarter Four revenues look a bit down ... time for some good ole fashioned load testing (ifyaknowwhatImean)."

      Sure would explain how it's still hanging around.

  • while the article is biased, most of it is true (Score:4, Insightful)

    by mcguyver ( 589810 ) on Thursday March 04, 2010 @05:27PM (#31363768) Homepage
    The article was written by an employee of the company that sold their testing software/services to MySpace. Of course they're going to have glowing reviews about their testing tools. With that said the author is right. Capacity testing in production is needed for high availability sites. Verifying your real time monitoring tools is also important.
  • You would never know it based on the times I've been there...

  • MySpace Planning Meeting (Score:5, Funny)

    by eldavojohn ( 898314 ) * <eldavojohnNO@SPAMgmail.com> on Thursday March 04, 2010 @05:53PM (#31364130) Journal
    MySpace Engineer: Which brings us to the issue of load testing.
    MySpace Exec: So do it.
    MySpace Engineer: Well, we can't.
    MySpace Exec: Am I missing something here? You just got done showing how stupid our users are. So just simulate them.
    MySpace Engineer: Look, sir, with all due respect, we can put lol-bots up to post crap it's just that we have no way of mimicking that amount of garbage.
    MySpace Exec: Well how much is it?
    MySpace Engineer: Let me remind you, our previous slides showed you the magic of the MySpace machine--millions of users putting garbage in with the result being unadulterated horse shit flying out of the site. But to load test we need a lot of garbage. Several billion metric tonnes of garbage. Otherwise we just wouldn't produce the same amount of browser destroying horse shit we produce at peak loads.
    MySpace Exec: Have you spoken with the City of New York?
    MySpace Engineer: Sir, twenty five New York Cities wouldn't produce the amount of garbage we need.
    MySpace Exec: Holy shit.
    MySpace Engineer: Yes, this indeed requires a shitstack of biblical proportions.
    MySpace Exec: What're we gonna do?
    MySpace Engineer: Well, to solve this problem we turned to the motherload of bullshit. The one thing that everyone keeps endlessly spewing garbage about.
    MySpace Exec: The Cloud!
    MySpace Engineer: Bingo.

  • Just post a story here on slashdot that there is free porn.

  • Since the MySpace's crapnet is completely covered with trojans, if MySpace blows their load tests, does this mean they are open to successful backdoor penetration & injection attacks?

Slashdot Top Deals

As you will see, I told them, in no uncertain terms, to see Figure one. -- Dave "First Strike" Pare

Close