Forgot your password?
typodupeerror
Microsoft Perl

Microsoft Bots Effectively DDoSing Perl CPAN Testers 332

Posted by timothy
from the stuck-in-a-rut dept.
at_slashdot writes "The Perl CPAN Testers have been suffering issues accessing their sites, databases and mirrors. According to a posting on the CPAN Testers' blog, the CPAN Testers' server has been being aggressively scanned by '20-30 bots every few seconds' in what they call 'a dedicated denial of service attack'; these bots 'completely ignore the rules specified in robots.txt.'" From the Heise story linked above: "The bots were identified by their IP addresses, including 65.55.207.x, 65.55.107.x and 65.55.106.x, as coming from Microsoft."
This discussion has been archived. No new comments can be posted.

Microsoft Bots Effectively DDoSing Perl CPAN Testers

Comments Filter:
  • by drinkypoo (153816) <martin.espinoza@gmail.com> on Monday January 18, 2010 @08:50AM (#30806824) Homepage Journal

    Anyone know what sites on Microsoft's front-facing sites are most computationally intensive, and yet always dynamically generated? :D

    • Re: (Score:2, Interesting)

      by Anonymous Coward

      Bing? ...But that would only help them to DDoS Bing.

      • Re: (Score:3, Funny)

        by jisatsusha (755173)
        All that'd serve to do is make them look more popular than ever. Traffic up 300%! Sounds like a good mar
        • Re: (Score:3, Funny)

          by Anonymous Coward

          That exactly what i said. Dont you dare leech the score from me jackass!

    • Re: (Score:3, Insightful)

      by Lennie (16154)
      http://blogs.msdn.com/

      I've seen it fail many times
      • Re: (Score:3, Funny)

        by mmontour (2208)

        Mission accomplished. I got this on the second link that I clicked.

        We are currently unable to serve your request
        We apologize, but an error occurred and your request could not be completed.
        This error has been logged. If you have additional information that you believe may have caused this error please report the problem here.

    • Re: (Score:3, Insightful)

      by SharpFang (651121)

      No, we just make mistakes writing our Perl programs for automatic downloading stuff from MSDN. Like, download() unless success, and forget to set success=true;

    • by jlp2097 (223651) on Monday January 18, 2010 @09:31AM (#30807138) Homepage Journal

      Not necessary. A Bing Product Manager has already commented on the CPAN Testers blog entry [perl.org] upon which the article is based:

      Hi,
      I am a Program Manager on the Bing team at Microsoft, thanks for bringing this issue to our attention. I have sent an email to barbie@cpan.org as we need additional information to be able to track down the problem. If you have not received the email please contact us through the Bing webmaster center at bwmc@microsoft.com.

      As said below, never ascribe to malice that which can be adequately explained by stupidity. (Insert lame joke about MSFT being full of stupidity here).

      • Seems like the CPAN admin has already solved the "issue".

      • by kulnor (856639) on Monday January 18, 2010 @09:42AM (#30807224)
        Well, with Barbie(TM) on the case, this should be quickly resolved (unless she's too busy with G.I.Joe(TM))
      • by Anonymous Coward on Monday January 18, 2010 @10:55AM (#30807924)

        "as we need additional information to be able to track down the problem."

        IP addresses aren't enough? You're MS--if you can't fix the problem and IP addresses are given, damn, that's just sad. You're freaking massive multi-billion dollar tech companies, and this is the best you can do?

        No wonder Chinese hackers own our asses.

        Then again, it took Comcast 9 months to fix a security hole in customer accounts (which would have required an s to http to make pages SSL'd), and the only reason it was "fixed" was because they did their annual website makeover and changed their entire system to something Flash based. Then again, I had contacted a VP, VP's security, referred to web security, and talked to web security 3x, talked to a manager. The last 3 groups verified the problem. It was referred to their web applications team by that point, who sat on it.

        Lovely world we live in.

        • Mod parent up (Score:4, Insightful)

          by Lonewolf666 (259450) on Monday January 18, 2010 @11:55AM (#30808568)

          While he could be more polite, it is indeed embarrassing for Microsoft if they cannot check their own network
          a) for the existence of computers with given IPs
          b) what these computers are doing

          I think that deserves an "insightful" that cancels out the "flamebait".

        • Re: (Score:3, Funny)

          by gbjbaanb (229885)

          IP addresses aren't enough? You're MS--if you can't fix the problem and IP addresses are given, damn, that's just sad. You're freaking massive multi-billion dollar tech companies, and this is the best you can do?

          I've seen and used Vista. The answer to your question is "yes".

      • Re: (Score:3, Insightful)

        by Penguinisto (415985)

        As said below, never ascribe to malice that which can be adequately explained by stupidity. (Insert lame joke about MSFT being full of stupidity here).

        Given the back-story on the whole Danger data loss affair [arstechnica.com], stupidity is the FIRST thing I'd ascribe to Microsoft these days...

      • by Short Circuit (52384) <mikemol@gmail.com> on Monday January 18, 2010 @11:23AM (#30808224) Homepage Journal

        A quick guess? Identifying unique sites by domain name, rather than by IP address, and either the bot or server not respecting HTTP 301 redirects.

        With Rosetta Code, I once had www.rosettacode.org serving up the same content as rosettacode.org. My server got pounded by two bots from Yahoo. I could set Crawl-Delay, but it was only partially effective; One bot had been assigned to www.rosttacode.org, while another to rosettacode.org, and they were each keeping track of their request delay independently. I've since corrected things such that www.rosettacode.org returns an HTTP 301 redirect to rosettacode.org, and have was eventually able to remove the Crawl-Delay entirely.

        I've since worked towards only serving up content for any particular part of the site on a single domain name, and have subdomains such as "wiki.rosettacode.org" redirect to "rosettacode.org/wiki", and "blog.rosettacode.org" to "rosettacode.org/blog". Works rather nice, though it does leave me a bit more open to cookie theft attacks.

        YMMV; As I said, that was a quick guess.

      • by jc42 (318812) on Monday January 18, 2010 @11:35AM (#30808346) Homepage Journal

        As said below, never ascribe to malice that which can be adequately explained by stupidity. (Insert lame joke about MSFT being full of stupidity here).

        Yeah, though this particular sort of stupidity has been going on for a long time, and not just at Microsoft (though they seem to be the worst culprit).

        I run a couple of sites that, among other things, has links to return the "content" in a list of different formats (GIF, PNG, PS, PDF, ...). Periodically, the servers get bogged down by search sites hitting them many times per second, trying to get every file in every format. The worst cases seem to come from microsoft.com and msn.com, though it happens with other search sites, too. Actually, the first attempts I saw at "deep search" like this came from googlebots around 10 years ago, though they quickly backed off and haven't been a serious problem since then. MS-origin "attacks" of this sort have been happening every few months, for nearly a decade.

        I've generally handled them with a couple of techniques. One is to check the logs for successive requests from the same address, and insert sleep() calls with progressively longer sleeps as more messages arrive. The code prefixes the "content" with a comment explaining what's happening, in case a human investigates.

        Another technique is to look for series of "give me this in all your output formats" requests, verify that it's a search bot, and add the address to a "banned" list of sites that simply get a message explaining why they aren't getting what they asked for, plus an email address if they want to get in contact. So far nobody at any search site has ever used that address. I did once get a response from a guy who was studying sites with such multi-format data, for a school project, to see how the various output formats compared in size and information content. I took his address off the banned list, and suggested that he add a couple-second delay between requests, and he finished his project a few days later.

        I suspect that the googlebot folks may have read my explanation of the delays and added code to spread their requests out over time, since that's what their bots seem to do now. But I never heard from them. They must have gotten complaints (and bans) from lots of web sites when they started doing this, so they probably realized quickly that they should add code to prevent such flooding of sites.

      • Re: (Score:3, Insightful)

        by Alpha830RulZ (939527)

        You know, it's easy to poke fun at the Microsofty, but is it possible that he was just trying to find out what was being hit so that he could figure out who in his organization he should contact? Maybe there is some uber technical way he could have figured this out, or maybe he should have RTFB, but his response sounded well intentioned and responsive. What would you prefer? The microsoft of old?

    • by Anonymous Coward on Monday January 18, 2010 @09:42AM (#30807218)

      As much spam as I get from ir@infousa.com , I wish that someone would DDOS that damned company. If I knew of a way to get extra spam to ir@infousa.com I would probably do it so that company could get a taste of its own medicine. ir@infousa.com sent me unsolicited spam and it drives me nuts. Thanks for nothing, ir@infousa.com . It makes me want to call the company at (402)593-4500 and complain, but I don't have time. I guess I'll email them at ir@infousa.com instead. maybe.

    • by PetoskeyGuy (648788) on Monday January 18, 2010 @10:08AM (#30807434)

      Why make things worse? Block the ip address or range and notify the admins. This isn't a chan mob.

  • Until I read the summary I thought it was another article about windows botnets and was wondering why the "microsoft" was tacked on since windows is the default OS assumption. Of course it would be interesting if these were new CPAN mirrors that MS was settings up.
  • Sooooo, lets all go to the testers blog and DDOS that too. Dumbass...
  • I've seen it before (Score:5, Interesting)

    by LordAzuzu (1701760) on Monday January 18, 2010 @08:54AM (#30806860)
    I manage some networks in my home city in Italy, and in the past year I've often seen strange traffic coming from some of their IP addresses. Guess they have been exploited by someone long time ago, and didn't even notice it.
    • by beadfulthings (975812) on Monday January 18, 2010 @10:51AM (#30807866) Journal

      It's interesting to read this, as I've had some random and somewhat incomprehensible port scans coming from an IP address identified as one of theirs. If you're just an insignificant slob, you can't write to their abuse address, either; you'll get bounced. I simply blocked that particular IP address. Let them worry about who's gotten to them.

  • by strredwolf (532) on Monday January 18, 2010 @08:58AM (#30806900) Homepage Journal

    Looks like Microsoft's Bing managers are on it. They'll make it worse in no-time flat. :)

    BTW, the difference between a DDOS and a Slashdotting? You know why your site went down -- you got linked!

    • by Anonymous Coward on Monday January 18, 2010 @09:22AM (#30807082)

      BTW, the difference between a DDOS and a Slashdotting?

      The DDOS bots actually read TFA.

  • MS ineptitude? (Score:2, Insightful)

    by Anonymous Coward

    From TFA:

    Hi,
    I am a Program Manager on the Bing team at Microsoft, thanks for bringing this issue to our attention. I have sent an email to nospam@example.com as we need additional information to be able to track down the problem. If you have not received the email please contact us through the Bing webmaster center at nospam@example.com.

    I mean, what additional information is needed wrt "respecting robots.txt" and "not letting loose more than one bot on a site at a time"?

    Bing. Meh.

  • by tjstork (137384) <todd,bandrowsky&gmail,com> on Monday January 18, 2010 @08:59AM (#30806910) Homepage Journal

    I know everyone likes to assume that Microsoft is being evil here, but wouldn't the more realistic assumption be that they were just being incompetent?

    • by Lloyd_Bryant (73136) on Monday January 18, 2010 @09:06AM (#30806976)

      I know everyone likes to assume that Microsoft is being evil here, but wouldn't the more realistic assumption be that they were just being incompetent?

      Sufficiently advanced incompetence is indistinguishable from malice. For additional examples, see Government, US.

      The simple fact is that ignoring robots.txt is effectively evil, regardless of the intent. It's not like robots.txt is some new innovation...

      • by gmuslera (3436) on Monday January 18, 2010 @09:15AM (#30807026) Homepage Journal
        They are not ignoring robots.txt, probably just that they understand that file in their slighly different, but in the end incompatible, format. As every other file.
      • The simple fact is that ignoring robots.txt is effectively evil, regardless of the intent. It's not like robots.txt is some new innovation...

        Since when did Microsoft feel existing standards were something to honour? How many times have its browsers changed behaviour? Re-defined entrenched URL standards (you cannot specify username/password in an Internet Explorer URL but this is a legal standard form of URL)?

        It stands to reason Microsoft would take no notice of anything your website has to say.

        Unless.. of course.. Microsoft define a certificate type that can sign your Microsoft-specific format exception list after payment on an annual licens

        • by blueZ3 (744446) on Monday January 18, 2010 @10:36AM (#30807710) Homepage

          What's amusing about the issue in the kb is that the problem that they're "solving" by breaking the username/password in a URL standard is NOT a problem with username/password URLs, but a problem with how IE displays the URLs. In other words, rather than fixing the behavior of IE's address and status bars to display such URLs correctly, they just stopped supporting them.

          Incompetence at that level isn't just indistinguishable from malice, it IS malicious.

      • by Suki I (1546431) on Monday January 18, 2010 @09:35AM (#30807164) Homepage Journal
        Try saving a copy as robots.docx and see if that works ;)
    • by fish waffle (179067) on Monday January 18, 2010 @09:06AM (#30806982)

      I know everyone likes to assume that Microsoft is being evil here, but wouldn't the more realistic assumption be that they were just being incompetent?

      Probably. But since incompetence is the plausible deniability of evil it's sometimes hard to tell.

      • by paiute (550198)

        I know everyone likes to assume that Microsoft is being evil here, but wouldn't the more realistic assumption be that they were just being incompetent?

        Probably. But since incompetence is the plausible deniability of evil it's sometimes hard to tell.

        "incompetence is the plausible deniability of evil"

          fish waffle, that is great sig material.

    • by mspohr (589790)
      Occam's razor (or Ockham's razor[1]), entia non sunt multiplicanda praeter necessitatem, is the principle that "entities must not be multiplied beyond necessity" and the conclusion thereof, that the simplest explanation or strategy tends to be the best one.

      Rough translation: "Never ascribe to malice that which can be adequately explained by stupidity."

      • by MrMr (219533) on Monday January 18, 2010 @09:19AM (#30807058)
        The problem is, there is no evidence that:
        Never ascribe to stupidity that which can be adequately explained by malice.
        Is invoking more entities.
        In fact, claiming that the commercially most successfull software company got there through stupidity rather than malice sounds extremely implausible to me.
        • by Opportunist (166417) on Monday January 18, 2010 @09:38AM (#30807186)

          Like my grandpa said, it doesn't matter how dumb you are. As long as you find someone even dumber to sell to.

    • Re: (Score:3, Insightful)

      by alexhs (877055)

      these bots 'completely ignore the rules specified in robots.txt.'

      Microsoft ignoring standards is not incompetence, it's policy (NIH syndrome).

    • by djupedal (584558) on Monday January 18, 2010 @09:12AM (#30807012)
      > "I know everyone likes to assume that Microsoft is being evil here, but wouldn't the more realistic assumption be that they were just being incompetent?"

      We assume MS is evil...

      We know they are incompetent.

      We feel this is typical.

      We pray they'd just go away.

      We think this will never end...
    • by Yvanhoe (564877) on Monday January 18, 2010 @09:16AM (#30807034) Journal
      There is such thing as criminal incomptence. If a script kiddie can be arrested for having a virus "out of control" I don't see why Microsoft engineers DDOSing a website couldn't be charged.

      By the way a philosopher once told that "evil" did not exist. That it was most of the time just a kind of hidden stupidity.
      • by hairyfeet (841228) <[bassbeast1968] [at] [gmail.com]> on Monday January 18, 2010 @10:07AM (#30807428) Journal

        But MSFT is a corporation, which thanks to our corporate butt kissing congress and courts can just go "ooopsie", maybe cut a small check at most, and walk away scott free.

        And as for your philosopher? I saw an interview with Joss Whedon on writing evil characters that I thought really hit the nail on the head. He said, and I paraphrase "The villain never sees himself or herself as evil. To them there is a perfectly justifiable reason for their actions. I have known some truly evil people, those that have intentionally hurt their fellow man out of pure malice, and to them their actions were justified and noble. They simply didn't see what they did as wrong."

        Which is how you get MSFT and Intel paying backroom deals to crush competition, or Jack Trammell and his "business is war" philosophy. To the ones making the decisions "the other guy would do it to us if they could, so why shouldn't we do it to them?". I'm sure that if you talked to Gates or the head of Intel you could never get them to believe that crushing your competition any way you can is wrong. To them that was/is business 101 and not evil. That is why I think Whedon was right, the villain always thinks they are noble.

    • Yes, Evil more so
    • AFAIK, the one doesn't exclude the other.

      However, assuming evil is more fun :-)

    • by Xest (935314)

      Yes, and I like the solution too- rather than contact Microsoft to find out what the fuck is going on, post it to Slashdot and get Slashdotted as well.

      Pure genius.

  • Its not a bug, its a feature to index a site with a new, rapid, powerful, direct, personalised crawler :)
    http://arstechnica.com/microsoft/news/2010/01/microsoft-outlines-plan-to-improve-bings-slow-indexing.ars [arstechnica.com]
  • by Anonymous Coward on Monday January 18, 2010 @09:11AM (#30807010)

    I had a registration page - static content basically. The only thing that was dynamic was that it was referred to by many pages on the site with a variable in the querystring. Bing decided that it needed check on this one page *thousands* of time per day.

    They ignored robots.txt.
    I sent a note to an address on the Bing site that requested feedback from people having issues with the Bing bots - nothing.

    The only thing they finally 'listened' to was placing "" in the header.

    This kind of sucked because it took the registration page out of the search engines' index, however it was much better than being DDOS'd. Plus, the page is easy to find on the site so not *that* big a deal.

    Bing has been open for months now and if you search around there are tons of stories just like this. Maybe now that a site with some visibility has been 'attacked', the engineers will take a look at wtf is wrong.

    • Seems like a better solution would have been to setup a test for the either the User-Agent, or the IP/blocks that Bing was attacking your site from, and dropping those requests in /dev/null - your site would still exist on 'real' search engines, and Bing doesn't pound on your bandwidth anymore.

      • Replying to myself: if testing the UA or the IP in the httpd itself was too much load, you could have also just nullrouted the IP blocks the Bing spider was coming from, either in the kernel table, or in your router.

  • Flooding... (Score:5, Informative)

    by Bert64 (520050) <bertNO@SPAMslashdot.firenzee.com> on Monday January 18, 2010 @09:15AM (#30807030) Homepage

    I have noticed the microsoft crawlers (msnbot) being fairly inefficient on many of my sites...
    In contrast to googlebot and spiders from other search engines msnbot is far more aggressive, ignores robots.txt and will frequently re-request the same files repeatedly, even if those files haven't changed... Looking at my monthly stats (awstats) which groups traffic from bots, msnbot will frequently have consumed 10 times more bandwidth than googlebot, but is responsible for far less incoming traffic based on referrer headers (typically 1-2% of the traffic generated by google on my sites).

    Other small search engines don't bring much traffic either, but their bots don't hammer my site as hard as msnbot does.

  • Are you sure? (Score:5, Insightful)

    by Errol backfiring (1280012) on Monday January 18, 2010 @09:21AM (#30807070) Journal
    Are we sure this traffic comes from Microsoft? Could it not consist of forged network packets? You don't need a reply if you are running a DDOS. On the other hand, why would anyone, including Microsoft, want to bring down CPAN?
    • Re: (Score:3, Funny)

      by Anonymous Coward

      Because they are coming out with P# and don't want the competition?

    • Re: (Score:2, Informative)

      by Anonymous Coward

      You only see an IP in an apache log after a successfull TCP handshake. This is hard (not impossible, but really, really hard) to do with a forged IP.

    • Re:Are you sure? (Score:5, Informative)

      by TheRaven64 (641858) on Monday January 18, 2010 @09:45AM (#30807246) Journal

      Are we sure this traffic comes from Microsoft? Could it not consist of forged network packets?

      It's a TCP connection, so they need to have completed the three-way handshake for it to work. That means that they must have received the SYN-ACK packet or by SYN flooding. If they are SYN flooding, then that would show up in the firewall logs. If they've received the SYN-ACK packet then they are either from that IP, or they are on a router between you and that IP and can intercept and block the packets from thatIP.

      You don't need a reply if you are running a DDOS.

      You do if it's via TCP. If they're just ping flooding, then that's one thing, but they're issuing HTTP requests. This involves establishing a TCP connection (send SYN, receive SYN-ACK with random number, reply ACK with that number) and involves sending TCP window replies for each group of TCP packets that you receive.

      On the other hand, why would anyone, including Microsoft, want to bring down CPAN?

      Who says that they want to? It's more likely that their web crawler has been written to the same standard as the rest of their code.

  • I suppose Microsoft can offer a simple explanation: "Our servers and other internal infrastructure are so vulnerable that they have been hacked and being used as remote-controlled botnets."
  • Can anyone here clarify what robots.txt stands for, as in:

    Is it an 'agreement' to not scan the site at all (by a search engine bot), or is it meant to just not -display- those results in the search engine?
    I'd assume, since everything on a site is more or less public, that it would be the second. And if so, I can't see anything wrong with what Microsoft's bots did.

    I can see how scanning a site's content (even if you're not going to list the results in your search engine) can have some value to a company
    • Re: (Score:3, Informative)

      by Ogi_UnixNut (916982)

      It's the first. Whatever you specify in the robots.txt as no-follow etc... means not to spider the pages, so no scanning of them at all.

      You use it for when you only want part of your site to appear in search results, such as just the front page (for example). The rest of the site should not be touched by the bot at all.

    • Re: (Score:3, Informative)

      by afidel (530433)
      It's basically a rough pattern filter that the bot is supposed to follow on parts of the site not to crawl. One reason it's used is that you can have dynamically generated pages that create an infinite loop that's impossible for the bot to detect.
    • Re: (Score:3, Informative)

      by John Hasler (414242)

      Is it an 'agreement' to not scan the site at all...

      It is a request not to scan part or all of a site. robots.txt [wikipedia.org]

      And if so, I can't see anything wrong with what Microsoft's bots did.

      Every site does not have dozens of powerful servers and terabytes of bandwidth, nor is every site an ad-supported one that wants to maximize traffic. Common courtesy requires that a bot operator minimize his impact on any given site and honor requests not to index. Of course "courtesy" and "honor" are concepts that baff

  • > ...issues accessing their sites...

    "Issues"? What's wrong with "problem"? "Issues" is marketing-speak. Microsoft marketing-speak.

    And yes, get off my lawn.

  • by N1ckR (1289800) on Monday January 18, 2010 @10:13AM (#30807476)
    I redirect lost bots home, seems a polite thing to do. 301 www.microsoft.com
  • DDoS? Really? (Score:3, Informative)

    by Siberwulf (921893) on Monday January 18, 2010 @10:33AM (#30807666)
    I'm pretty sure the first "D" in DDoS stands for "Distributed."

    If it was really a DDoS, you wouldn't be able to filter the IP out with a simple regex (like the /^65\.55\.(106|107|207)/. from TFA).

    To boot, TFA didn't even say DDoS. Maybe that's too much to expect the editors to oh... I don't know...say... RTFA or Fact-Check it?

    I should drop my bar a bit, I suppose.
  • No problem (Score:5, Informative)

    by rgviza (1303161) on Monday January 18, 2010 @10:51AM (#30807868)

    ipchains -A input -j REJECT -p all -s 65.55.207.0/24 -i eth0 -l
    ipchains -A input -j REJECT -p all -s 65.55.107.0/24 -i eth0 -l
    ipchains -A input -j REJECT -p all -s 65.55.106.0/24 -i eth0 -l

    problem solved

  • by jchawk (127686) on Monday January 18, 2010 @10:57AM (#30807954) Homepage Journal

    The CPAN folks could complain to their ISP and have them drop the traffic that's coming in to their boxes.

    Most ISP's will work with you to correct DDOS problems.

  • hello? firewall? (Score:3, Insightful)

    by v1 (525388) on Monday January 18, 2010 @12:15PM (#30808752) Homepage Journal

    if it's a scan (TCP established stream, taxing the SERVERS, not the NETWORK) that's the problem, as opposed to a SYN flood etc, and the IP addresses are in a very small range, why aren't they just using a hardware firewall at the router and blocking the IPs? There's not a whole lot to "distributed" when it's coming from a pair of C's.

    Not saying they should be DOING it, but this is not a Denial of Service, it's a Denial of Stupid.

Aren't you glad you're not getting all the government you pay for now?

Working...