Become a fan of Slashdot on Facebook

 



Forgot your password?
typodupeerror
Get HideMyAss! VPN, PC Mag's Top 10 VPNs of 2016 for 55% off for a Limited Time ×
Open Source Python Technology

Python 3 Is Coming To Scrapy (scrapinghub.com) 87

New submitter Valdir Stumm Junior writes: Scrapy with beta Python 3 support is finally here! Released through Scrapy 1.1.0rc1, this is the result of several months of hard work on the part of the Scrapy community and Scrapinghub engineers.

This is a huge milestone for all you Scrapy users (and those who haven't used Scrapy due to the lack of Python 3). Scrapy veterans and new adopters will soon be able to move their entire stack to Python 3 once the release becomes stable. Keep in mind that since this a release candidate, it is not ready to be used in production.

This discussion has been archived. No new comments can be posted.

Python 3 Is Coming To Scrapy

Comments Filter:
  • by Anonymous Coward on Thursday February 04, 2016 @08:43PM (#51442989)

    What the fuck is Scrapy?

    • by vux984 ( 928602 ) on Thursday February 04, 2016 @08:50PM (#51443035)

      What the fuck is Scrapy?

      Pretty sure it's that irritating dog they added to scooby doo to try to inject new life into it but which only made it worse.

      • That was Scrappy. Scrappy, Scrappy Poo, what are you?

        • What the fuck is Scrapy?

          You are a geek . You read the word scrapy along with the word python. The word is strikingly similar to the word scrape. You asked the above question.

          You are not a geek

    • by Anonymous Coward
      • Re: (Score:2, Informative)

        by Anonymous Coward

        Nope, that link just goes to a page that says:

        let methat for you [sic]

        [Google Search] [I'm Feeling Lucky]

        Enable javascript to use LMGTFY.

      • By your logic, all the Slashdot editors have to do in their summaries is say "there's some news, head to a news site to find out what it is!" Having to google what a summary is talking about means that it is a bad summary.

    • by pem ( 1013437 )

      It's the sheep equivalent of mad cow disease.

    • by SeaFox ( 739806 )

      I thought they were talking about a Debian or Ubuntu release but the alphabetizing of version numbers is off.

    • I had scrapy once but the ointment cleared it right up.

    • by Anonymous Coward on Thursday February 04, 2016 @09:23PM (#51443215)

      Scrapy (/skrepi/ SKRAY-pee) is a free and open source web crawling framework, written in Python. Originally designed for web scraping, it can also be used to extract data using APIs or as a general purpose web crawler.

      • Scrapy (/skrepi/ SKRAY-pee) is a free and open source web crawling framework, written in Python. Originally designed for web scraping, it can also be used to extract data using APIs or as a general purpose web crawler.

        And their tagline is "Scapy is crappy"?

    • three problems scrappy solves:
      1. Scrappy Scrappy Scrappy? Scrappy, Scrappy Scrappy, Scrappy Scrappy.
      2. Scrappy Scrappy: Scrappy, Scrappy Scrappy!
      3. Scrappy, Scrappy Scrappy Scrappy. Scrappy Scrappy Scrappy, Scrappy Scrappy? Scrappy.
      4. ?????
      5. Profit!

      Scrappy Scrappy Scrappy Scrappy Scrappy Scrappy Scrappy. Scrappy Scrappy Scrappy Scrappy. Scrappy Scrappy Scrappy Scrappy Scrappy Scrappy Scrappy Scrappy? Scrappy Scrappy Scrappy Scrappy Scrappy Scrappy Scrappy Scrappy Scrappy Scrappy Scrappy Scrapp

    • What the fuck is Scrapy?

      A rather ugly prion disease. It's like naming your new application Smegma or Pus.

    • by squiggleslash ( 241428 ) on Thursday February 04, 2016 @11:25PM (#51443819) Homepage Journal

      It's an collaborative cloud-ready framework that leverages Python based open source technologies to extract data across multiple standards based web sites.

      (Wow, actually looking at their website, I got it right, and I made that up initially.)

  • WTF Is Scrapy? (Score:5, Insightful)

    by Anonymous Coward on Thursday February 04, 2016 @08:47PM (#51443013)

    Jesus christ I hate submissions like this. Not only is it a blatent product-pimping ad, they cant even be bothered to explain WTF they are pimping.

  • This is the most amazing thing I have ever heard! Scrapy is ... a thing, which ... does ... stuff, and now I can use things which do stuff with Python ... 3 is it? I can hardly contain my joy.

    • I also fail to see the big deal.

    • This is the most amazing thing I have ever heard! Scrapy is ... a thing, which ... does ... stuff,

      I know, isn't it incredible? It's the most amazing thing I've never heard of. I can't wait for another "article" about some "thing" that does "stuff".

      Thank you, Slashdot editors, for maintaining the high journalistic stands we've come to never, ever expect!

      • by arth1 ( 260657 )

        Who is this "yaelk" submitter anyhow? He or she makes an overworked timothy look like an outstanding editor in comparison, and that's quite an achievement.

        • He or she makes an overworked timothy look like an outstanding editor in comparison,

          A monkey with a severe head injury would look like an outstanding editor in comparison with timothy.

    • Hey, I wrote a couple of words about what Scrapy is in a comment: http://developers.slashdot.org... [slashdot.org]
    • by plopez ( 54068 )

      That's worth at least $10 million in crowd funding!

  • by OzPeter ( 195038 ) on Thursday February 04, 2016 @09:13PM (#51443157)

    This is the sort of thing I mark as Binspam in the Firehose. Blatant advertising.

    Unfortunately I was doing other things when it popped up.

  • by Art3x ( 973401 ) on Thursday February 04, 2016 @09:22PM (#51443207)

    From the summary:

    Scrapy with beta Python 3 support is finally here!

    Here's how I would write it:

    Scrapy, a web spider, now has beta support for Python 3.

    This is why I get paid the big bucks.

  • What the fuck is Scrapy? How do you even pronounce it? Scray-pee or Scrap-e?

  • Seriously, would it be SO hard to include a couple of words about what "Scrapy" is? Just a couple of words?

    Fuck it. You know, I had high hopes that the new owners of slashdot might exercise just the tiniest fucking bit of editorial skill or acumen, but I must be one of those perpetual optimists, doomed to disappointment.

    Pro Tip to the "editors": When you write an article about something, it's considered good journalistic practice to explain potentially obscure references so people know what the fuck you're

    • hey, I just included a comment with a few words about what scrapy is: http://developers.slashdot.org... [slashdot.org]
  • by Valdir Stumm Junior ( 4424309 ) on Thursday February 04, 2016 @10:27PM (#51443537)
    Sorry guys for the not so informative story up there. Well, Scrapy is an open source framework to build web crawlers in Python. The lack of Python 3 support was a huge blocker for people who wanted to use Scrapy, but didn't want to use Python 2.7 anymore. This is why this release is a milestone for Scrapy users. It's very popular between python developers working with web crawling/web scraping. You can check the project page at GitHub here: https://github.com/scrapy/scra... [github.com]
    • It's very popular, but somehow took years to see Python 3 support? This is something that would take a single developer an afternoon to commit. I'm trying to decide if this speaks badly for the python community, or the scrapy community.
      • Also, we all know that scrapers are a bunch of degenerate parasites (and yes I include myself in that group before the downvotes come in).
    • And all 3 of you that use it or care already know about python3 support, and the entire rest of the Internet doesn't give a fuck.

      I know this because I have github projects that aren't working at all with more pull requests and forks than you have and I didn't do any spam advertising on slashdot for them.

      Slashdot is not a ad platform for your little pet project to reimplement (poorly) something that has been done properly at least 10 times already and done poorly like your hundreds of times.

    • This is a good back about web scraping and it includes material about scrapy:

      http://www.amazon.com/Web-Scraping-Python-Collecting-Modern/dp/1491910291/ref=sr_1_1?s=books&ie=UTF8&qid=1454730640&sr=1-1&refinements=p_27%3ARyan+Mitchell
  • WTF is Scrapy?
  • Thank God (Score:2, Funny)

    by Anonymous Coward
    The only thing keeping me from using Scrapy were two things:

    1) Lack of Python 3 support
    2) Knowing what Scrapy is


    Now only 2) is the thing that is keeping me from using it!
  • Something you may appreciate while reading this thread is that Slashdot users are very, very intolerant of advertising motivated-clickbait like this. We're here for, what was the phrase?..."News for Nerds"; thought-provoking science and technology discussions. We're not here to give our eyeballs all day to every website that paid you a few bucks to steer clicks to their new shiny product, though we realize bills need to be paid.

    If you're looking for ad revenue, you need to be smarter and less obnoxious abou

Center meeting at 4pm in 2C-543.

Working...