Comparing R, Octave, and Python for Data Analysis 61
Here is a breakdown of R, Octave and Python, and how analysts can rely on open-source software and online learning resources to bring data-mining capabilities into their companies. The article breaks down which of the three is easiest to use, which do well with visualizations, which handle big data the best, etc. The lack of a budget shouldn't prevent you from experiencing all the benefits of a top-shelf data analysis package, and each of these options brings its own set of strengths while being much cheaper to implement than the typical proprietary solutions.
Get the Popcorn (Score:5, Funny)
Well then [imgur.com].
Re: (Score:2)
If people thought Idle was bad, the Business Intelligence takes Slashdot an order of magnitude lower.
How long until the BI editors demand outright access to the frontpage?
Re: (Score:1)
It's even more moronic when you consider that the articles comments had more useful content than the actual article!
It's no wonder that taco left... /. It was a nice ride but you've really fallen by the wayside in the last few years as in nearly irrelevant with late story postings and garbage like this one.
Did I seriously miss something? (Score:5, Informative)
Re:Did I seriously miss something? (Score:5, Interesting)
The whole article was not much more than a high level review. The graphic naturally draws attention to the parameters the writer wanted to cover but he did not back up his graphic with any sort of serious textual review of what he felt were the weaknesses or advantages of the different programming language at least not in any detail.
And what he has is flawed as well. For example, he marked R as having issue with big data which is quite wrong - I routinely analyze multi-GB datasets in memory, and my databases go into TB. Of all the three languages R is the only one to have a native format (data.frame) that interfaces easily to database queries. Both Octave (Matlab) or Python have to use compound types which make addressing difficult.
Also, I found R easier to master than either Octave or Python, but this is probably because I am familiar with Lisp.
Re: (Score:3)
There are also packages like ff and others which handle absolutely gigantic files by offloading parts of them to storage and only allocating memory for them (and storage) when required. R certainly has some problems with dealing with huge amounts of data, but they aren't insurmountable for datasets less than 1T.
Re: (Score:2)
I think the difference is when you use file formats that are flatter than databases and certain GUIs. In those cases, rather than taking the data as it needs it, it attempts to load all of it into memory and can max out the memory allowed to the process in 32 bit systems. But even then, there are ways around that through smart planning, variable use, and multiple data files for different variables so not all are in memory at once (of course databases implements all three at once internally).
This only happens if you issue a call like read.table("mytable.txt") - you can read the file piece by piece if you want to. Granted, this requires some work (unlike SAS), but in return you can do loops ;)
Re: (Score:2)
So effectively your rich exploration environment is unusable unless you refrain from using all but the simplest operations, or you write your own versions of commands from scratch with efficiency in mind.
This is
Re: (Score:3)
32 bits? are you serious?
Python does have data.frame.. (Score:4, Informative)
Through pandas [sourceforge.net], for a start. The SciPy/NumPy stack is quite nifty, I'm especially interested in how to apply it for working with irregular time series data.
Not to say anybody should ditch R, I still support our researchers most weeks at work in using it. But it's not as clear-cut as you seem to think it is, especially in terms of memory efficiency.
Re: (Score:2)
Re: (Score:1)
And what he has is flawed as well. For example, he marked R as having issue with big data which is quite wrong - I routinely analyze multi-GB datasets in memory, and my databases go into TB.
Dude. That's not what people mean when they say big data. HP and Dell will both quite happily sell you machines with 2TB of main memory, and SGI will go to 16TB, and anything which can fit in memory on a single machine without custom hardware isn't big data. It's only big data once you get up to a few hundred terabytes.
Re: (Score:2)
And what he has is flawed as well. For example, he marked R as having issue with big data which is quite wrong - I routinely analyze multi-GB datasets in memory, and my databases go into TB.
Dude. That's not what people mean when they say big data. HP and Dell will both quite happily sell you machines with 2TB of main memory, and SGI will go to 16TB, and anything which can fit in memory on a single machine without custom hardware isn't big data. It's only big data once you get up to a few hundred terabytes.
Heh ! I am sure I can use R on such hardware, as long as I have access to it ;)
Re: (Score:2)
If you know Lisp and OOP R is easy. Unfortunately Lsip has become arcane and most programmers I met did not understand OOP.
Re: (Score:1)
he did not back up his graphic with any sort of serious textual review
She [slashdot.org] is Geeknet's "Senior Director of Analytics".
Re: (Score:3)
Indeed. This is high level "meeting for the suits" bullshit. I can picture this showing up on powerpoint presentation.
Here are your three options.. this is the one that sucks, this is the one that sucks for a different reason, and this is the one I want you to go with. Oh, and here is a chart with some pretty checkmarks and stuff to help clarify! Lets do lunch!
SlashBI is very disappointing (Score:2)
I think a lot of Slashdot readers (me included) would be interested to get an introduction in various practical aspects of analytics, especially with Open Source tools we can experiment with ourselves. SlashBI could be a good gateway for that. So far every article I have read there has seems like a waste of time.
Re: (Score:2)
It wasn't even that. It came down to one of the last paragraphs:
"In my [limited and misleading] experience...."
Python isn't good at visualization? I guess the author has never used VTK-Python or Matplotlib. R isn't good with big data? I suppose that comes from R not having great database interactivity... so just feed it data via Python using rpy2.
I wish he had learning resources. (Score:4, Insightful)
I wish there was also a column for availibility of resources for learning like: tutorials, free books, example code, etc ....
Never selected that way (Score:5, Insightful)
how analysts can rely on open-source software
I've done that kind of stuff at work and those criteria are NEVER how a package is selected.
If I need a commercial product I need all manner of signoffs requiring at least weeks of delay and massive IT involvement so they can insert it into windoze images automatically or whatever it is they do.
If I'm doing FOSS it just ... gets done that day. No agony. And it just works, and instead of a call center script reader in India who can only tell me to reinstall the software over and over, with FOSS the "whole internet" is my support system and they as in the whole internet know what they're doing.
Nothing about this has changed in about 15 years, so I'm not sure how this is "news". This would have been a good "news" story in the early/mid nineties.
Re: (Score:1)
Even with proprietary software, the "whole internet" can support your system, it is bunk to say that only happens with FOSS.
And to say it just works is bunk too, I see plenty of problems with FOSS where the "whole internet" has no f*cking clue other than, go to the source and figure it out yourself - not always a trivial exercise.
But go ahead and and keep believing your own bullshit.
Re: (Score:2)
... except the 'whole internet' often says "too bad, you'll have to wait for a fix" with proprietary software whereas "Oh, try this patch over here" often happens on FOSS instead.
Re: (Score:3)
This is a thinly veiled attempt to put Python on the same level as R. /shakes head/
Re: (Score:1)
Both! (Score:4, Insightful)
The best option is to use python and R, through rpy for example.
R rocks for statistical libraries and good documentation.
Python rocks for everything else.
Re: (Score:2, Insightful)
Besides, in research, using something opensource (or at the very least gratis) makes it that much easier for others to replicate what you did. Getting SAS scripts just isn't fun.
Re: (Score:2)
Where I work the whole Cisco fiasco put the fear of god into the high level suits. The fall out is a huge and cumbersome process for getting approval to use FOSS tools... even though we arn't modifying or distributing them. It's to the point where it's less headache to _buy_ something than go through the lengthly FOSS approval process.
What was the Cisco fiasco? My company uses Opensource tools routinely and I've never even heard of the Cisco fiasco.
Re:Never selected that way (Score:5, Insightful)
I'm an astronomer. At this point in my career, I move to a new research institution every couple of years. Each institution may have a site licence for some piece of commercial software like IDL or Matlab, but I use free software (Python, in my case) because I know that I can keep using it, rather than rewriting all my scripts for a new language every time I move.
What an awful article. (Score:1)
n/t
Re: (Score:1)
This is one of the most lazily writting articles I've seen Slashdot link to.
Mod +1 Ironic
More crap from /. (Score:5, Insightful)
"Here is a breakdown of R, Octave and Python ..."
No there isn't - that's there is not much more than a shitty 'feature' table, too high level to be anything other than facile, which is "Based on [the author's] own user experience and research".
As an student user of all 3 I would have been interested in reading a good comparative review or explanation aimed at outsiders. This ain't it; it's just more slashvertising.
Re: (Score:1)
Yes, but the advantage of the author's approach is that it'd be real easy to extend the review to include Scilab.
Re: (Score:2)
Spoken like a man who earned a C in freshman year intro to programming, but for some reason didn't switch to a humanities major.
Or if you can't make up your mind (Score:2, Interesting)
Sage math http://www.sagemath.org/
Julia? (Score:4, Informative)
Oh.. (Score:3, Insightful)
Now that's just desperation.
Come on .. keep this shit in bi. Either it takes off or it doesn't.
Read this while listening to the Mensroom segment (Score:1)
My suggestion is to try all three, and see which offering’s toolbox solves your specific problems.
Well no **** Sherlock!
I don't understand (Score:5, Informative)
These languages also have a target audience. R is for statisticians and scientists. Octave is for mathematicians, and Python is for programmers.
Re: (Score:2)
Re: (Score:2)
Oh oh.... you mentioned Fortran. Here come the "Fortran is ugly and out of date" posts. To nip it in the bud I will link to http://en.wikipedia.org/wiki/Fortran#Fortran_2008 [wikipedia.org]
Check out Fortran 2008,which is way cool! Everything you could want from a modern programming language.
apples vs oranges (Score:2)
I still don't get it. How can you compare specialized statistical and number crunching languages with a general purpose programming language.