Facebook Details the Software Engineering Behind Graph Search 41
Nerval's Lobster writes "Facebook's Graph Search, its new and powerful way of searching the social network for all manner of information, has drawn a lot of attention since its January unveiling. Some have praised its innovation; others have wondered openly whether its search abilities will end up threatening Google and LinkedIn. Still more have questioned what it all means for users' privacy—always a touchy subject in conjunction with Facebook. The social network previously revealed how it's adjusting its hardware infrastructure to deal with the spike in traffic that will come from interactions with Graph Search (short answer: the Disaggregated Rack, which will break up hardware resources and scale them independently of one another). Now, in a new blog posting, it's offering a bit more with regard to the software side of things, and how the company repurposed an existing system to solve Graph Search's enormous engineering challenge. Bottom line: Facebook's engineers and executives finally decided on Unicorn, an inverted-index system they'd had in development for quite some time."
Let's rewrite that (Score:5, Funny)
"Facebook's Stalker Search, its new and powerful way of searching the social network for all manner of information about you, has drawn a lot of negative attention since its January unveiling. Few have praised its innovation; fewer have wondered openly whether its search abilities will end up threatening Google and LinkedIn. Most have questioned what it all means for users' privacyâ"always a touchy subject in conjunction with Facebook. The social network previously revealed how it's adjusting its hardware infrastructure to deal with the spike in traffic that will come from interactions with Stalker Search (short answer: the Disorganized Rack, which will break up hardware resources and scale them independently of one another). Now, in a new blog posting, it's offering a bit more with regard to the software side of things, and how the company repurposed an existing system to solve Stalker Search's enormous engineering challenge. Bottom line: Facebook's engineers and executives finally decided on Unicorn, a mythical flying horned horse they'd had in the basement for quite some time."
Search isn't enough. Social network analysis is it (Score:2)
We need the ability to run any algorithm we want, we need an API to build apps to take advantage of it, we need the full range and capabilities of social network analysis, we need to be able to use intelligent agents to regularly collect opinion and other analytic information about our friends to help us make better decisions.
How many of my friends like X is important, but that's not going deep enough in my opinion. How many of my friends use certain phases, now we are getting somewhere useful. How popular
Re: (Score:3)
Re: (Score:2)
We need to allow app makers to do the things and offer the services we can't, the really intrusive stuff that we need plausible deniability over, and by monetizing our data via licensed app services which perform tasks which we find morally ambiguous we can keep our new and desperate shareholders happy in both ways.
But it's not really intrusive. People can change their privacy settings. Also big corporations are allowed to do it, so why not let everyone else in on it?
Re:It had to be unicorns. (Score:5, Funny)
No no, it was a highly complex set of sql queries devised by a highly complex set of mathematical algorithms which took a team of washed up college drop outs to devise. I was able to speak to Mark Zuckkerberg on this matter and he divulged to key areas of the system's source to me.
$searchquery = 'people who are hipster losers';
$sqlquery = 'select * from pages p inner join friends f on p.userid=f.friendid where 1'
mysql_connect("localhost", "leet", "hax0r");
mysql_select_db("facebook_db");
$page = mysql_fetch_array($sqlquery);
if($page['like']>0) {
if(strpos($page['content'], $searchquery) === true) {
echo $page['content'];
}
}
Re: (Score:2)
Zuckerberg (Score:1)
What could be more social? (Score:1)
Re: (Score:2)
Re: (Score:2)
There is some evidence of this. The younger crowd that was initially attracted to FB are starting to move on to other things like Tumblr. This is not just me talking...I've read a few articles that have pointed out this trend. Why are they moving on? Same thing that kills every other trend...their parents start getting into it so it's no longer cool. But...the corporations are still heavily invested in FB.
It seems to me that FB is becoming more of an advertising medium than a social network. The corporation
What it means for user's privacy. (Score:4, Insightful)
You get as much privacy from Facebook/Gmail/Hotmail/etc as you pay for. Sometimes, you get less.
If you're unhappy with those terms, you probably shouldn't use the service.
Re: (Score:3)
It's still worthwhile making sure as many people as possible know just how shitty Facebook is about respecting privacy.
They've made every effort to obfuscate what they're doing, and not everybody has the time and energy to search out the details of the privacy risks of using Facebook.
I salute the public-spirit minded people who make an effort to inform. As often as Facebook changes their privacy policy and acts dishonestly, I don't think you can say "People who use Facebook know what they're in for."
Re: (Score:3)
I agree. But I think the simple axiom is this: if you use a free service, assume that any information can be sold to anyone for any purpose at any time.
If people could be made to understand that, the specifics would almost be uninteresting.
Re: (Score:3)
An even simpler one dates way back to when we were connecting computers together by modems. "Don't post online what you don't want the world to know." Because anything posted online IS pretty much available for the world.
Yes, even with privacy settings. Privacy settings a
Re: (Score:3)
Yes, except how much information comes not from what you PUT on the internet, but what you LOOK AT on the internet?
Re: (Score:2)
In which case Google should be the bigger offender than Facebook because Google is literally everywhere. Facebook's like buttons are on popular sites and such. Google's tentacles are everywhere, and gather data through practically everyone using Google Analytics, or a Google-owned ad service (not just AdWords, but DoubleClick, AdMob, and other advertising agencies). Or YouTube videos which are
Re: (Score:2)
That's my point.
But I think you may be understating Facebook's use of the like buttons. They are much more widespread than just "popular sites and such".
I've been seeing them everywhere. It's become unusual for me to view a page that does NOT have a Facebook button. And that means Facebook is tracking you. And lately, I've noticed that even block
too soon (Score:1)
Cool stuff (Score:2)
Would really like to see Unicorn become open source.
Where I work we use datamarts spread across several data warehouses, which is quite similar to the FB way.
Since we use a bottom-up design model, creating so called solutions using this indexer would be very straightforward.
Still others commented about it on facebook.... (Score:2)
Make benefit glorious engineering teams (Score:4, Insightful)
Software Eng. or Comp. Sci. ? (Score:2)
And does anyone on staff at Slashdot know the difference?
This is what I think of when I read this... (Score:2, Interesting)
Needs more complex SNA components. (Score:2)
Right now it's just a toy. They need to expand on it greatly so that we can do legitimate social network analysis research with it and even use it to make decisions.
If it can do SNA, we could learn and predict a lot (Score:2)
Such as the political positions of our friends, how useful is that?
What products our friends might want in the future, how useful is that?
How our friends feel about certain things and who in the social network have feelings in common, how useful is that?
The job prospects and career prospects of our friends, how useful is that?
When all the data points are connected and social network analysis properly conducted you can learn a lot of the mysteries about people that wouldn't ordinarily be easily known. This i
Even on SlashDot...no one cares about Graph Search (Score:4, Insightful)
>> Some have praised its innovation
Er...what? 28 comments in 8 hours tells me no one cares about Graph Search - not even on SlashDot.
Graph search on linked open data (Score:1)
RAM sled? (Score:1)