Forgot your password?
typodupeerror
Facebook Software

Facebook Details the Software Engineering Behind Graph Search 41

Posted by Soulskill
from the powered-by-notepad-and-mountain-dew dept.
Nerval's Lobster writes "Facebook's Graph Search, its new and powerful way of searching the social network for all manner of information, has drawn a lot of attention since its January unveiling. Some have praised its innovation; others have wondered openly whether its search abilities will end up threatening Google and LinkedIn. Still more have questioned what it all means for users' privacy—always a touchy subject in conjunction with Facebook. The social network previously revealed how it's adjusting its hardware infrastructure to deal with the spike in traffic that will come from interactions with Graph Search (short answer: the Disaggregated Rack, which will break up hardware resources and scale them independently of one another). Now, in a new blog posting, it's offering a bit more with regard to the software side of things, and how the company repurposed an existing system to solve Graph Search's enormous engineering challenge. Bottom line: Facebook's engineers and executives finally decided on Unicorn, an inverted-index system they'd had in development for quite some time."
This discussion has been archived. No new comments can be posted.

Facebook Details the Software Engineering Behind Graph Search

Comments Filter:
  • by Anonymous Coward on Wednesday March 06, 2013 @05:47PM (#43097673)

    "Facebook's Stalker Search, its new and powerful way of searching the social network for all manner of information about you, has drawn a lot of negative attention since its January unveiling. Few have praised its innovation; fewer have wondered openly whether its search abilities will end up threatening Google and LinkedIn. Most have questioned what it all means for users' privacyâ"always a touchy subject in conjunction with Facebook. The social network previously revealed how it's adjusting its hardware infrastructure to deal with the spike in traffic that will come from interactions with Stalker Search (short answer: the Disorganized Rack, which will break up hardware resources and scale them independently of one another). Now, in a new blog posting, it's offering a bit more with regard to the software side of things, and how the company repurposed an existing system to solve Stalker Search's enormous engineering challenge. Bottom line: Facebook's engineers and executives finally decided on Unicorn, a mythical flying horned horse they'd had in the basement for quite some time."

    • We need the ability to run any algorithm we want, we need an API to build apps to take advantage of it, we need the full range and capabilities of social network analysis, we need to be able to use intelligent agents to regularly collect opinion and other analytic information about our friends to help us make better decisions.

      How many of my friends like X is important, but that's not going deep enough in my opinion. How many of my friends use certain phases, now we are getting somewhere useful. How popular

      • We need to allow app makers to do the things and offer the services we can't, the really intrusive stuff that we need plausible deniability over, and by monetizing our data via licensed app services which perform tasks which we find morally ambiguous we can keep our new and desperate shareholders happy in both ways.
        • by elucido (870205)

          We need to allow app makers to do the things and offer the services we can't, the really intrusive stuff that we need plausible deniability over, and by monetizing our data via licensed app services which perform tasks which we find morally ambiguous we can keep our new and desperate shareholders happy in both ways.

          But it's not really intrusive. People can change their privacy settings. Also big corporations are allowed to do it, so why not let everyone else in on it?

  • I think it's really funny how they call a drop in traffic a "spike in traffic". Facebook was a fad and people have moved on.
    • It's differentiation based on values, interuser traffic going down is less consequential than metauser traffic going up. The value system in place is obviously corporate. Has there ever been any doubt in your mind? If so, then consider yourself slow or idealistic. At least you can decide that for yourself.
    • Haven't you ever played volleyball?
    • There is some evidence of this. The younger crowd that was initially attracted to FB are starting to move on to other things like Tumblr. This is not just me talking...I've read a few articles that have pointed out this trend. Why are they moving on? Same thing that kills every other trend...their parents start getting into it so it's no longer cool. But...the corporations are still heavily invested in FB.

      It seems to me that FB is becoming more of an advertising medium than a social network. The corporation

  • by QilessQi (2044624) on Wednesday March 06, 2013 @06:00PM (#43097827)

    You get as much privacy from Facebook/Gmail/Hotmail/etc as you pay for. Sometimes, you get less.
    If you're unhappy with those terms, you probably shouldn't use the service.

    • by PopeRatzo (965947)

      It's still worthwhile making sure as many people as possible know just how shitty Facebook is about respecting privacy.

      They've made every effort to obfuscate what they're doing, and not everybody has the time and energy to search out the details of the privacy risks of using Facebook.

      I salute the public-spirit minded people who make an effort to inform. As often as Facebook changes their privacy policy and acts dishonestly, I don't think you can say "People who use Facebook know what they're in for."

      • by QilessQi (2044624)

        I agree. But I think the simple axiom is this: if you use a free service, assume that any information can be sold to anyone for any purpose at any time.
        If people could be made to understand that, the specifics would almost be uninteresting.

        • by tlhIngan (30335)

          I agree. But I think the simple axiom is this: if you use a free service, assume that any information can be sold to anyone for any purpose at any time.
          If people could be made to understand that, the specifics would almost be uninteresting.

          An even simpler one dates way back to when we were connecting computers together by modems. "Don't post online what you don't want the world to know." Because anything posted online IS pretty much available for the world.

          Yes, even with privacy settings. Privacy settings a

          • by PopeRatzo (965947)

            The only real way to have privacy is to not put that information online in the first place.

            Yes, except how much information comes not from what you PUT on the internet, but what you LOOK AT on the internet?

            • by tlhIngan (30335)

              Yes, except how much information comes not from what you PUT on the internet, but what you LOOK AT on the internet?

              In which case Google should be the bigger offender than Facebook because Google is literally everywhere. Facebook's like buttons are on popular sites and such. Google's tentacles are everywhere, and gather data through practically everyone using Google Analytics, or a Google-owned ad service (not just AdWords, but DoubleClick, AdMob, and other advertising agencies). Or YouTube videos which are

              • by PopeRatzo (965947)

                In which case Google should be the bigger offender than Facebook because Google is literally everywhere. Facebook's like buttons are on popular sites and such.

                That's my point.

                But I think you may be understating Facebook's use of the like buttons. They are much more widespread than just "popular sites and such".

                I've been seeing them everywhere. It's become unusual for me to view a page that does NOT have a Facebook button. And that means Facebook is tracking you. And lately, I've noticed that even block

  • Tech sites love to post over and over again about these new Facebook features. But by the time they actually roll out to users everyone's forgotten about them.
  • Would really like to see Unicorn become open source.

    Where I work we use datamarts spread across several data warehouses, which is quite similar to the FB way.
    Since we use a bottom-up design model, creating so called solutions using this indexer would be very straightforward.

  • ...and still others replied to those comments.
  • by Drunkulus (920976) on Wednesday March 06, 2013 @06:57PM (#43098523)
    "Hard part of startup is make money from wheel after reinvent it." -Devops_Borat
  • And does anyone on staff at Slashdot know the difference?

  • “What are the differences between Mark Zuckerberg and me? I give private information on corporations to you for free, and I’m a villain. Zuckerberg gives your private information to corporations for money and he’s Man of the Year.” – Julian Assange I can't confirm if, where, and when he said this, but regardless the idea rings true for me.
  • Right now it's just a toy. They need to expand on it greatly so that we can do legitimate social network analysis research with it and even use it to make decisions.

  • Such as the political positions of our friends, how useful is that?
    What products our friends might want in the future, how useful is that?
    How our friends feel about certain things and who in the social network have feelings in common, how useful is that?
    The job prospects and career prospects of our friends, how useful is that?

    When all the data points are connected and social network analysis properly conducted you can learn a lot of the mysteries about people that wouldn't ordinarily be easily known. This i

  • by xxxJonBoyxxx (565205) on Wednesday March 06, 2013 @11:53PM (#43101185)

    >> Some have praised its innovation

    Er...what? 28 comments in 8 hours tells me no one cares about Graph Search - not even on SlashDot.

  • I would like to see their platform work on the combined datastores of linked open data (http://linkeddata.org/ [linkeddata.org]). Get fast answers because much of the processing is pre-indexed.
  • Re: "run it on a RAM sled with between 128 GB and 512 GB of memory" Google gave me absolutely nothing on RAM sleds. I've used RAM disks for years and even know of hard disk's that are flash-backed RAM for performance. 128GB-512GB of RAM? If I needed that in a server, SGI (rip) and others have it. I doubt that's what they mean, though, as it's expensive custom stuff. So, what is a RAM sled? And where are they bought or how are they set up? Thanks ahead of time for any answers.

"Tell the truth and run." -- Yugoslav proverb

Working...