krewemaynard writes "Google has announced that they were able to sort one petabyte of data in 6 hours and 2 minutes on 4000 computers. According to the Google Blog, "One petabyte is a thousand terabytes, or, to put this amount in perspective, it is 12 times the amount of archived web data in the U.S. Library of Congress as of May 2008. In comparison, consider that the aggregate size of data processed by all instances of MapReduce at Google was on average 20PB per day in January 2008." The technology making this possible is MapReduce, "a programming model and an associated implementation for processing and generating large data sets." They have also posted a video from their Technology RoundTable discussing MapReduce." Link to Original Source
This discussion was created for logged-in users only, but now has been archived.
No new comments can be posted.
Google Sorts 1PB in 6 hours 0 Comments More Login /
Get More Comments