Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Businesses Databases Programming Software Data Storage IT

Wal-Mart's Data Obsession 581

Posted by timothy from the what-do-you-want-today dept.
g8oz writes "The New York Times covers Wal-Mart's obsession with collecting sales data. Fun fact: 'Wal-Mart has 460 terabytes of data stored on Teradata mainframes, at its Bentonville headquarters. To put that in perspective, the Internet has less than half as much data, according to experts.' That much information results in some interesting data-mining. Did you know hurricanes increase strawberry Pop Tarts sales 7-fold?"
This discussion has been archived. No new comments can be posted.

Wal-Mart's Data Obsession More

Wal-Mart's Data Obsession

Comments Filter:

  • Yeah (Score:4, Funny)

    by Anonymous Coward on Sunday November 14, 2004 @05:00PM (#10814550)
    and shopping there means your income has dropped 7-fold

    • Re:Yeah (Score:5, Funny)

      by takeya ( 825259 ) on Sunday November 14, 2004 @05:23PM (#10814741) Journal
      Wal-Mart has 460 terabytes of data stored on Teradata mainframes, at its Bentonville headquarters. To put that in perspective, the Internet has less than half as much data, according to experts.' ...

      normally, but I guess they didn't check when I was sharing my pr0n on direct connect.

    • Re:Yeah (Score:3, Insightful)

      by vettemph ( 540399 )
      No, Working there means your income has dropped 7-fold.
  • Who says how much data the Internet has available?

  • I, for one, (Score:5, Funny)

    by fiftyfly ( 516990 ) <mike@edey.org> on Sunday November 14, 2004 @05:00PM (#10814552) Homepage
    would like to welcome our new (evil) data collecting overlords.

  • 230 terabytes? Please (Score:5, Interesting)

    by Anonymous Coward on Sunday November 14, 2004 @05:01PM (#10814569)
    My company alone has over 50 terabytes of data available for download on the internet. Whoever thinks there's that little data on the internet is very poorly-informed.

  • Haha... (Score:5, Funny)

    by GR1NCH ( 671035 ) on Sunday November 14, 2004 @05:02PM (#10814575)
    you fools have no idea that I would never let you hurt the Wall-Mart

  • More than the Internet ?! (Score:4, Funny)

    by architimmy ( 727047 ) on Sunday November 14, 2004 @05:03PM (#10814576) Homepage
    Someone at Walmart has ALOT of pr0n!

  • economies of scale (Score:5, Insightful)

    by man_ls ( 248470 ) on Sunday November 14, 2004 @05:03PM (#10814584)
    When you have 460TB of data, how the hell do you even begin to search it?

    Seems like they'd need to license map-reduce from google or something. (That's a distributed data correlation engine. With extremely high fault tolerence, to boot.)

    • Re:economies of scale (Score:5, Insightful)

      by Sexy Bern ( 596779 ) on Sunday November 14, 2004 @05:06PM (#10814613)
      More to the point - how do they back it up?

    • Re:economies of scale (Score:5, Funny)

      by seann ( 307009 ) <notaku@gmail.com> on Sunday November 14, 2004 @05:08PM (#10814627) Homepage Journal
      select sFirstName,sLastName,iPhone from LargeAssDatabase where bWelFare = False;

      go on vacation for a week or ten..

      deal with resulted data.

    • Re:economies of scale (Score:5, Informative)

      by kimanaw ( 795600 ) on Sunday November 14, 2004 @05:15PM (#10814668)
      When you have 460TB of data, how the hell do you even begin to search it?

      With SQL.

      Teradata was built to handle processing very large datasets from day 1. 460 Terabytes distributed across a large number of CPUs and disks working in parallel with a robust SQL implementation isn't really the challenge. The hard part is keeping all those disks spinning when you start pushing MTBF limits, handling the thousands of concurrent users all banging away at the data, and the constant streaming of new data into the system in order to support near real-time DSS.

      For those inclined to know more, check here. [ncr.com]

    • Re:economies of scale (Score:4, Informative)

      by sql*kitten ( 1359 ) * on Sunday November 14, 2004 @05:17PM (#10814684)
      Seems like they'd need to license map-reduce from google or something.

      As the article says, they're using Teradata [ncr.com]. This is not a product that I'd expect the average Slashbot, who thinks "IT" and "internet" are synonymous, to have heard of. Nevertheless, if you work with industrial amounts of data, you will know that Teradata databases can reasonably claim to be to Oracle as Oracle is to MySQL.

      • Re:economies of scale (Score:4, Funny)

        by kimanaw ( 795600 ) on Sunday November 14, 2004 @05:30PM (#10814794)
        "...Teradata databases can reasonably claim to be to Oracle as Oracle is to MySQL."

        Except it takes 8 Teradata DBAs to manage the 460 TBytes, and 23 Oracle DBAs to manage 1 Gig ;^) (Not a slam on Oracle DBAs, but on the ridiculous management burden of Oracle)

    • It's simple really. Each piece of data is given 2 32 bit keys. One is called the selector, the other the offset. The selector key is left-shifted 8 bits and added to the offset to generate the physical address of the data. Thus, any piece of data can be accessed if you know hits selector and offset.

    • Re:economies of scale (Score:5, Informative)

      by MC Negro ( 780194 ) on Sunday November 14, 2004 @05:28PM (#10814784) Journal

      Seems like they'd need to license map-reduce from google or something. (That's a distributed data correlation engine. With extremely high fault tolerence, to boot.)
      I know a guy who worked for Wal-Mart for ~8 years as some sort of data analyst and architect at the main offices in Bentonville. While he didn't go into too much detail, he told me that a lot of the back-end querying is done, surprisingly, with Perl-DBI on Oracle databases. When I asked why his team didn't use something like flat C, C++ or Java, portability was cited as a principal motivation and that, after a certain point, speed gains were only marginal. He also said when he left ~1.5 years ago, that a small cluster migration to DB2 was being talked about. I have no idea if they license search and query code, but I got the distinct impression that there was a team of software engineers who custom crafted search algorithms for the data.

  • You gotta love "experts" (Score:5, Funny)

    by broothal ( 186066 ) <christian@fabel.dk> on Sunday November 14, 2004 @05:04PM (#10814590) Homepage Journal
    the Internet has less than half as much data, according to experts

    What's the word I'm looking for? Oh yeah - it's bullshit

  • And in other news... (Score:5, Insightful)

    by wesmills ( 18791 ) on Sunday November 14, 2004 @05:05PM (#10814594) Homepage
    ...Microsoft has an astonishing amount of information collected from Windows Update users (none of it personally identifiable, of course).

    I highly suspect Wal-Mart didn't get into the position it's in of being the largest retailer by being stupid, at least business-wise. This is the sort of project that allows them to stock a 120,000 square-foot big box store from JIT shipments every night, and why every Wal-Mart in a region looks the same. Though I would be interested to read more on the pop-tart to hurricane correlation...

  • the real interesting part is... (Score:5, Funny)

    by Coneasfast ( 690509 ) on Sunday November 14, 2004 @05:05PM (#10814595)
    they're storing them on a huge cluter of their $200 lindows systems. ;)

  • Correlation doesn't imply causation!!!!! (Score:5, Insightful)

    by Baldrson ( 78598 ) on Sunday November 14, 2004 @05:05PM (#10814602) Homepage Journal
    Did you know hurricanes increase strawberry Pop Tarts sales 7-fold

    Correlation doesn't imply causation!!!!!

    I mean what if a third factor caused both the hurricanes and strawberry Pop Tart sales to increase 7-fold????

    Somebody was going to blurt that bromide out at that statement, so it may as well be me.

    • Re:Correlation doesn't imply causation!!!!! (Score:5, Insightful)

      by krymsin01 ( 700838 ) on Sunday November 14, 2004 @05:10PM (#10814637) Homepage Journal
      It makes sense though. If you are going ride out a storm, you are going to need lots of food that will not require refrigeration nor cooking.

      Beer makes sense also. There are always a hell of a lot of hurrican parties in Florida whenever a hurrican comes 'round.
    • Yes there could be a third reason, but lets think about this. When a hurricane comes, you want non-perishable foods. Computer geeks like myself, like poptarts cuz you just open them up and eat em, and those things don't go bad for a while. No need for a microwave or stove, something you would want for soup and such. SO if a hurricane comes by and wipes out gas & electric and everything is friggen wet, you need something that requires no cooking or heating -> poptarts Of course 7 fold does seem a b

      • Not that high, consider other contributing factors (Score:4, Informative)

        by xant ( 99438 ) on Sunday November 14, 2004 @06:14PM (#10815108) Homepage
        First of all, most Walmarts don't primarily sell food, they primarily sell loads of other stuff. In fact, what they sell is a lot of stuff that people might need to survive a hurricane, including various kinds of hardware, containers, lights, reading material. So a hurricane would naturally drive lots of people into Walmart. Naturally those people will buy food products while they're in there, and the standard Walmart sells mostly junk food. So it's not as if people are seeking out pop-tarts in hurricane season, but the massive influx of people buying all kinds of things will also increase the number of people buying non-perishable junk food.

        Consider also that people will not be worrying about their diets when they're primarily worried about not being killed by their own rooftops...

        Combine a bunch of these factors together, and yes, I can easily believe 7x.

    • Re:Correlation doesn't imply causation!!!!! (Score:3, Funny)

      by Anonymous Coward
      It's a well-known fact that hurricanes bring toasters and mini-fridges, so Pop Tarts and beer are logical purchases.

    • Re:Correlation doesn't imply causation!!!!! (Score:4, Funny)

      by Daniel ( 1678 ) <dburrows@deb i a n.org> on Sunday November 14, 2004 @05:48PM (#10814906)
      Soooooooooooooooooo....

      If pop tart sales go up, head for high ground? :-)

      Daniel

  • Seen it! (Score:5, Informative)

    by Number44 ( 41761 ) on Sunday November 14, 2004 @05:06PM (#10814610) Homepage
    As a guest of WalMart I was able to enter their data center and see this Terraplex first hand. It's massive. It's thousands upon thousands of disks in ~8' frames, rows upon rows of racks. I walked down it and across it and up it and was simply awestruck by the idea of that many disks in one spot.

    The gentleman who gave me the tour indicated they have something like 72 weeks (1 year plus 2 weeks) of purchase data on LIVE disk arrays, plus huge archives of the same data on tape. If you buy anything and use your credit, debit, or whatever card they can figure out your sales history obscenely quickly. Be afriad. Be very afraid.

    I also got to see Walmart.com (Sun E15k) and Samsclub.com (A bunch of HP boxes in a smallish frame), they were creepy, in a sense... all those sales going on at once, converging on a spot not a few feet from me.

    • Re:Seen it! (Score:3, Informative)

      by nizo ( 81281 )
      I wonder how many people they have running around replacing failed disks in the arrays. It would have to be at least several full-time jobs worth of people, not to mention they must have a gigantic pile of disks waiting on-site.

    • Re:Seen it! (Score:5, Insightful)

      by SamMichaels ( 213605 ) on Sunday November 14, 2004 @05:24PM (#10814748)
      The gentleman who gave me the tour indicated they have something like 72 weeks (1 year plus 2 weeks)

      According to Google:

      1 year = 52.177457 weeks

      So 72 weeks is 1 year plus 19.822543 weeks.

    • Re:Seen it! (Score:3, Funny)

      by alkali ( 28338 )
      The gentleman who gave me the tour indicated they have something like 72 weeks (1 year plus 2 weeks) of purchase data on LIVE disk arrays, plus huge archives of the same data on tape. If you buy anything and use your credit, debit, or whatever card they can figure out your sales history obscenely quickly. Be afriad. Be very afraid.

      Did he happen to mention anything about an attack on Zion?
    • Being there must've been similar to being a time traveler being sent back to WWII and meeting Hitler (yeah yeah, Godwin can cram it), knowing full well that if you just took him out now, you'd save a lot of trouble later on.

      No seriously, why didn't you trash their data and free us all?

    • Be Afraid? Why? (Score:5, Insightful)

      by mveloso ( 325617 ) on Sunday November 14, 2004 @07:31PM (#10815703)
      Why should we be afraid of Wal-Mart? They're using their data to be more responsive to their customer. They want to make sure that if you want something, it's in-stock and ready to go.

      What could they do with their data, really, that would hurt anyone? It wouldn't be like "Bob Smith is buying condoms again." It would be more like "there's a condom spike in area code 78750 every Thursday, let's ship more out."

      People who are afraid of data aggregation are jumping at shadows. Nobody cares what you in particular are buying. An individual as a data point is useless, unless you're an exemplar or something like that (which would be unusual).

      Let's face it, individuals just aren't that interesting. More importandly from Wal-Mart's point of view, there's no return on looking at individuals.

  • Comment removed (Score:5, Funny)

    by account_deleted ( 4530225 ) on Sunday November 14, 2004 @05:07PM (#10814618)
    Comment removed based on user account deletion

  • Please remind me (Score:4, Funny)

    by nerd256 ( 794968 ) on Sunday November 14, 2004 @05:08PM (#10814626) Homepage
    I've been reading the comments
    I forgot, are we supposed to hate Wallmart?

    On one hand they are a large corporate empire and on the other, they promote cheap linux computers.

    arg, Im so confused

    • Re:Please remind me (Score:5, Interesting)

      by Phantasmo ( 586700 ) on Sunday November 14, 2004 @06:53PM (#10815389)
      You can be a socialist Slashdotter and hate that they treat their employees, their suppliers, and their supplier's employees (i.e. fire your American staff and relocate to Indonesia or we're dropping your product) like shit.
      Or you can be a privacy-advocate Slashdotter and hate that they want RFID tags in everything.

      Or you can be a Republican or Libertarian Slashdotter and admire that Wal-Mart opposes government interference in business (you do NOT tell Wal-Mart how to operate).
      Or you can be an apolitical Slashdotter and just agree that, for some products, it's the cheapest place to go.

      I'm the socialist Slashdotter. I know it's not much better but if I need something that I know is at a big retailer I make the trip to Zeller's [hbc.com] first. SILE (Solution Involving Least Evil)

  • Pop Tarts (Score:5, Funny)

    by evilviper ( 135110 ) on Sunday November 14, 2004 @05:08PM (#10814628) Journal
    Did you know hurricanes increase strawberry Pop Tarts sales 7-fold?

    Yes I did. God help me!

  • Heh, lets see if this "predicting" works (Score:5, Interesting)

    by UncleJam ( 786330 ) on Sunday November 14, 2004 @05:09PM (#10814630)
    A few years ago when I worked in retail, everything was going smoothly. Every night the managers would go around with electronic guns and see what needed ordering the next day. Except for the busiest times of the year the backroom was pretty much empty of stock, and on top of the aisles the extra stock was minimal.

    Then one day, the managers were really excited, as we were going to have a computer order everything for us, from records of sales from before and it would "predict" what we would need. They said the extra stock on top of the aisles would be eliminated. We would be able to concentrate on customer service.

    Well, the day came, and for a few months you could tell the computer was fighting with limited data. Some weeks would be rediculously overstocked on a few items, others, the leading sellers in the store would have empty shelves. When it finally settled down after a year, it was worse than before the computer.

    The top of aisles were jammed to the ceiling with stock, there was never any room to put anything up there, and getting to the bottom for something you needed cost a lot of time. Plus, the backroom was packed with stock. You could hardly move around, and trying to find the last box of something buried underneath these huge piles was a task that killed your morale. During the slow months, one stocker for the whole store was enough for a night, now 3 were common to deal with all the stock.
    • Yep, I worked at a walmart and it was rediculous. Unemployed with an engineering degree for a year and a half, I decided it was time to get a move on it. They hired me as a "shipping manager" for the shoe department. Little did I know that "shipping manager" was actually -- the guy who PUT AWAY all the shipments, and was *THE* most hated job in the entire store -- the janitors ("cleanup crew") even told me they wouldn't do my job.

      The Walmart shipping system is was very efficent, but it was designed to serve walmart, not the individual stores. We had an extremely finite space in which to store things, and an extremely finite shoe department, yet the thing shipped us INCREDIBLE ammounts of shoes. And you'e been to a walmart right? They were *EXTREMELY* ugly, horrible shoes.

      One night I recall the system sent me *5* palettes of shoes (1-2 is normal) which took a herculean effort to find *somewhere*, *anywhere* to store them.

      And that was the job, every night. Somehow put away the incredible ammount of shoes that come. Every night, re-arrange "the stacks", re-arrange "the steel" to fit shoes that nobody wanted, that nobody could stop from coming.

      One morning the manager walks up to me and says "Good news, they've decided to keep you full time!" to which I replied "Oh no dont you dare".

  • even the mango is tracted (Score:4, Insightful)

    by loid_void ( 740416 ) * on Sunday November 14, 2004 @05:13PM (#10814659) Journal
    My brother sells mangoes to the Wal Mart Beast. He says it's all computerized, beginning with an order for the fruit, following the trucks, even the rotation of the ripening process in the warehouses is computer related. It's as close to virtual management as any company comes.

  • Just imagine (Score:3, Insightful)

    by nizo ( 81281 ) on Sunday November 14, 2004 @05:13PM (#10814660) Homepage Journal
    Imagine what evil could be done with this data: how about a service where you can track your spouse's/SO's buying habits? See if they buy condoms and flowers every night they work late for example. Imagine what would happen if they started keeping track of fingerprint data off of cash/checks that people use in stores too. Well I am off to go buy some tin foil now (with cash, wearing gloves) :-)

  • There's a name for this.. (Score:5, Insightful)

    by k98sven ( 324383 ) on Sunday November 14, 2004 @05:15PM (#10814676) Journal
    The Law of truely large numbers. [skepdic.com]

    Basically, the more data you have, the more likely you'll find weird coincidental correlations.

    I guess these kinds of 'statistical finding' will become more and more prevalent in the future, given that we're living in an age where we're collecting ever-larger amounts of data, and have the resources to process all this data automatically.

    It would be a good thing if people were a bit more sceptical of this kind of stuff. Correlation isn't causation.

    • Re:There's a name for this.. (Score:5, Insightful)

      by sql*kitten ( 1359 ) * on Sunday November 14, 2004 @05:21PM (#10814720)
      It would be a good thing if people were a bit more sceptical of this kind of stuff.

      Ermm, RTFA.
      1. They predicted that pop tart sales would increase
      2. They shipped additional pop tarts in anticipation
      3. The pop tarts sold like, umm, hot pop tarts

      You can be skeptical all you want. Someone at Walmart made the call, and they were right.

      • Big deal (Score:3, Funny)

        by rsilvergun ( 571051 )
        Ok, people are going to be without power for a while, possibly a long while, and Walmart predicted the sale of nearly unperishable dry goods would rise? My God, the sheer genius of it baffles me!

        Call me when they can Mathmatically prove which flavors are most popular in a Hurricane.

      • Re:There's a name for this.. (Score:5, Insightful)

        by k98sven ( 324383 ) on Sunday November 14, 2004 @06:08PM (#10815068) Journal
        I did RTFA.

        And, firstly: that's not exactly a proper test.
        (Supply does create demand. Why do you think stores like building big pyramids of merchandise, and so on.. Hint: It's not just because it looks pretty.)

        Perhaps you should read my comment again and try to get the point. I wasn't neccesarily being sceptical about pop-tarts. I was being sceptical about the method in general.

        Obviously some of the correlations they'll find are real too. That's not what I was referring to.

        What I was referring to, was that it's very easy to become blind to the statistics. To fall into the trap of seeing correlations where there are none. The human brain has a remarkable pattern-finding ability. Unfortunately that ability does lead us astray sometimes.
        (For instance reading human faces into natural formations, and so on)

        Besides this, the Wal-mart people probably aren't very interested in talking about the times their fancy new method failed, are they?
  • Did you know?
    EVERY TIME A LOAF OF BREAD IS BAKED,
    APPROXIMATELY
    150,000,000 YEASTS ARE
    KILLED

    Come to the award-winning 1987 film,
    "The Very Small and Quiet Screams"
    -- a cinematic electromicrograph of yeasts being baked.

    A must for those who care about yeast, and especially for those who don't.

    SPONSORED BY
    Brown Anaerobe Rights Coalition (BARC)
    Student Bakers for Social Responsibility
    Coalition for the Elevation of Life (CELL)

    Defend all life: "From greatest to least, from human to yeast!"

  • The Problem? (Score:3, Informative)

    by squirel_dude ( 810037 ) <squirrel@iraqi-cabbages.tk> on Sunday November 14, 2004 @05:19PM (#10814709) Homepage
    I hate to sound like some pro-totalitarian next generation Big Brother, but it's not as if they are collecting personal information on customers without the customer's consent. Wal-Mart are just doing some major (I agree with obsessive though) market research so as they can optimise their stores to maximise profits, exactly the same as every other business in the world.

  • That doesn't mean they know what to do with it... (Score:3, Interesting)

    by Anonymous Coward on Sunday November 14, 2004 @05:19PM (#10814711)
    Coworkers who have worked with Wal-mart IT tell me that Wal-mart does indeed have mountains of data. However, they have so much data that they do not know what to do about it. They can't interpret it all because there is just too much of it.

    This makes me wonder... there must be some ideal point where a certain amount of data collected is worth the most money because you can act on that data. After that point, collecting additional data is increasingly more costly and counterproductive unless you invest in an infrastructure that lets you process more data. How does one figure out that ideal point? Just a thought.

  • Did you know... (Score:5, Interesting)

    by GoMMiX ( 748510 ) on Sunday November 14, 2004 @05:20PM (#10814718)
    Wal-Mart employees who use their employee discount cards have every purchase tracked and monitored.

    Activity of the cards is ACTUALLY monitored for discrepencies in buying habits to find abusive employees who buy things for their friends?

    Did you also know Wal-Mart's employee name badges have RFID tags (and have had for many years) that allow Wal-Mart to track where an employee is at any given time?

    Another interesting tidbit, did you know at Wal-Mart's Jewelery warehouses they actually WEIGH the amount of metal in your body when you enter a leave? (And I don't mean they ask you to put things in a dish and weigh the dish - they scan YOU)

    Another interesting thing, Wal-Mart has a fallout facility in Oklahoma that has a near-real-time backup of each BIT of that 460 terabytes of data?
    Wal-Mart could survive a direct nuclear blast and still keep on a truckin'.

    And, of course, if you're in a Wal-Mart home office - ISD building - distribution center - et al... and dial 911 - BOOM - you get Wal-Mart's private security? Niiice, hope it's not a real emergency, you first have to explain it to them - then if they deem it neccessary THEY will call the REAL 911!

    • Re:Did you know... (Score:5, Interesting)

      by Spydr ( 90990 ) on Sunday November 14, 2004 @05:33PM (#10814811) Homepage
      possibly the best part is how they make more customers... the endless cycle...

      1) poor people shop there because it's cheaper than the other stores because wal*mart gets their stuff all from china and stong arms their suppliers to give them cheaper and cheaper products.

      2) to keep up with walmarts demands, the companies have to outsource more and more to china and other cheap labor countries (or just move there entirely)

      3) so more people lose their jobs, become poor and have to shop at wal*mart beacuse 1) it's cheaper than everything else around, and 2) all the other local businesses are now out of business because they can't compete with the special deals wal*mart gets for buying in such huge quantities...

      (goto 1)

    • Re:Did you know... (Score:5, Funny)

      by Trogre ( 513942 ) on Sunday November 14, 2004 @06:46PM (#10815330) Homepage
      then if they deem it neccessary THEY will call the REAL 911!

      You mean like 912?

  • And I really hope it's not on SQL (Score:3, Informative)

    by The-Bus ( 138060 ) on Sunday November 14, 2004 @05:22PM (#10814733)
    To put that in perspective, the Internet has less than half as much data, according to experts.'


    How the hell can they estimate that? Assuming "less than half" means about 45%, that gives us about 207 TB. Let's just round that up to 240.148445 TB to make it a nice, even number.

    Google is searching 8,058,044,651 "webpages"* -- who knows what that means. Now, Google isn't searching every single page on the internet, certainly. But also, they can't be searching pages that don't exist. So the 8bn Google pages aren't certainly all the internet. But Google isn't double or triple counting pages. Still, at 240.148445 TB (my rough estimate), we come up with a page size of exactly> 32KB per page.**

    Is this just counting the text? The code for this page right here (comments.pl) weighs in at about 14KB. Wal-Mart, in no way, has twice as much info as the internet. I would say the "internet" should be measured in at least petabytes. Archive.org itself already has 1PB, and I consider any of that content available to me "on the internet".

    * I'm not even counting the Google cache.
    * Which means Mr. Gates over-estimated by a factor of 20 when considering how much memory we all needed!

  • I'm not afraid.. (Score:3, Funny)

    by Robber Baron ( 112304 ) on Sunday November 14, 2004 @05:22PM (#10814735) Homepage
    Did you know hurricanes increase strawberry Pop Tarts sales 7-fold? ...and if you needed a 460 TB data array to tell you that then you're too stupid to live.

  • 460 TB is nothing we have 25X that (Score:4, Interesting)

    by gelfling ( 6534 ) on Sunday November 14, 2004 @05:25PM (#10814764) Homepage Journal
    My company has 300,000 employees each of whom has about 40GB on their desktops. That's 12,000,000 GB which is 12,000 TB most of which is junk.

  • Welcome to the United States of WalMart (Score:4, Funny)

    by eclectro ( 227083 ) on Sunday November 14, 2004 @05:26PM (#10814770)
    For which it stands, one store under God, indivisible, with sales and product for all.

    From the article;

    "You can see the pattern of Wal-Mart's mandates, and as Wal-Mart grows in power, it is getting more dictatorial.....Wal-Mart lives in a world of supply and command, instead of a world of supply and demand."

  • Hurricanes and Pop-Tarts? Bah... (Score:4, Interesting)

    by cyranoVR ( 518628 ) * <{moc.liamg} {ta} {RVonaryc}> on Sunday November 14, 2004 @05:28PM (#10814778) Homepage Journal
    Political parties are using consumer shopping patterns to figure out who to reach with 1-to-1 political messages.

    Stuff like: women who buy from catalogs, eat "crunchy" peanut butter, own a cat and drive a minivan you are 87% more likely to react positively to prayer in schools as a "motivating issue."

    I just made that up, but it's the sort of thing they find out. No tin-foil hats here - corporations and pollsters are shelling out millions of dollars for this stuff.

    Here's a few google searches links to get you started:

    Acxiom [google.com]

    Seisint [google.com]

  • This is all fine and dandy, but ... (Score:5, Funny)

    by isometrick ( 817436 ) on Sunday November 14, 2004 @05:32PM (#10814810)
    ... do they have a freezer big enough for 460TB worth of drives?

  • half as much data until... (Score:3, Informative)

    by Flamesplash ( 469287 ) on Sunday November 14, 2004 @05:43PM (#10814874) Homepage Journal
    To put that in perspective, the Internet has less than half as much data, according to experts.

    someone realized that the DB servers are actually accessible from the internet and then bam, instand 2x increase in the amount of data on the internet.

  • incredible! (Score:4, Funny)

    by Scrameustache ( 459504 ) on Sunday November 14, 2004 @05:55PM (#10814947) Homepage Journal
    That much information results in some interesting data-mining. Did you know hurricanes increase [non-perishable food item] sales 7-fold?

    It took them 460 terabytes of data to figure out that hurricanes make people buy more non-perishable food than usual?

    Wow, data mining is "usefull"...

  • chaos in the mix (Score:3, Interesting)

    by drwho ( 4190 ) on Sunday November 14, 2004 @06:17PM (#10815129) Homepage Journal
    If you are concerned about all this consumer information being used as 'big brother', maybe you ought to start doing something about it. Lying on the census or your income taxes is illegal, but marketers are fair game. The easiest way to mess with them is to tell them the opposite of the truth. Or, camouflage your true interests by entering a lot of junk. I.E. if your are pissed off that you didn't get a refund you were due from MicroCenter (notorious refund scammers) just fill out several hundred bogus refund forms. Jam the system.

    If you're willing to break the law, you can even do worse harm. But I don't condone that.

    Using legal methods to increase the entropy are the best way to fight the marketing databases.

  • New data measurement type (Score:3, Funny)

    by snaphu ( 412871 ) <snaphu@h o t . ee> on Sunday November 14, 2004 @06:28PM (#10815208)
    As mentioned by a friend when referrering to his video clip collection (but it doesn't help the videos/films he makes):
    "Oh, I have a few frigabytes of data."
    "Frigabyte? What's that?"
    "Oh, that's a friggin lot of data."

  • WalMart BS (Score:5, Insightful)

    by Doc Ruby ( 173196 ) on Sunday November 14, 2004 @06:34PM (#10815241) Homepage Journal
    WalMart's 460 TB of data, shared among about 300M Internet users [clickz.com], would spread about 1.5MB to each person. That is, of course, a tiny amount of data - probably just the indices on each person's inbox, let alone their email data itself. Each of those people average storage capacity is over 20GB [jr.com], on new computers, excluding upgrades which are probably usually about 80GB. So just typical end user computers alone account for at least 10,000 - 40,000 times WalMart's big data dump. And then of course there are all the other servers on the Internet, like the SABRE airline reservation system, the US Federal databases of publications, Google's image cache, all the albums and other MP3/SHN/FLACs in P2P, and of course the endless stream of porn.

    WalMart is trying to make itself look like it is turning its customer data into success, and benefits for its customers. That serves to downplay its reliance on labor exploitation, monopolistic competition when it enters local markets, and political favors that structure labor and market laws to give it a competitive edge. And WalMart might just be believing the IT sales hype that it spends millions of dollars on. But that's no reason we should buy their IT BS as much as we seem to buy their wares.

  • What they do with computers... (Score:4, Interesting)

    by telemonster ( 605238 ) on Sunday November 14, 2004 @08:05PM (#10815960) Homepage
    A friend who worked briefly @ a local walmart during the downturn in tech employment told me about the huge datacenters. Evidentially he was told this in training, or a manager filled him in. Basically they are an IBM shop from what he said.

    The systems have the layout of every walmart store in them, and the stores respond to orders from the main office to move products around on the shelves. The systems will tell various stores to move products into different places, and anaylyze the results. If a store is making more money with XYZ sitting near the entrance, then the WOPR tells more stores the move that product into place, but still plays games against shoppers with a few more. It's basically an insanely well oiled statistical war against the shoppers to squeeze every last penny out of them. I hate to say it, but it doesn't work on me when I go there. But overall, it's creepy, and impressive at the same time.

    PS- I had this evil idea. If anyone is into the hactivism role, embed a voice recorder IC into a telephone set that matches your local WalMart's phones. Get the code to get on the PA system, and setup your "rouge" telephone to bump onto the PA every 5 hours or so. Be sure to include sounds to make it sound like someone is picking up the phone, and hanging it up. It will drive them nuts. Some stores seem to use Lucent sets on the wall (MLX-xxx) which are most likely ISDN on the back. Other stores seem to have analog ports on a lucent system. Just remember to give me props. Feel free to announce all shoppers a winner of a contest where they get everything they can stuff into a cart for free. Or remind them about the $700,000 in taxes the minimum wage making people cost the community at every WalMart.

  • Comment removed (Score:5, Interesting)

    by account_deleted ( 4530225 ) on Sunday November 14, 2004 @08:15PM (#10816019)
    Comment removed based on user account deletion

  • 640TB ought to be enough for anybody (Score:5, Funny)

    by StikyPad ( 445176 ) on Sunday November 14, 2004 @09:20PM (#10816406) Homepage
    'Wal-Mart has 460 terabytes of data stored on Teradata mainframes, at its Bentonville headquarters. To put that in perspective, the Internet has less than half as much data, according to experts.'

    Apparently the "experts," overlooked alt.binaries.*

  • I've actually worked on this data before... (Score:5, Interesting)

    by mbd1475 ( 18047 ) <markduch@noSpam.mac.com> on Sunday November 14, 2004 @10:07PM (#10816648) Homepage
    I graduated from the Sam M. Walton College of Business at the University of Arkansas with a B.S.B.A in Information Systems. Wal-Mart was nice enough to donate a big chunk (~1 Terabyte) of information for us to datamine. It's pretty interesting stuff and very CPU intensive, as you can probably imagine; we tried not to do any CD burning while waiting on our results ;)
    IIRC, It seems like one of the strange correlations we found is that the two items most commonly purchased together were beer and baby diapers. Go figure...

Slashdot Top Deals

It is easier to write an incorrect program than understand a correct one.

Close