US Census Bureau Offers Public API For Data Apps 47
Nerval's Lobster writes "For any software developers with an urge to play around with demographic or socio-economic data: the U.S. Census Bureau has launched an API for Web and mobile apps that can slice that statistical information in all sorts of nifty ways. The API draws data from two sets: the 2010 Census (statistics include population, age, sex, and race) and the 2006-2010 American Community Survey (offers information on education, income, occupation, commuting, and more). In theory, developers could use those datasets to analyze housing prices for a particular neighborhood, or gain insights into a city's employment cycles. The APIs include no information that could identify an individual."
Re: (Score:1, Interesting)
political power advantage (Score:2)
Re: (Score:2)
Yes, or they could use one many widely-available population density maps. This is going to make lots of cool things easier, and I really doubt its going to make terrorist plotting much easier than it already is.
Re:political power advantage (Score:5, Insightful)
yes but terrorists can use this type of information to locate populations centers and high value targets
You know what? Let's camouflage our cities in case the Jihadists find out where people live.
FFS.
Either there's a joke that I'm not getting or some people are sad sad individuals who see terrorists under every bed. It's like when that tall viaduct was built in France people here were posting "b ... b ... but won't terrorists want to blow it up?" It was the same every time something big or tall was built. You know what I say? Get. The. Fuck. Over. Yourselves.
You'd think terrorism was invented in 2001 to listen to some people. What do you think the rest of the world has been putting up with for decades? The Brits were having their town and city centres blown to pieces long before most USAians even heard of terrorism. But you know what? They didn't let it govern their lives. They carried on shopping in their town and city centres. They carried on building tall buildings out of glass. They carried on riding on trains at 110MPH. They carried on cramming into crowded buses and underground train systems. They didn't become a crowd of pathetic little scaredycats who lock themselves in the room and didn't move for fear that the terrorists would get them. Sure they took a few precautions (like stopping the left luggage service in train stations and removing trash cans from airports) but they didn't become paranoid neurotic wrecks.
Re: (Score:2)
They carried on shopping in their town and city centres. They carried on building tall buildings out of glass. They carried on riding on trains at 110MPH. They carried on cramming into crowded buses and underground train systems.
Let's see, do Americans shop in town and city centers? Yes. Do they build tall buildings out of glass? Yes. Do they ride on trains at 110 MPH? Yes, although only between Boston and D.C., because trains here generally suck. Do they cram into crowded buses and underground train systems? Yes.
Re: (Score:3)
OK how about commentary on this accurate part:
become paranoid neurotic wrecks.
Re: (Score:2)
Re: (Score:2)
Yeah, armed to the teeth. Where else in the world are people so paranoid that they feel the need to carry guns everywhere?
Re: (Score:1)
Yeah, armed to the teeth. Where else in the world are people so paranoid that they feel the need to carry guns everywhere?
Switzerland? Ok, maybe they don't carry them around everywhere. But most Americans would consider giving every young man a machine gun and a pack of ammo on their 18th birthday to be a little bit crazy. Because that is what Switzerland does, just in case France decides to invade.
Re: (Score:2)
BS. Only in America can you walk into a Wal Mart and buy a machine gun.
Re: (Score:2)
Re: (Score:3)
Any terrorist that has to use the US Census Bureau to find a population center, as opposed to buying a map from AAA or using Wikipedia, is an idiot.
Re: (Score:2)
the gov can use this data for themselves in the campaign.
What campaign?
Re: (Score:1)
Re: (Score:2)
Oh you mean the administration. When I hear "the government" in the USA it generally refers to institutions.
Re: (Score:1)
Re: (Score:1)
Re: (Score:1)
You do know that 'the gov' does not run the political campaigns, right? And that this information has been available for over 200 years (but not with a fancy API, so that makes all the difference)?
Even when it doesn't it does. (Score:2)
Whether or not the data contains any personal information is the real question because if there is any then someone will figure out how to get at it.
Even when personal information "has been stripped" it can be rediscovered in various ways.
For instance: At the start of WWII, US authorities used census data to round up people of Japanese descent. They didn't have the individuals' names. But they had the number on each block. So they just raided until they had accumulated that number.
Re: (Score:3)
For those who don't get it,
(statistics include population, age, sex, and race)
(offers information on education, income, occupation, commuting, and more).
Treat is as a multidimensional data source. So you figure out who someone is using perhaps 6 factors, then you've got the unknown data for the other 1315 data points.
I almost got in quite a bit of trouble at a previous employer by pointing out a public distributed incredibly detailed analysis of an "anonymous" corporate employee attitude survey mean it was completely 100% non anonymous. So... 100% of 25 year old engineers who are white single males who drive a red car and have a
Re: (Score:3)
all of this data has always been available to those who ask for it
they have just made it easier for people to get at it
what is your complaint again?
Re: (Score:2)
That would be the "they have just made it easier for people to get at it" part
Making it easier to mush databases together to gather inappropriate levels of personal data.
Re: (Score:2)
That's not how it works (Score:5, Informative)
That's not how Census information is either collected or stored. First off, there are two different data sources at issue - the decennial census, which gathers a very limited set of information on (theoretically) every person in the country, and the American Community Survey, which uses sampling to get estimates on a much wider range of information. You cannot link those two datasets, since the only public factors they share are far too broad - e.g., age, race, sex, etc., and the time periods during which they are conducted are totally different.
Besides, the information is not released at person-level. The lowest level you can get sampled information at (e.g., the detailed ACS stuff) is the "block group", which on average contains 39 blocks. You can get decennial census information at the block level, and a "block" may correspond to a city block, or a much larger area for lesser-populated areas.
So, you can find some interesting information about your city street (I've looked up my own, and found the number of people living alone, owning/renting, age, sex, etc. for the 24 houses on my block), but these data are not per person, they are per block - in other words, if there is only one Native American living on my street, I cannot then find out whether they are owning/renting. I can only find out the number of renters on the entire block.
Re: (Score:2)
So? (Score:2)
That paper points out that three factors - DOB, place, and gender - are often enough to uniquely identify a person. How is that relevant to Census Summary File information, or ACS information?
I think the point that many people misunderstand is that Census/ACS public information is not a database where each row represents one response, and some data items have been withheld. It's not at all like that - it's aggregate totals for geographic areas of varying sizes. That row-by-row information is not made public
Re: (Score:2)
The Census site has a little info about this: http://www.census.gov/privacy/data_protect [census.gov]
Re: (Score:2)
OK so am I correct in stating the TLDR version is Census dept doesn't release data containing logical AND statements? Or in SQL we'll let you have exactly one "GROUP BY" clause, sorta?
So in an example above, the census will release the number of native americans on a block and a separate table of number of renters, but would never respond to a query of "NAs AND renters" because that might get a response table of single digit rows etc. That Seems reasonable.
Its a PR campaign disaster to describe that as "a
Re: (Score:2)
Actually you *can* do that kind of multi-dimensional filtering, equivalent to multiple AND statements followed by a GROUP BY. There are different data sets here, with different usage models. Perhaps most interesting is the Public Use Microdata Sample (PUMS). Docs here: http://www.census.gov/acs/www/data_documentation/public_use_microdata_sample/ [census.gov]
PUMS contains records representing individual responses to the American Community Survey (ACS). These individual responses include detailed data including hou
Re: (Score:3)
census data has been public all along
before now you had to go to washington and look it up yourself
now it's easier to get at
Re: (Score:3)
census data has been public all along
before now you had to go to washington and look it up yourself
now it's easier to get at
No, it's been online for years. There just hasn't been a good, uniform way to query it and write apps against it.
Re: (Score:2)
Or load up the shapefiles posted at census.gov, seeing as census data has been available online for at least a few years now. As the summary said, this made it easier, but in the past couple of years there has also been an explosion in free and open source GIS tools that translate the raw data into something more readable.
Stripped-Down Data (Score:2)
I recently pulled the census data and it's pretty much useless since any information you could use to look at results by city or region have been stripped out in the version available to the general public. Sucks.
Not really... (Score:3)
I don't know what specifically you tried to do, but there is a lot of data available down to the block group and block level, which are relatively small geographic units. There's even more data available by "place", which would include any major city and many smaller cities and towns. Some of the tax data is redacted for confidentiality (e.g., when there is only one employer of a certain type in a geographic area, they won't release payroll information for it), but that's pretty unusual in larger areas.
You
A check on redistricting? (Score:1)