Developers May Be Getting 50% of Their Documentation From Stack Overflow 418
New submitter gameweld writes "Software companies, such as Microsoft, create documentation for millions of topics concerning its APIs, services, and software platforms. Creating this documentation comes at a considerable cost and effort. And after all this effort, much documentation is rarely consulted (citation) and lacking enough examples (citation). A new study suggests that developers are increasingly consulting Stack Overflow and crowd-sourced sites over official documentation, using it as much as 50% of time. How should official documentation be better redesigned? What are the implications of software created from unruly mashups?"
Blame Google (Score:5, Insightful)
Whenever I have a problem, I google it, and StackOverflow is always in the top of the results. If Microsoft want me to use their documentation they better make sure google indexes it in a way than matches my queries.
These are two different use cases (Score:5, Insightful)
Documentation and asking others for help when you get stuck complement each other. You can't really learn to use something completely new on Stackoverflow, and you can't predict all the ways people will screw up or misunderstand you in a documentation.
Microsoft docs (Score:5, Insightful)
| Microsoft's MSDN website changes frequently, and is confusing to use (on some iterations of their website, on others it works better). Currently to find anything, you have to use the Bing search on their web page, and it doesn't always work well. I find myself using Google search to search for functions in MSDN, because I get better results.
As a result, if something for Stackoverflow comes up in the search results ahead of the MSDN docs, I'll probably look at that one. From that, I hypothesize that if people are looking at Stackoverflow, it's because they've done their SEO better (and probably have more motivation to do SEO).
It's not the the docs are bad or not used (Score:5, Insightful)
The problem is lack of usage examples and feedback. When you follow the API and your program doesn't work, the solution is to google your problem to find the solution from the 1000 others who have hit the same problem.
Vetted Examples (Score:4, Insightful)
Re:What does StackOverflow run on? (Score:5, Insightful)
A good coder will write good code even in a bad language. A bad coder will write bad code even in a good language.
But let a good coder use a good language, and you'll get great code. Just don't let a bad coder use a bad language, else you get, well, 90% of the stuff in VB6.
Unruly mashups? (Score:4, Insightful)
For some development problems it is far quicker to search sites like Stackoverflow for a question / answer / example relevant to your specific case than it is to read the official (often poor) documentation and figure out exactly how it is "supposed" to work.
Basically, someone else did the work - possibly found some "gotchas" - and shared the fruits of their labour. Remind me how is that a bad thing? Isn't this exactly what the World Wide Web was designed for? :-/
Peace,
Andy.
Astroturfing Detected (Score:3, Insightful)
All or nearly all of your comments are pro-Microsoft and anti-Microsoft competitors.
Unruly mashups? (Score:4, Insightful)
The nice thing about Stack Overflow (and such) is that someone, somewhere, has (usually) encountered the same problem I am currently working on. The official documentation I check when I want some basic examples on how to use something and what the different methods are supposed to do.
I may have created a few mashups from examples, but most of them weren't all that unruly. Perhaps the implication is that the wheel isn't invented all that often?
Re:Blame Google (Score:4, Insightful)
Whenever I have a problem, I google it, and StackOverflow is always in the top of the results. If Microsoft want me to use their documentation they better make sure google indexes it in a way than matches my queries.
..and when they do that, better make fucking sure they're not doing an annual switching around of documentation site urls during that time, landing you on a wild goose chase - only to find documentation that is vague about the answer for political reason or documentation that is just a brochure of the sdk with promises about what the sdk does but not actually telling how, why and what actually works.
AND I DON'T FUCKING WANT TO ANSWER A QUESTIONNAIRE ON MY FIRST FUCKING VISIT ON YOUR FUCKING DEVELOPER SITE. how the fuck am I supposed to know if I found what I came looking for?? (Nokia used to run these quizzes every 6 months. fucking annoying - especially when it was an indication of that they were putting a lot of money into their dev stuff but not getting much out of it). and I most definitely don't want a fucking pdf essay 10 pages in length that _might_ have the answer(but more probably has just lies about what they thought the api would behave like).
but yeah, stack overflow comes up so often on "how to do blabla on blebleble" things that it's where devs quite often end up. and why shouldn't they, if the answer is right there or a link to an answer. oftentimes it's more current than actual docs anyways.
a rather important aspect of this whole "getting information from" is that most devs are not asking questions and receiving answers - they're reading other peoples questions and answers to them because it's quite rare to have a really unique installation problem, a rare api bug that nobody else hit or a rare question about how to get some layout engine xyz to behave in ysz way. so it's not about being lazy, it's about finding information and saving time, not getting other people to do the work for them.
Re:Documentation Shitty so Developers Turn to Web (Score:3, Insightful)
Re:It's not the the docs are bad or not used (Score:4, Insightful)
When you follow the API and your program doesn't work,
That's a pretty good indication that the docs are bad
Re:Documentation Shitty so Developers Turn to Web (Score:5, Insightful)
Lots of truth to this.
What you get from MSDN must be read like a lawyer parsing the law books.
Miss some casual reference and the whole API call fails. Or worse, it almost always works, but
fails inexplicably on odd numbered Tuesdays.
Things also go missing. You will find something this week, only to find it missing with the next update to the website.
That said Microsoft documentation is still more extensive than most. I often start there then end up on Stackoverflow
of one of the other sites for more lucid examples, and often find that problems with a particular feature not working
as documented are common knowledge, except, apparently, to Microsoft.
How about actually trying to get it right? (Score:5, Insightful)
Minor rant, but look at the "InConnectionString Argument" section (which I can expand/collapse [useless] but can't link directly to, which is annoying) of this page [microsoft.com]. Try to read their grammar for a connection string. Confused yet? There are line breaks that have completely disappeared, causing words to merge together (e.g. "connection-stringattribute" should be "connection-string" with "attribute" being on a new line). I filled out the little "did you find this helpful" thing at the bottom of the page explaining the problem a year ago, and it hasn't been fixed. Dumping half-assed documentation on the web and not fixing (reported!) errors wastes the time of each individual developer that has to read/decipher it. The PHP online documentation is one of the most useful ones I've found, largely because it allows users to add comments/examples that make things clearer. Microsoft does the opposite -- not only can users not add to it, but the improvements that users suggest (through the "did you find this helpful" thing) are ignored. Perhaps all of the useful information is on StackOverflow because Microsoft doesn't allow it to be added to their own documentation.
More generally, it should be easy to bookmark pages (URLs should NOT break, even when new versions are released!) and sections within pages so it is easy to refer back to important things, as you could with paper documentation. Documentation for each function/object should link back to an overview that explains how it fits into things, and it should link to examples that show how all of the arguments (not just one special use case) works. Documentation should explain any differences between new/old behavior of any function/object because not everyone is developing for the latest version of the OS or development platform. And, just to beat a dead horse, users should be able to submit improvements/clarifications that actually get used.
Perhaps it's Give Me The Codez? (Score:5, Insightful)
A lot of Stack overflow questions I see are along the form of "I need to do X, how do i do it?".
Basically they want a HOWTO of which APIs to string together in order to accomplish their task, if not someone else to completely code it for them. This is often referred to as "task based" documentaiton - to do X, you do A, B, C, and D. This often fails if you need more details on individual API calls.
Official documentation like MSDN exist to document all the APIs, but often lack what's known as "task-based" documentation.
They're both required pieces - task based is often used to learn how to do things (e.g., how to create a window on Windows), while the API documentation serves to comprehensively adjust various settings (do you want a scroll bar? A resize box? etc). Unfortunately, putting in extensive examples inside such documentation often serves to confuse (you won't believe how many people assume you can copy and paste it into a program and have it run).
Unfortunately, Stack overflow also suffers from developers merely copying and pasting code and expecting others to do their work for them (see thedailywtf), as well as many "give me the codez" stuff posted by students wanting others to do their homework.
But when used properly, the two complement each other. Its like man pages versus HOWTOs on Linux - one documents the commands and APIs, while the other tells you how to properly string them together to accomplish things.
Re:Blame Google (Score:5, Insightful)
Only if you come from Google (or spoof your referrer to come from Google). They did that to evade de-listing for "deceptive indexing" or whatever.
Go to Expert Sex Change from DuckDuckGo or Qrobe, e.g., and you'll still find the answers behind their crappy little paywall.
How the hell do they even still exist, in a world with StackOverflow, anyway?
Re:You can learn something new from SO (Score:5, Insightful)
When all else fails, read the directions.
Documentation should be the absolute authority on every detail of a system's operation. It should be the reference material for experts. On the other hand, people who aren't experts don't know the available options, and often don't really know the terms to look for in the detailed documentation, and can't spare the time or effort to read (and grok) the whole documentation end-to-end. StackOverflow is a great place to describe the problem you have, and experts (who know the system more fully) can point you in the right direction or even provide a solution. Then you can read the relevant documentation to understand better what's going on, and hopefully provide similar help to others.
Do you use a dictionary to write an essay? (Score:5, Insightful)
When you write, do you form and choose your prose based on the dictionary on your desk (or online)? Of course not. A dictionary is the ultimate reference for words in your language, though. If you have a word, you can look up its part of speech, spelling, definition, pronunciation, even sample usage in some cases. But if you're writing an essay, or a book, or a brief, or a memo, a dictionary is very close to unusable. If you want to describe the action of a bipedal animal moving swiftly over land by means of propulsive contact with the ground, you're not going to find what word to use in a dictionary. If you don't know what the word run means, or how to use it, a dictionary is ideal.
Sometimes - no, often - the official documentation is exactly the *wrong* reference to use when creating from scratch. I'm not a programmer, but anyone who has ever even used software to do anything - from Autocad to Wordperfect - knows that the official documentation is almost never going to give you a useful answer to a problem you are having. You have to know the command to use before you can look it up. I still have programs whose documentation is a list of definitions, in order, of every menu and submenu command. And when I get stuck, I know that the answer I'm looking for is never going to be in that "help" file.
Re:What does StackOverflow run on? (Score:4, Insightful)
It depends on what you are using it for. PHP is great for sites with a very small number of pages. There are no bad languages, just languages being misused.
Morse Code is a great language for certain purposes. But you don't use it when speaking to someone in person. Baby talk is a great language for certain purposes. But you don't use it in a meeting with your bosses.
Re:Documentation Shitty so Developers Turn to Web (Score:3, Insightful)
This isn't just Microsoft documentation either. I wish you were exaggerating, as it always amazes me how little people must give a crap if they write this and think it's acceptable documentation.
Re:Documentation Shitty so Developers Turn to Web (Score:5, Insightful)
One of the most annoying things about the MS API documentation is all the unexplained dependencies. You see a function call that takes 2 structure pointers as parameters and returns another structure... now you've got to open 3 additional documentation pages to read what those structs mean. And they might contain other structures of their own, so soon you can be up to half a dozen or more tabs, all for one API call you want to perform.
Re:Documentation Shitty so Developers Turn to Web (Score:5, Insightful)
Microsoft and others writes a kiloton specific for the function it concerns, but nothing about the context in which it shall be used. Here's what StackOverflow is a lot better at - a lot of examples, some good, some not so good and some esoteric.
Re:What does StackOverflow run on? (Score:5, Insightful)
A bike is still a bike...so if you can get from point a to point b, why trade in your old bike for a newer and 2000$ prettier bike???
You wouldn't. But when the frame finally goes, you might consider replacing all of the other components and not just the frame. And if technology has moved far enough, it can get harder and harder to find an affordable frame that fits and works with your ancient components.
I'm not being obtuse... this is a real problem. There are whole factories that run on commodity DOS hardware from the early 90s. It starts to get very hard to replace that old hardware... at the point where you find yourself hitting eBay, you should probably accept that you are on borrowed time and plan a transition to newer stuff.
Re:What does StackOverflow run on? (Score:2, Insightful)
Define good and bad. Novice programmers generally fare better in more restrictive languages like C or Java. On the other hand, an experienced hacker can do amazing stuff in a powerful language such as C++.
Documenting Reality (Score:5, Insightful)
Documentation should be the absolute authority on every detail of a system's operation.
The Documentation has a huge problem. It lays out how the thing in question was MEANT to work.
What StackOverflow offers is understanding of how it ACTUALLY works (or doesn't).
Furthermore Documentation is written almost always with either very generic examples or examples imagined by the documentors or framework builders. StackOverflow offers examples from people who are trying to build something real that works.