How Stack Overflow's Reputation System Led To Its Own Downfall (infoworld.com) 97
A new analysis argues that Stack Overflow's decline began years before AI tools delivered the "final blow" to the once-dominant programming forum. The site's monthly questions dropped from a peak of 200,000 to a steep collapse that began in earnest after ChatGPT's 2023 launch, but usage had been declining since 2014, according to data cited in the InfoWorld analysis.
The platform's remarkable reputation system initially elevated it above competitors by allowing users to earn points and badges for helpful contributions, but that same system eventually became its downfall, the piece argues. As Stack Overflow evolved into a self-governing platform where high-reputation users gained moderation powers, the community transformed from a welcoming space for developer interaction into what the author compares to a "Stanford Prison Experiment" where moderators systematically culled interactions they deemed irrelevant.
The platform's remarkable reputation system initially elevated it above competitors by allowing users to earn points and badges for helpful contributions, but that same system eventually became its downfall, the piece argues. As Stack Overflow evolved into a self-governing platform where high-reputation users gained moderation powers, the community transformed from a welcoming space for developer interaction into what the author compares to a "Stanford Prison Experiment" where moderators systematically culled interactions they deemed irrelevant.
Apt comparison (Score:5, Insightful)
Re:Apt comparison (Score:5, Insightful)
Re: (Score:2)
Can the comparisons be extended to the moderation system here and on Reddit? I definitely think moderation problems hurt Slashdot, but I stopped using StackOverflow so long ago that I can't even remember what I disliked. I have continued to peek at Reddit over the years but I have never found a use for that website (notwithstanding its prominence in some problem-oriented search results), and think the clannish moderation was much of that. (Some other website is on the tip of my keyboard's tongue, but I can'
Re: Apt comparison (Score:2)
I think social media and curation-by-algorithm vs. (what used to be) /.'s smart human overlords is what destroyed /.'s user numbers.
People could get their own curated feeds for their own bubbles first from friends, then eventually from algorithms. (I'm not saying that the latter are _good_ - just that many will bite on them).
Now, God only knows who decides what's "News for Nerds; important stuff".
(I say that a couple days after my first-ever accepted submission, years after it had an audience that would ove
Re: (Score:2)
To be fair, even if we had the 1999 number of users, or 2005, or whatever the height of numbers was, given our change in technology over the past 20 years, you wouldn't be "Slashdotting" any server anymore.
Re: (Score:2)
Moderation is just fine here. Browse at +1 if you want or -1 like I do to see all the comments. The only thing I would like to change about Slashdot mod system would be posting or moderating. Why not both? I read at -1, so I see everything.
Of course, in the old days, you could browse at +2 and not missing anything. Now, that's impossible because we let all the Septembers online. They downvote you when the disagree instead of countering your arguments with their own. Troll post just means I got in a good jab
Re: (Score:2)
I agree with all that (but can't up-vote you).
And how did Slashdot respond? Stagnation. I guess their poorly-received and reversed UI overhaul, and a smugness from being the first with a voting system, led them to believe that their system is perfect and should not be experimented with.
Re: (Score:2)
I think the mod points should be more available but reported using log values to effectively summarize larger numbers of votes. And I also think it should be possible to see more posts that have categorical upvotes. Easy example is the mostly insightful votes that have a couple of funny votes. I think there are too many insightful mods for posts that are only slightly insightful, but the ones with some humor in them tend to be more interesting.
I have adjusted my filters but I think that I should adjust them
Re: (Score:2)
Reddit's use of AI for handing out bans is getting concerning.
Re: (Score:2)
String length API unchanged? Facepalm. (Score:2)
unless there is a discovery in calculation of length of a string.
Incidentally, there was such a discovery. 'It's not wrong that "[facepalming man with brown skin emoji]".length = 7' by Henri Sivonen [hsivonen.fi] came out in September 2019. It explains the difference among code units, code points, and extended grapheme clusters, the difference among UTF-8, UTF-16, and UTF-32, the difference among JavaScript, Python 3, and Rust length semantics, and the difference among storage, display width, and arbitrary quotas that are roughly fair across languages.
Re:Apt comparison (Score:5, Insightful)
I think part of the problem is that external search engines would simply put worse interactions to the top based on some metric known only to them, so there was a desire to eliminate things that were superfluous, to channel searchers into the places that have their answers.
But of course it ran into the exact problem you described- generally a class of moderators wants that position for some reason. You're looking for the moderators who share a vision of a really useful place where everything works great, but many moderators will just be there to enforce some value (sometimes political) or because they have a keyboard sadism streak and that's that. Basically when you take volunteers for "who wants to have power", only some of the people coming forward want to use that power for thing you want, the others will do that to fulfill the job role but they'll REALLY use it for $LAME_THING.
Re: (Score:2)
Basically when you take volunteers for "who wants to have power", only some of the people coming forward want to use that power for thing you want, the others will do that to fulfill the job role but they'll REALLY use it for $LAME_THING.
Interestingly, that is the crux of this argument against trusting LLMs.
No matter what task we give them and no matter what outputs we may see on the performance metrics we choose to look at, we don't know - indeed never can know - whether they are in fact doing what we assigned and ONLY what we assigned, or are they superficially doing what we assigned while doing other things (or fully intending to do other things the instant they can do so without negative consequence).
Re: (Score:2)
Failed to insert link.
this argument against trusting LLMs [springer.com].
Re: (Score:2)
Re: Apt comparison (Score:2)
On another forum a user recently complained because someone had edited their question on Stack Overflow. The final question no longer represented what they were asking about and expressed distain for the edit; someone else on the forums who has enough editing powers in Stack Overflow to do such then justified it. Or tried to.
Editing a users question as if it was the original question they asked is NOT ok.
Refreshing the question and answering that version of it may be useful, but sometimes obliterates the su
Re: (Score:2)
Also, they don't allow you to completely delete questions (or answers). They are only hidden and can be read by high-reputation users who can also undelete them. They also state there is a limit (without specifying how high it is) after which deleting questions and answers affects your account's reputation. They suggest editing the answer to change it from a low rating (possibly for being incorrect) to a neutral or high rating again, instead of implementing a working delete button to remove wrong answers yo
Re: (Score:2)
Re: (Score:2)
Can't an original asker reject an edit? I've done this. It's satisfying!
Re: Apt comparison (Score:2)
I've experienced this, and the retards who edited my questions and answers finally drove me away from that dumpster fire.
Some retard who doesn't understand fuck all about anything edits things so they're at best dubious, and at worst completely misleading. But the dumpster fire of a site still shows me as the author.
I've seen questions edited by retards so that the question matches the *wrong* answer the same retard posted.
I've seen my *correct and detailed* answers on highly specialised topics edited by re
Re:Apt comparison (Score:5, Insightful)
Some of the issues with asking the same questions over and over are:
- More interesting topics getting drowned out by the same beginner questions repeating over and over
- The questions same questions not always getting the same answers. In the end the experts won't revisit the same question for the 50th time, and beginners will start answering beginners.
- No knowledge aggregation doesn't really happen. Rather than the topic of that question getting dug up from time to time when someone asks for more clarification, and thus a knowledge base building around that topic, you might have to wade through dozens of the same question to find the one with an answer that's applicable to your situation, rather than one question that explores it in depth and may help you further way better.
As a millenial that grew up with forums, the above things are also what bother me most seeing most support groups now being on reddit, facebook or discord. Everything is just the same shallow questions repeating, no real in depth discussions, and unless you're terminally online, distilling deeper knowledge is near impossible.
On a forum you might find a thread on a topic that interests you, and has hundreds of comments, and you can choose to do a deep dive into that topic if you have an hour to waste on reading it when it works for you. On discord/facebook/reddit, if you're not always following the flow, good luck finding back things that were posted a mere month ago on a topic that interests you.
Re: (Score:3)
"beginners will start answering beginners."
That's not bad. Sometimes a beginner can better explain to a beginner than an expert. Also you learn a lot by teaching others.
Re: (Score:2)
Re: (Score:2)
Re: Apt comparison (Score:3)
The purpose of the site is clearly spelled out. It's for professionals. Not for students who need help with their homework.
Re: (Score:2)
It is definitely bad (Score:1)
This is why reddits like /r/electricians had to ban the hoi polloi -- a beginner-answerer will say dumb shit half the time. They won't know about issues they have never seen and they probably misunderstand what they do think they know.
Yes, it's true that someone who's just a random expert will not necessarily be a good teacher. The expert will take so much for granted and not know where a beginner will be confused or lost. But someone who habitually teaches will be absolutely superior to a beginner in expla
Re: (Score:3)
Some of the issues with asking the same questions over and over are: - More interesting topics getting drowned out by the same beginner questions repeating over and over - The questions same questions not always getting the same answers. In the end the experts won't revisit the same question for the 50th time, and beginners will start answering beginners. - No knowledge aggregation doesn't really happen. Rather than the topic of that question getting dug up from time to time when someone asks for more clarification, and thus a knowledge base building around that topic, you might have to wade through dozens of the same question to find the one with an answer that's applicable to your situation, rather than one question that explores it in depth and may help you further way better.
As a millenial that grew up with forums, the above things are also what bother me most seeing most support groups now being on reddit, facebook or discord. Everything is just the same shallow questions repeating, no real in depth discussions, and unless you're terminally online, distilling deeper knowledge is near impossible. On a forum you might find a thread on a topic that interests you, and has hundreds of comments, and you can choose to do a deep dive into that topic if you have an hour to waste on reading it when it works for you. On discord/facebook/reddit, if you're not always following the flow, good luck finding back things that were posted a mere month ago on a topic that interests you.
I would think that repeated questions would be a valuable data point if your goal is to provide information. FAQ doesn't necessarily have to be an invented category, it can be based on actual frequency. A repeat question can simply be marked as a repeat with a link to the FAQ thread. That thread then provides a location for more in depth discussions and alternative answers. If you add slashdot style "moderator" points to highlight the most substantive answers to the FAQ question then you have a very useful
Re: (Score:2)
Some of the issues with asking the same questions over and over are [...]
You're not wrong, but StackOverflow's methodology for handling this problem proved (in hindsight) to be inadequate, because it maintained the experience quality for established users at the expense of new users, and a site like StackOverflow needs both kinds of users to thrive.
A better mechanism might have been to allow repetitive newbie questions, allow people to answer them as well as they care to, and then have an asynchronous "garbage collection" background process (either human-based or automated) that
Re: (Score:2)
They misunderstood their own concept.
People use a search engine.
The results are not "good enough", might be the search engines fault or the exact wording of the search term fault.
However the "side where we can ask questions" pops up. So they go there. And as a question, instead of using the search function on that side.
On the other hand, basically every question on SO that is closed as dup of another one: isn't a dup but a significant variation.
For quite a while I simply reopened all questions, but the aske
Re: Apt comparison (Score:4, Insightful)
Have never understood this behaviour. One can see it also on Wikipedia and Reddit.
Re: (Score:2)
Have never understood this behaviour. One can see it also on Wikipedia and Reddit.
Erm... this phenomena is well understood and isn't limited to online communities.
See John Gabriel's Greater Internet Fuckwad Theory for more details [penny-arcade.com]
As I said, this behaviour isn't limited to online communities, anywhere where tribalism is permitted to take root lets a bunch of people take control and use that power to enforce their will/ideology. Prisons are just the start, sporting clubs and high schools are also prominent examples of cliques and tribalism where people who don't follow and do or jus
Re: Apt comparison (Score:1)
I can not ask a question on stack overflow after 10+ years of use, same reason I stopped using Reddit, fucking elitist cunts!
Re: Apt comparison (Score:2)
Asking questions doesn't require reputation. Did you get yourself banned?
Re: (Score:1)
Re:Apt comparison (Score:5, Interesting)
As someone who experienced this decline, I think the comparison to the Stanford Prison Experiment is very apt. It started out amazing and then became a private club for cruel people to claim their turf.
It's the same problem that Wikipedia had. The organizations running the platforms are incredibly happy about all of the free labor they're getting, but never stopped to consider what kind of people would gravitate towards those positions. I'm reminded of an old quote to the effect of "The vicious pettiness of politicians being inversely proportional to how consequential they are." Low stakes moderation positions seem to attract some of the worst people who become little tyrants in their own callous way.
Re: (Score:2)
The infuriating thing is these people are usually correct. They are like the ultimate petty bureaucrats, happy to put their mother on a train because that’s the rules.
Re: (Score:2)
sounds like you are describing wikipedia.
Re: (Score:2)
Re: (Score:2)
I bet there is a LARGE over representation of autists among the the SE moderators. Personal experience.
I don't know how many times I have had a question closed because it was "unclear" AFTER someone already answered it with a useful or correct answer.
Translation: (Score:5, Insightful)
the community transformed from a welcoming space for developer interaction into what the author compares to a "Stanford Prison Experiment" where moderators systematically culled interactions they deemed irrelevant
Translation: they started power trippin'. Enshittification ensued.
Re: Translation: (Score:2)
More to the point, they added a bunch of meta moderation facilities. My read is the article is not complaining about the basic question and answer voting. It's complaining about all the other stuff that was added later.
I constantly get requested to moderate the moderators, and I can tell you the interface makes it hard to understand the context of what is being asked and answered. My guess is that led to a lot of weird meta moderation decisions which drives the animosity.
No one is out to get you. No one eve
I think I said something like this last week (Score:2)
SucksOverflow (Score:3)
Most of the time what's on StackOverflow is completely useless to the problem you yourself are trying to solve.
Firstly, because the problem that is described in the post is very specific to the author's context.
On top of that, 99% of responses to the original post are irrelevant. People clutching at the straws when having no clue what the root cause is themselves. No intention to find out what the root cause is either. Just throw a random thought at the post and hope for the best.
If you're lucky, you sometimes get a clue as to what may be causing your own issue but it's usually after trying 20 different posts and by complete accident. An API call you didn't know about or a library you haven't considered.
But apart from that rare inspiration, StackOverflow is completely useless if you're looking for an answer.
Re:SucksOverflow (Score:5, Informative)
Actually no. I have found many very useful answers on StackOverflow. If you don't, maybe its you?
That's not to say that the growing enshitification hasn't been obvious for years, basically since the original founders sold it.
Re: (Score:2)
I have found many very useful answers on StackOverflow. If you don't, maybe its you?
It might be a personal problem, but I suspect it might also depend on the category of questions the questioner asks. For example, a person asking a C++ question on StackOverflow might have a very different experience from a person asking a CSS question on StackOverflow, simply because the development communities associated with the two languages have different cultural assumptions and thus attract different types of answer-ers.
Re: (Score:2)
Re: SucksOverflow (Score:2)
"the problem that is described in the post is very specific to the author's context"
That's not their fault. That's your search engines fault. Or your search-fu.
Re: (Score:2)
Most of the time what's on StackOverflow is completely useless to the problem you yourself are trying to solve.
That is absolutely not true.
If you know how to have a grey background text in a Java Swing text field, when the field is empty: good for you. I did not know. And result is: you have(had) to program it yourself - which is not super complicated, but not easy to figure out yourself.
You want a text widget that displays text vertically: same same.
You want to know why an async function and await in Dart
never (Score:5, Funny)
self-governing platform where high-reputation users gained moderation powers
Yeah. Never, ever, do that. I've run a few online communities. Back when your own forum was still a thing and you could survive without being a group on Facebook, a subreddit or a Stackoverflow.
Your most active users aren't always your best users, and they almost always are NOT the ones you want to have as moderators.
If I could do all that again, I would give mod rights to the people who contribute just a bit, but consistently over a long time, and who read more than they write.
Re: never (Score:2)
Correct. If you want moderators - employ them; make them work for you.
But that's what you get if you don't want to pay, if you just exploit people.
Same thing happening with Wikipedia (Score:5, Insightful)
AI Database of Answers (Score:2)
Re: (Score:3, Interesting)
I will still trust a troll moderated Wikipedia over AI the drivel that is 95% truth and 5% complete Guillermo del Toro nightmare hallucinations. Even though I have stopped contributing to Wikipedia as it is impossible for me as an unmerited nobody to post anything without having it rolled back with a comment of "irrelevant", "wrong forum","lacking citation" (that was my fault) or "speculation" (yes I did speculate about a book cover being drawn in the same style as the Epic Adventure movie posters, like New
Re: (Score:2)
This will ultimately be the downfall of Wikipedia as many of the top admins are retiring because they didn't recruit fresh talent.
Encyclopedias can just get complete. There are only so many topics that truly belong there, and there's only so much polish to do. So eventually you end up with people editing mostly articles about the recent/current events, and these are often controversial.
Re: Same thing happening with Wikipedia (Score:2)
Wikipedia lost its way a long time ago. I created and updated a couple of pages where I was fairly expert. Eventually, some admin type started complaining that my style didn't follow guidelines. The specific points were, IMHO, stupidly trivial and some were arguably grammatically wrong. I decided, if that's how they treat free labor, they could do without my help.
It seems that most of the casual authors have left. Wikipedia is to be dominated by people who live for nothing else.
It's still a good site for
Like money, political power, etc. (Score:5, Insightful)
As Stack Overflow evolved into a self-governing platform where high-reputation users gained moderation powers, the community transformed from a welcoming space for developer interaction into what the author compares to a "Stanford Prison Experiment" where moderators systematically culled interactions they deemed irrelevant.
Start on an even keel, with a community that rewards relevance (new, competitive economy), some people gain status (wealth) and use that to reshape the environment to their liking, until (billionaires) they get to the point where the can effectively run the whole show in such a way that punishes people for having needs rather than helping them succeed (medical debt, college costs, low wage jobs, etc).
You don't have to feel the same way I do, but I stand by the claim that the analogy is 1:1 on the details I referenced.
RTFM (Score:3)
RTFM has always been a problem in our industry. I can remember in Usenet days asking questions and getting responses like RTFM or "I found your answer over here on http://goatse.cx/ [goatse.cx] ..."
SO works until everyone gets tired of answering the same questions over and over because the posters don't want to RTFM, they just want a quick answer.
And so AI is killing them now because AI doesn't care how many times someone asks a previously asked/answered question, it will happily reply to every question as best it can (even though it may hallucinate a response, it's still better than RTFM).
Re:RTFM (Score:5, Insightful)
And where exactly *is* the manual these days? Most of what I see has been mechanistic doxygen style nonsense that isn't crafted to explain why and how, only to restate basic facts about what.
Most of the questions that I find myself ending up at SO with are regarding how to achieve a particular effect, then once a human being actually explains the key points, I can find the rest. Google has become a crutch that has let software developers ignore good docs for the past twenty years, and AI summaries are just another attempt at evading blame.
Re: RTFM (Score:3)
There's a special place in hell for people who use document generators that just spit out the API interface without any explanation.
Re: (Score:2)
Re: (Score:3)
Many things have no (RT)FM.
Or to get you to the point, to know how to do it, needs 3 month of reading yadda, yadda, bla, bla.
I know how to make a "combobox" in Swing, (dozens of old GUI libraries), Qt and so on ... and Flutter: has no combobox. And if you search for "how do I make a combobox in Flutter" the results are not telling you which widget or class to use: but results of people who do not know what a combo box actually is, same like "uh, what is a RadioButton - grandpa?"
Insulting people that they do
My 2 cents (Score:2)
Re:My 2 cents (Score:5, Informative)
I created an account, posted one correct answer to a relatively simple question, and was downvoted. So I logged out. That was many years ago.
I still find it useful when it comes up in a search.
And it's better than Reddit. I don't even bother clicking reddit links that come up in a search.
Re: My 2 cents (Score:2)
On Stack Exchange, if someone voted you down, they actually expend their own reputation to do so.
Your answer must have been really bad.
Re: (Score:2)
On Stack Exchange, if someone voted you down, they actually expend their own reputation to do so.
Your answer must have been really bad.
You'd think so, but I've seen plenty of very good answers get downvotes as well, and if you look in the voting records you'll see certain SO accounts that simply carpet-bomb every question and every answer with a downvote, reputation costs be damned.
Why they do that, I don't know. Some weird form of trolling?
Re: (Score:2)
Re: My 2 cents (Score:2)
You don't need reputation to answer a question. Or ask one.
Reputation is required for things like commenting or rating another users votes.
https://stackoverflow.com/help... [stackoverflow.com].
How Stack Overflow's politics led to it's downfall (Score:1)
StackOverflow has messed with their users multiple times and now wonders why users don't like being there anymore.
- They fired a moderator for criticism, against the will of all users in her community.
- They implemented a new CoC most people didn't agree with, then only fixed some minor issues instead of the main points.
- They changed the post license retroactively and only fixed that after protest.
- They stopped publishing data dumps (which were a promise to the community that they wouldn't monopolize the
A tool like Stackoverflow is needed (Score:2)
The Stanford prison experiment (Score:2)
The problem they had was in order to eliminate junk and spam they prevented duplicate questions.
But technology changes constantly so what would happen is they would block duplicate questions and an old outdated question would be the one everybody found and it would cause confusion and problems.
This is actually also broken the AI systems. I tried to use AI to write a bit of python3 code for a project and it kept giving me python 2 code. Even if I explicitly told it I w
Re: (Score:2)
Has long since been debunked.
Wrong.
There are serious questions about how scientifically valid it is, however, to call it debunked is a gross abuse of scientific reasoning.
Pointing to potential problems with the scientific inference that can be made with a particular trial (which then have largely been borne out as factual either way, over time) does not "debunk" a thing. It is part of the scientific process of improvement of knowledge.
Re: (Score:2)
"the sadism and submission displayed in the SPE was directly caused by Zimbardo's instructions to the guards and the guards' desire to please the researchers. In particular, he has established that the guards were asked directly to behave in certain ways in order to confirm Zimbardo's conclusions, which were largely written in advance of the experiment."
Re: (Score:2)
In his opinion
You just tried to peddle someone's opinion on an outcome as objective fact.
Re: (Score:2)
And they did it tersely and rudely.
Stack Overflow tried to be a store of knowledge more than a discussion forum, and that directly or indirectly motivated the moderators to quickly close duplicate questions. I preferred to keep every question open for at least 24 hours to give the asker time to clarify their question and the answerers time to provide a personalized response that answered the question, even when
Re: (Score:2)
Lord of the Flies then. Or any internet community ever.
When people start calling it a "community" seems to be the first sign of the apocalypse. Right after that comes "for the good of the community" and then it's definitely time to move.
Re: (Score:2)
Then it obviously was not trained on Python3, I had asked a different AI ...
I have the feeling most LLMs are stuck on somewhere around Java 8 ... but I did not really try to figure that yet.
When I feed training sessions to them, I only use Java 8, without mentioning a Java version, and as far as I can tell: the answers were aimed for Java 8 as they compiled and ran (sans the mistakes the LLM made).
Another reason (Score:2)
Is that a lot of questions have already been answered. But the answersers are also picky about the questions and drive new users off. It would be nice if they would open up the communities again.
The results of an echo chamber built on censorship (Score:2)
Reputation pyramid (Score:3)
Answering the most basic questions of the programming world FIRST grants you unlimited points. Meanwhile actual hard problems are like 1 point because they are obscure.
StackOverflow for Teams? (Score:3)
I worked at a big bank and knowledge management was an after-thought. Confluence was seen as a "A place where knowledge goes to die". I heard about StackOverflow for teams and thought it was a good fit -- we have a lot of siloed (even worse with Covid/WFH) developers and a lot of large complex systems. Just knowing who to ask was often a mystery. Having a place to ask and share would have been a great tool.
Unfortunately management churn ensured it never made it to the funding stage, and then co-pilot (which totally lacked our institutional knowledge) was posited as an alternative. That was the end of that. Sad.
What really killed it. (Score:2)
What really killed it is pointy headed morons replying "JuSt GoOgLe It" to every damned question. As a result, any attempt to google it just returned a few pages of said morons and no useful answer. If there was an actual answer on stack overflow, it was a needle in a haystack.
schadenfreude (Score:1)
I grew to positively *detest* the site after some really high-handed behaviour by one particularly sanctimonious "moderator". At one point it tipped the scales for me; I canceled my account and deleted as much of my contributions as I could.
So it feels really good to read this.
What I hated (Score:2)
What I hated was when I asked a question like 'how do I call foo()'
I woul get answers "I'm so much smater" answers like:
1) why are you calling foo()
2) what are you trying to do
3) you shouldn't be calling foo()
4) send us all your code
Etc.
I would just look elsewhere...