Does AI Really Make Coders Faster? (technologyreview.com) 125
One developer tells MIT Technology Review that AI tools weaken the coding instincts he used to have. And beyond that, "It's just not fun sitting there with my work being done for me."
But is AI making coders faster? "After speaking to more than 30 developers, technology executives, analysts, and researchers, MIT Technology Review found that the picture is not as straightforward as it might seem..." For some developers on the front lines, initial enthusiasm is waning as they bump up against the technology's limitations. And as a growing body of research suggests that the claimed productivity gains may be illusory, some are questioning whether the emperor is wearing any clothes.... Data from the developer analytics firm GitClear shows that most engineers are producing roughly 10% more durable code — code that isn't deleted or rewritten within weeks — since 2022, likely thanks to AI. But that gain has come with sharp declines in several measures of code quality. Stack Overflow's survey also found trust and positive sentiment toward AI tools falling significantly for the first time. And most provocatively, a July study by the nonprofit research organization Model Evaluation & Threat Research (METR) showed that while experienced developers believed AI made them 20% faster, objective tests showed they were actually 19% slower...
Developers interviewed by MIT Technology Review generally agree on where AI tools excel: producing "boilerplate code" (reusable chunks of code repeated in multiple places with little modification), writing tests, fixing bugs, and explaining unfamiliar code to new developers. Several noted that AI helps overcome the "blank page problem" by offering an imperfect first stab to get a developer's creative juices flowing. It can also let nontechnical colleagues quickly prototype software features, easing the load on already overworked engineers. These tasks can be tedious, and developers are typically glad to hand them off. But they represent only a small part of an experienced engineer's workload. For the more complex problems where engineers really earn their bread, many developers told MIT Technology Review, the tools face significant hurdles...
The models also just get things wrong. Like all LLMs, coding models are prone to "hallucinating" — it's an issue built into how they work. But because the code they output looks so polished, errors can be difficult to detect, says James Liu, director of software engineering at the advertising technology company Mediaocean. Put all these flaws together, and using these tools can feel a lot like pulling a lever on a one-armed bandit. "Some projects you get a 20x improvement in terms of speed or efficiency," says Liu. "On other things, it just falls flat on its face, and you spend all this time trying to coax it into granting you the wish that you wanted and it's just not going to..." There are also more specific security concerns, she says. Researchers have discovered a worrying class of hallucinations where models reference nonexistent software packages in their code. Attackers can exploit this by creating packages with those names that harbor vulnerabilities, which the model or developer may then unwittingly incorporate into software.
Other key points from the article:
But is AI making coders faster? "After speaking to more than 30 developers, technology executives, analysts, and researchers, MIT Technology Review found that the picture is not as straightforward as it might seem..." For some developers on the front lines, initial enthusiasm is waning as they bump up against the technology's limitations. And as a growing body of research suggests that the claimed productivity gains may be illusory, some are questioning whether the emperor is wearing any clothes.... Data from the developer analytics firm GitClear shows that most engineers are producing roughly 10% more durable code — code that isn't deleted or rewritten within weeks — since 2022, likely thanks to AI. But that gain has come with sharp declines in several measures of code quality. Stack Overflow's survey also found trust and positive sentiment toward AI tools falling significantly for the first time. And most provocatively, a July study by the nonprofit research organization Model Evaluation & Threat Research (METR) showed that while experienced developers believed AI made them 20% faster, objective tests showed they were actually 19% slower...
Developers interviewed by MIT Technology Review generally agree on where AI tools excel: producing "boilerplate code" (reusable chunks of code repeated in multiple places with little modification), writing tests, fixing bugs, and explaining unfamiliar code to new developers. Several noted that AI helps overcome the "blank page problem" by offering an imperfect first stab to get a developer's creative juices flowing. It can also let nontechnical colleagues quickly prototype software features, easing the load on already overworked engineers. These tasks can be tedious, and developers are typically glad to hand them off. But they represent only a small part of an experienced engineer's workload. For the more complex problems where engineers really earn their bread, many developers told MIT Technology Review, the tools face significant hurdles...
The models also just get things wrong. Like all LLMs, coding models are prone to "hallucinating" — it's an issue built into how they work. But because the code they output looks so polished, errors can be difficult to detect, says James Liu, director of software engineering at the advertising technology company Mediaocean. Put all these flaws together, and using these tools can feel a lot like pulling a lever on a one-armed bandit. "Some projects you get a 20x improvement in terms of speed or efficiency," says Liu. "On other things, it just falls flat on its face, and you spend all this time trying to coax it into granting you the wish that you wanted and it's just not going to..." There are also more specific security concerns, she says. Researchers have discovered a worrying class of hallucinations where models reference nonexistent software packages in their code. Attackers can exploit this by creating packages with those names that harbor vulnerabilities, which the model or developer may then unwittingly incorporate into software.
Other key points from the article:
- LLMs can only hold limited amounts of information in context windows, so "they struggle to parse large code bases and are prone to forgetting what they're doing on longer tasks."
- "While an LLM-generated response to a problem may work in isolation, software is made up of hundreds of interconnected modules. If these aren't built with consideration for other parts of the software, it can quickly lead to a tangled, inconsistent code base that's hard for humans to parse and, more important, to maintain."
- "Accumulating technical debt is inevitable in most projects, but AI tools make it much easier for time-pressured engineers to cut corners, says GitClear's Harding. And GitClear's data suggests this is happening at scale..."
- "As models improve, the code they produce is becoming increasingly verbose and complex, says Tariq Shaukat, CEO of Sonar, which makes tools for checking code quality. This is driving down the number of obvious bugs and security vulnerabilities, he says, but at the cost of increasing the number of 'code smells' — harder-to-pinpoint flaws that lead to maintenance problems and technical debt."
Yet the article cites a recent Stanford University study that found employment among software developers aged 22 to 25 dropped nearly 20% between 2022 and 2025, "coinciding with the rise of AI-powered coding tools."
The story is part of MIT Technology Review's new Hype Correction series of articles about AI.
It depends on your skills level (Score:5, Insightful)
It depends on your skills level. For trivial beginner stuff, it's OK but then again.
For anything out of mainstream which no or very few examples are available for the model to train, it's pretty much useless.
Re: (Score:3)
Watch for the AI bubble crash in 2026.
Re:It depends on your skills level (Score:4, Insightful)
As much as I hate seeing a brute-force approach burn huge amounts of electricity to enable morally dubious applications based on inputs that are often corrupt or illegal, I think the AI bubble is as likely to pop as the Bitcoin bubble.
(You might ask: "Do you mean that AI is a brute-force approach that burns huge amounts of electricity, etc., or that Bitcoin is?" To which I answer: "Yes.")
Re: (Score:2)
Bitcoin doesn't lie on what it is and what it does. AI companies on the other hand ...
Re: (Score:2)
Well, plenty of blockchain proponents and companies do/did.
Re: It depends on your skills level (Score:2)
Not explicitly, but the idea that bitcoin holds "value" is an extremely tenuous concept relying entirely on duping human beings into using it. Crypto is often called a scam or a pyramid scheme and I tend to agree. All of the original ideas of a decentralised economy are there, but what actually drives bitcoin is the himan inclination towards gambling. It's very much like AI in that once you get beneath the surface things take dark turn.
Re: (Score:2)
Bitcoin has exactly the same intrinsic value as any fiat currency in circulation: zero.
Re: It depends on your skills level (Score:2)
Yeah, brining down developer productivity 20% as mentioned by the summary is bringing GREAT value. Sure.
Re:It depends on your skills level (Score:4, Insightful)
The bicoin bubble popped nearly a decade ago. You might remember that time when an iced tea beverage company changed their name to blockchain and shot up in value.
Just because bitcoin's still around doesn't mean it hasn't popped - does the existence of Amazon, Microsoft, Google, eBay, and dozens of other companies today mean the dot-com bubble never happened?
The AI bubble popping will bring these companies back down where we aren't going to see the giant circlejerk of Oracle buying Nvidia who invests in OpenAI who then places orders for more Oracle compute.
(And it might be cracking - given Oracle was recently rejected for billions of dollars in loans as their existing debt might not be repaid).
Doesn't mean we won't have ChatGPT at the end of the day, but we probably will see a slowdown - we went from GPT-3 to GPT-4 to GPT-5 in a few years, when it took far long for GPT-2 to come out and be "scary".
Some bubbles just pop and disappear, like NFTs. Other bubbles pop and the market goes saner but we still have the benefits, like the dot-com, blockchain and likely AI. Chances are we will see AI used strategically rather than it being the solution to everything. And several things have come out of it - including awareness of deepfakes
Re: It depends on your skills level (Score:2)
AI is killing bitcoin
The bitcoin data centers can make more money selling services to AI. While they will operate in parallel for a while I expect that to be displaced over time.
Bitcoin after 15 years has no easy practical use. Venmo is more useful than bitcoin. Even the big blockchain programs have mostly wound down or have collapsed in scope. There are still a few but all are now small time or specific use cases.
Once mining becomes unprofitable bitcoin will collapse. By design bitcoin is set to be unpr
Re: It depends on your skills level (Score:2)
Bitcoin sees a strong use case for illegal transactions.
Some of the rich and powerful like it because it allows them to do transactions with rogue governments such as Iran, Russia, North Korea, to exchange oil, drugs and weapons without using USD.
So overall it is a trading facilitator on black markets where the mighty dollar was bringing too much attention.
What percentage of the bitcoin value is based on the black market? IDK, but I really suspect that is is not negligible.
The only time I tried to get crypt
Re: (Score:2)
Now with Memecoins, there is no need for Bitcoin to launder money. Bitcoin is down for the year, too?
Re:It depends on your skills level (Score:4, Interesting)
Right now we are in the “first hit is free” phase of trying to get everyone hooked on this AI crap. All these upstarts are trying to get marketshare and get some spaghetti to stick to the wall and the usage is heavily subsidized by all the startup money gushing in. Once the influx peters out and these places have to pay their own rent we will see the reality of which companies are able to actually survive and which of the horde are houses of cards.
I fully expect there to be plenty of actual applications, but it will peter out to much more mundane than currently advertised.
Re: (Score:2)
Watch for the AI bubble crash in 2026.
"watch for" or "brace for"?
Re: (Score:2)
That's just what the LLM is trained on, rather than anyone's skill at using the LLM.
Re: It depends on your skills level (Score:3)
Re: (Score:3)
Aw bless your heart.
Re:It depends on your skills level (Score:4, Insightful)
I am a professional with decades of experience. I think that AI is best at prototyping. If I have an idea, I can ask AI to write a program that does that. That program does not work properly, it is full of bugs, but it will instantly tell me a lot of things. Most valuable thing it can tell me is that "this idea does not work". That alone can save hours of my work. It will also reveal things that I didn't even think about, but which are essential for the idea to work.
Because the code is just used as a prototype, bugs and quality doesn't matter. But because I have that prototype, I can write the actual program with much more accurate plans, making the actual code much better in quality.
Re: (Score:2)
I agree with your approach,
Re: (Score:2)
It depends on your skills level. For trivial beginner stuff, it's OK but then again.
For anything out of mainstream which no or very few examples are available for the model to train, it's pretty much useless.
I agree with you. I find it also true that free, open-source frameworks with tons of documentation and examples on GitHub/gitlab and the 'net work well with artificial intelligence. Especially Drupal because it is so heavily API driven (14 major APIs I think, last time I asked AI to list them all). The major LLMs, especially Claude are well-trained on both Drupal and Ansible in my experience. So go vibe-code a website and deploy it [ansistrano.com] using Configuration As Code.
Especially after January when DrupalCMS 2 [drupal.org] is rel
Re: (Score:2)
I keep hearing it's great for boilerplate stuff. Okay, fair enough. But how much boilerplate code do I have to write before it becomes worth my time? I really don't spend much time bootstrapping files, and 95% of the time I'd rather just use a previous thing that *I* wrote that I know works.
Here's What Happens To Me (Score:5, Informative)
Here is what keeps happening to me. I keep falling into the same trap.
I throw various simple things at the AI, sort of a Google replacement, and it gives me what I want. I'll have several successes. And then it will lead me down the wrong rabbit hole, misdirect me, refuse to break out of it's mistaken path, and waste way way way too much of my time chasing my tail.
Eventually, I'll arrive at my destination and I'll ask it why the fuck it took such a circuitous route full of errors and straight up lies to get to the simple and correct answer. It'll respond saying that it's complicated and that it doesn't have all the answers, sorry.
I'll then swear not to use it anymore.
Tomorrow, I'll start with it all over again like a crack addict.
Re: (Score:1)
Yep, this is when the context is full. Nuke the chat and start again.
My current favourite is "Oh, now I understand completely what's happening" (for the seventeenth time in a row - all of which were too-hasty.
Re: (Score:2)
Yeah, one of the things I like about Claude (and Gemini 3 as opposed to 2.5) is that they really clamped down on the use of "Oh, now I've got it! This is absolutely the FINAL fix to the problem, we've totally solved it now! Here, let me write out FIX_FINAL_SOLVED.md" with some half-arse solution. And yep, the answer to going in circles is usually either "nuke the chat" or "switch models".
Re: (Score:3)
A good bit of my effort with using LLMs has been in trying to avoid and correct it.
I've found it gets easier when you start to treat the LLM and its entire context window as a single unit rather than thinking about prompts.
Coding agents are variably successful at this.
For my own agentic tests, I've had good results "context engineering" the LLM to solve tasks reliably that it previously couldn't.
In the end- I'm not sure it's worth the effort, but hey, it keeps me ent
Re: (Score:2)
Re: (Score:2)
The key here is that it helps, but it can't replace you. Not that I care whether you get replaced, but there are a couple trillion bubble bux riding on whether you can be replaced, so it's a big deal.
Re: (Score:2)
Re: Here's What Happens To Me (Score:2)
I have mostly experience with a large app that I have been building lately. I use VSC with Claude. I have some background in coding but I do not do it for a living. As a tool, AI works best when there is a plan to follow and a master document for it to update to record progress. Starting from a proof of concept first and then expanding from there provides clarity. Sometimes I have used another AI to create a refined method to be implemented by another AI.
What I do not like about AI coding: the intellectual
Re: Here's What Happens To Me (Score:2)
Re: (Score:2)
What I do not like about AI coding: the intellectual and memory challenges fade away. There is no more brainwork that I have liked about coding. Copy-pasting and especially auto-coding become boring quite fast, and I have no deep knowledge of the code. I do not have problems with it to think about: solutions to feel accomplished for. Those only come when I catch an AI doing something stupid.
I have exactly the same problem copying code I have found on the web and now AI. Typing it in instead of copy pasting is a huge help, especially if I change variable and function names and reformat on the fly.
Quick! (Score:4, Funny)
AI companies should pivot to predicting Anthropogenic Global Warming, I'm sure it will be perfect for that.
Re: (Score:1)
All that matters is they have everyone's money, can influence elections and are too big to fail. Give them a break - AI is hard.
At first (Score:5, Interesting)
Professional dev in my third decade of experience speaking here. At first, these products really did assist quite a bit. In 2023 and 2024, I found the tools to be pretty decent at offering suggestions for small to medium snippets of code.
Something changed late last year. It may just be that the shine is wearing off, but I find most of the AI products producing less quality results than they did previously.
I rarely ever reach for them anymore. I sure would not rely on them over even an inexperienced junior dev, either.
Re:At first (Score:4)
Professional dev in my third decade of experience speaking here.
Only second decade, here.
I rarely ever reach for them anymore. I sure would not rely on them over even an inexperienced junior dev, either.
I find them comparable, unfortunately. But my new hires may not be as good as yours.
Re: (Score:2)
Something changed late last year. It may just be that the shine is wearing off, but I find most of the AI products producing less quality results than they did previously.
Empirically, speaking around to a few people yeah something now somehow feels not quite as good as it used to be. I think the yes-man problem has got worse. If you're trying to find the API/argument/etc to do X it will always tell you what a great idea it is and give you the code, even if there is no way to do it. I think it's got more syc
Re: (Score:3)
In my fourth decade, but been in C# since 2013, had a need to learn Angular and .NET 8 beginning of 2024. Professional engineer since 2012.
Where chatGPT is an enormous help, is in the following:
Re: (Score:3)
Fourth decade here. Up until a few months ago I would have agreed with virtually all of the negative comments in here, but after a re-org I am now on a team supporting multiple AI products, and have become immersed in anything AI, including vibe coding.
For vibe coding, I've had mixed results, but I want to make a couple of important points. First, the whole vibe-coding landscape is evolving very quickly. New IDEs and CLI tools are being announced almost daily. Second, the backend architecture of these tools
What happened to search engines (Score:2)
Is A.I. on the same path towards en-stuffification as search?
Google search was once really good, but then it became less capable over time, maybe because of more Web content becoming paywalled or perhaps The Algorithm was tinkered with to help those with money to spend, e.g., pay-for-play in page ranking?
Is A.I. going to end up the same way, eventually becoming useless because all it will do is serve ads?
It has its uses (Score:5, Interesting)
If you can describe exactly what you want, it can do a fine job accelerating that.
If you are stuck, asking it to try to solve your problem can at least be entertaining.
Today's AI may suffer from a critical flaw! (Score:1)
Of course it does. (Score:3)
Only if your a true believer! (Score:3)
Re: Only if your a true believer! (Score:2)
Nope! (Score:2)
Re: Nope! (Score:2)
Bloat Industrial Complex (Score:4)
AI seems to be feeding the bloat habit instead of trimming it. It's becoming an auto-bloater.
Very few in the industry are interested in parsimony. Devs would rather collect buzzwords for their resume rather than try to trim out layers and eye-candy toys. It's kind of like letting surgeons also be your general doctor, they'd recommend surgery more often than you really need it.
The principles of typical biz/admin CRUD haven't really changed much since client/server came on the scene in the early 90's. Yet the layers and verbosity seem to keep growing. An ever smaller portion of time is spent on domain issues and ever more on the tech layers and parts to support the domain. Something is wrong but nobody is motivated to do anything about it because bloat is job security.
YAGNI and KISS are still important, but is dismissed because it reduces one's resume buzzword count. The obsession with scaling for normal apps is an example of such insanity: there's only like a 1 in 50k chance your app or company will ever become FANG-sized, yet too many devs want to use a "webscale" stack. You're almost as likely to get struck by lightning while coding it. They patients are running the asylum.
Humans, you are doing CRUD wrong!
Re: Bloat Industrial Complex (Score:3)
Re: (Score:3)
Very few in the industry are interested in parsimony.
I've come to accept that this as true, and further conjecture that bloat is often a corporate/institutional goal.
This seems to be a joke [zerobugsan...faster.net], but in reality corporate incentives are aligned to make things more bloated. If you're a manager, then the more people you have under you, the more power you have. This means you want your people to go slower so you have to hire more of them.
I don't have a solution but there must be one.
It's not about being faster (Score:4, Insightful)
The goal here for AI is to eliminate wages. There is more than one way to skin a cat.
Remember good enough is always good enough especially when monopolies exist and we refuse to vote for politicians that will enforce antitrust law because we're busy freaking out about whatever Petty moral panic or culture War bullshit the boob tube tells us to today
Re: (Score:2)
Eliminate wages, assuming the result still works, would be classed as a productivity gain.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
"i need that small annoying snippet that does..." (Score:2)
In this, it is decent, and need to be something you actually understand as a result so you can proof read it.
But letting it loose on the big code at large is pretty dumb.
Re: (Score:3)
But letting it loose on the big code at large is pretty dumb.
I do this frequently, with both packaged agents and my modifications to them, just to see what comes out the other side.
Sometimes it's great, sometimes it's pretty bad.
I do it as a side job, not my regular work, so the consequences of the failure are minimal- I just throw it away and try again with another modification.
If it were my actual main workflow... I think that would stress me the fuck out- each failure being significant wasted time and money.
Re: (Score:2)
Re: "i need that small annoying snippet that does. (Score:2)
Re: (Score:2)
As I said, if it did- I'd have other feelings about it.
\o/ (Score:2)
No (but also yes).
If you let it write code freely, you can spend hours or days trying to figure out what the f**k it's done and why it's not done what you asked or why it randomly introduces a complete re-write for a single-line fix.
Heavily constrained yes.
It complements me perfectly. (Score:2)
Hidden cost (Score:2)
The Flow State (Score:5, Interesting)
Quite frankly, I don't want to talk to anyone all day, including an agent. And it's not really about being anti-social. As most people know, we have two primary brains - logical and emotional. I find that trying to explain what needs to be done the model engages my emotional/social brain too much, and the logical part can't focus on getting the job done. In order to do quality work, I need to completely ignore the emotional side of my brain and let the logical brain get into the state of flow, where I can accurate track the state of what I'm working on and follow logical workflows only. Talking to an agent all day engages my social side too much to do quality work.
Re: The Flow State (Score:2)
Re: (Score:2)
Thanks
Re: (Score:3)
I wonder what you think about pair programming. Sounds like that would not match your style either. And some agent/human interactions are thought to be similar to that, either the agent monitors if you mistype something, or you review each step of the agent before letting it continue.
Re: (Score:2)
My god, pair programming... I was interviewed once for a job once and they told me mid interview, with no prior pre-interview warning, they they work exclusively in pairs. I said "thank you" and terminated the interview within 2 minutes of that statement.
It just doesn't work. Besides, I am in indivudual, independent human being. If you work in pairs, you've got to synchronise your working hours, breaks and leave with someone else. Just one big "fuck no". On top of everything else that is wrong with talking
Re: (Score:2)
Is it the human factor (and the time coordination) or the pairing factor? Let's say AI gets a bit better and you have an AI driven system available. It works at your pace and is always ready when you're working. Would this work for you even though pairing with humans doesn't?
Yeah... (Score:2)
I'm sure this is all worth trillions of dollars or whatever
A simple, but lame, example. (Score:2)
Write a C++ program which calculates a "stardate" where each date is a 24 hour Earth day, and the origin (0) is January 1st, 2001 at midnight.
Finished
🌐
🌐
🌐
To calculate a "stardate" based on Earth days since January 1, 2001 at midnight, you can use the C++ <chrono> library to compute the number of days (and fractional days) that have passed since the epoch.
Here's a complete C++ program that calculates the stardate:
#include <iostream>
#include <chro
Re: A simple, but lame, example. (Score:2)
Re: (Score:2)
Would be interesting to see where the code snippets are originating from</quote>
Actually, I doubt that. If you look at the original prompt, I defined the problem. Because I used the word "stardate" it hallucinated into the Trek universe. The point is that plenty of "vibe coders" are making that mistake and the code get woefully conta
It helped research some 25-year-old code (Score:5, Insightful)
I came across some Emacs elisp code I'd written about 25 years ago, and it looked pretty useful. Emacs didn't like it. I researched the functions and variables and they apparently had been rejiggered about 5 years later. I said to myself, Self, sez I, this could be an interesting AI test. I could probably make this do what I want in a few minutes now if I did it from scratch, but that wouldn't help me understand why it was written that way 25 years ago.
So I asked Grok. I was pleasantly surprised to find it understood 25 year old elisp code just fine, explained when and how they had been rejiggered, and rewrite it for the current standards. That was more than I had expected and well worth the time invested.
One other time Grok surprised me was asking how much of FDR's New Deal legislation would have passed if it had required 2/3 passage instead of just 1/2. Not only did it name the legislation which would not have passed, it also named all the legislation which had passed by voice vote and there was no way to know if 2/3 had voted for it. The couple of bills I checked did match and were not hallucinations. The voice vote business was a nice surprise.
I program now for fun, not professionally. The idea of "offshoring" the fun to AI doesn't interest me. But trying to find 25-year-old documentation and when it changed doesn't sound like fun, and I'm glad to know I can offshore at least some of the dreary parts.
Re:It helped research some 25-year-old code (Score:4, Funny)
I program now for fun, not professionally.
UID checks out.
Gardening time (Score:4, Insightful)
I've worked for myself as an independent developer for more than a decade now.
Apps and websites and I do well working on my own.
I'm getting old though that the saying "can't teach an old dog new tricks", is starting to make sense.
AI couldn't have come at a better time in my life.
As I've always warned youth thinking of getting into tech at higher education, the older you are the less valuable you become. The complete opposite of the other white collar grad workers. You want the old experienced doctor, lawyer, accountant etc. Not the fresh faced grad; unless you are hiring software devs.
Since I started using AI I've found I'm a magnitude more productive in my output and my overall success.
It's such a time saver my home looks fab this summer (I'm in NZ) as I've had so much spare time to enjoy gardening.
AI has knowledge. What it doesn't have is wisdom.
As long as you remember that and have the wisdom and intuition to know when it is wrong you can't lose.
Which gives me hope as an old timer in this game.
Maybe after all I have what the other white collar grad workers have that is most valuable.
Experience and wisdom which is why AI is no threat to any of us right now.
Faster, no. Multi-tasking yes. (Score:2)
As a human, AI workflows let me have a life. I can let the agents knock out the easy things while I'm working on other tasks. I still need design out what's to be worked on, review the code, fix bone mistakes they make, etc. It's basically like having a junior developer assigned to you.
Which brings up an important point. Junior developers need clear instructions/requirements and so do AIs
Re: Faster, no. Multi-tasking yes. (Score:2)
Re: (Score:2)
Re: (Score:2)
As a human, AI workflows let me have a life. I can let the agents knock out the easy things while I'm working on other tasks. I still need design out what's to be worked on, review the code, fix bone mistakes they make, etc. It's basically like having a junior developer assigned to you.
Every time I see someone talking about AI being a junior developer, I am quite certain they have never worked with a junior developer.
One thing is faster - increase of technical debt (Score:2)
I really do think coding using AI tools is a bit faster, at least it seems that way to me. As most of the morning but lengthy work can be done faster by AI.
But I am also pretty sure it's VERY easy to rapidly incur technical debt, especially if you are telling AI to review its own work. Yeah it will do some stuff but who is to say post review fixes it's really better?
More than ever I think the right approach to coding with AI is to build up carefully crafted frameworks that are solid (maybe use AI to help
"Coding" is not software development (Score:3)
AI might make newbies faster at producing... something. Probably something full of bugs and security holes.
But it won't help non-newbies with software development, of which "coding" is a relatively minor part.
Re: (Score:2)
Still, people like to concentrate on coding -- probably because software matures a
Brittle tech (Score:4)
I've been playing with these genAI system both as code producer but as helper on various tasks.
And overall, I find the models quite brittle unless they are fine tuned on the precise task that you want.
The main problem that I see is that the tool is fundamentally a string in string out. But the strings could be absolutely anything including completely insane things without proper fine tuning.
Today, I am writing a simple automatic typo correction tool. The difficult bits are making sure that the tool didn't crap out. I mean, it is easy to check you actually got an answer from the tool. The problem is that sometimes the tools will tell you: "Sure, I can fix typos. Here is your text corrected: ". Ans so you have to toss that output out probably. But how do you figure out that it shat the bed? Well, you can't really, it is just as hard as the original task in some cases. So you bake various heuristics, or you get a different LLM to check the work of the first one.
At the end of the day, you really can't trust anything these tools do. They are way to erratic and unpredictible. And you should treat any of these tools as being possibly adversarial. It's exhausting to use really.
Make up your minds (Score:3)
These wild swings between AI thinning out the workforce and making all our jerbs obsolete to not being sure if AI is even useful is giving me a headache.
Re: (Score:3)
It'll do both... dumb management and bean-counter types will replace people with AI, and the AI will suck at actually getting work done.
Lose-lose!
Re: Make up your minds (Score:3)
Been using Claude.ai CLI (Score:2)
Been using the Claude CLI the last few weeks and it has definitely been a great assistant in working with Qt 6, C++, and QML. The CLI interface is one of the best interfaces I have evey seen, and it's native use of markdown is ideal. I am still writing 90% of the code but Claude's a great way to get information on some aspects of the library that I'm not as familiar with. I'm not ready to set it loose with nothing but a specifications document yet.
I've had it port some code from OpenGL to QRhi (similar t
AI fails the detail problem. (Score:2)
When using AI to quickly mock up small chunks of code I find it an accelerator. And I do mean small.
But when code scales up simple systems or API's it falls apart pretty quickly. As the scale of the system grows the requirements grow even faster. GDPR, PII, FIRB, NIST, all start to pile up on as code bases grow. AI lacks the understand of the "business" need. So you get this blob of code out and then you have to spend large amounts of time understanding it so you can re-factor it because the AI engine
My productivity is up 5x. At least. (Score:3)
I use AI regularly, at least once or twice a week. It's a real productivity boost. It's completely replaced searching for me. It's basically an API expert I can talk to and get answers from in 20 seconds. Good stuff.
Example: I'm working on a bad code base of a legacy application. The backend is quite a mess which I don't really like to touch, so I push a lot of my new logic into our Postgres DB. I don't really like SQL and anything beyond one or two joins I'd usually avoid. With progbuddy AI I'm doing triggers, procedures, functions, variables, etc. in SQL like a champ, sometimes 30 lines or more. Getting this good in SQL would take me at least a year of systematic practice.
The AI still does some mistakes or talks nonsense, but I catch those mistakes easily because that much I do know about SQL and coding in general. I'm the sole programmer in a company of 70 people and still manage to get off work at 5 o'clock whilst doing everything on my own.
So, yeah, AI definitely is a sold productivity boost for me and my work.
Re: (Score:2)
That's... horrifying.
You will eventually need to understand it yourself (Score:3)
This was until I reached a problem concerning one of the embedded application servers, that none of the AI's could crack. I tried everything - new context windows, new AI's, providing all conceivably useful context info (stack traces, even decompiled files etc.) and symptoms based on my own hunches and debugging skills - but nothing helped. I also let the AI suggest what further info could be needed. But nothing helped. This problem was completely out of reach for the AI's - even with my debugging help. And this was the worst - because I had let the AI do so much stuff and not researched it myself, I kept trying to get the AI to fix it and so wasted a lot of time because it ultimately couldn't. Also, I was no longer in a position to debug it myself. I basically didn't fully know what I was doing, I was missing steps on the knowledge ladder. Eventually I had to throw in the towel and basically retrace everything the AI had done, why it was done that way, how/why it worked, and properly research everything and THEN deep-debug it.
This is not a criticism of AI's as such, it is unreasonable to expect a quick fix to everything. But it is just to show how some of the initial time savings (that were indeed huge) can be eaten away when the quick fixes stop working.
AI assisted coding is kind of like AutoTune... (Score:2)
I use AI assisted coding in Agent mode daily. I see it as something similar to AutoTune for vocals - it'll make a good singer great, and a great singer world class. Anything less than a good singer, autotune is going to move to the wrong note and make it worse.
It's the same with coding - it'll make a good and great developers better, and mediocre developers worse. It's probably also important to acknowledge how I'm using it as it's likely that your experience will vary depending on environmental variables-
-
my experience (Score:2)
As a software engineer I can easily say that AI coding assistants have solved problems that would have taken me weeks to figure out.
Things like how to install something complex or how to write a relatively simple program in an unfamiliar language and framework.
It's also created bugs, inefficient code and masked problems in order to get to that all important "it's now running perfectly" state.
My worst fear is when the AI says something like "I should take a completely different approach" and then it begins t
Imagine being interrupted by a confident moron (Score:3)
Simple answer: if you're skilled at your job, it slows you down. Best case scenario, it's an improved StackOverflow. If you're clueless and don't care if your code works correctly...or even works at all, it can help. I use Claude daily, mandated by work. For things I know how to do, like Java, it doesn't save any time. I wish it would. It's a perfect use case. Write a class...have it write a unit test. Well, the unit tests pass about 3/4 of the time. However, they're pretty much all garbage. In fairness, my team and I are sticklers about writing good tests, (don't assert bean methods, for example, it's a waste of time and makes your class unreadable). However, I'll frequently come across old code written in a for loop and want to convert it to a lambda or optimize something where I know it can be written more efficiently. For refactors? it gets the basic Java syntax correct about 50% of the time. It gets VERY confused when methods are chained, like a lambda. Overall, it's definitely less than 50% success rate, just on compilation.
However, my big problem is that LLMs just guess. I can fix syntax errors easily. They guess method names, despite having full access to the project structure, so they hallucinate methods that don't exist or pass in the wrong arguments. Normally, the compiler catches this, but sometimes it doesn't. It can send the wrong number and it gets very confusing to debug. The problem with guessing machines is it guessed that value for a reason...and eyeballing it, it looks correct! It makes common sense, based on common naming schemes...it just is wrong...and if it's swapping 2 numeric or string args, it would compile and then you wonder why things aren't giving the expected results.
This has lost me a lot of time, especially when I started using Claude. I had Claude generate code that really looks right, but then I get a wrong answer in my tests and spend hours thinking I made a mistake elsewhere. I assume I made the mistake or my inputs were wrong or my logic was wrong...half a day later, nope...claude just swapped a similarly named method in the Jackson lib that does something similar, is named similarly, has the same args, but not the same. I don't have the ObjectMapper API memorized, so I saw it, thought "hey, I thought that was named something else...oh well, maybe this is their new API, this method name is more intuitive anyway", it makes sense, the code compiles, even a shallow glance at the docs looked good. It was only until I carefully read the docs that I realized it was the wrong method.
Now, for things I suck at?...OK, it helped. I forgot how to do a simple RegEx...asked Claude, it got it right the first try. It was too complex and kinda stupid...but it worked and I was able to write a unit test and jog my memory on the right way to do it and correct it with something that wouldn't get me reprimanded at work.
For Java, the code it writes almost always sucks....even ignoring syntax errors and hallucinations. This is something you need to remember. Even if Sam Altman and every CEO touting this shit wasn't a fraud, it's only as good as it's training data. Java has been around since 1995. JavaScript and Python have been around for similar amounts of time. It was trained on old code. If a programmer did a coding test and wrote a C style for loop (for(int i=0....) to loop through a List on an coding interview, I'd reject them. It shows they haven't bothered to crack open a book or read an article in 11 years. Claude does Java 5 (2004) syntax by default...because it was trained on old code instead of Java 8 syntax (2014).
If you vibe coded Java at my place of work, myself and a dozen other programmers would fill your pull requests with all sorts of nastygrams, even if it was working, because as a professional, we'd expect you to know best practices and modern syntax.
Hit or miss (Score:3)
But since they don't have shame, they can't feel bad about sending you down the wrong path, pressuring them the next time to say "I don't know" versus possibly giving misinformation. I wasted several hours, over two weeks, trying to repair a common laptop hardware issue--even though I told it the goal in the original query, at the end I made it admit that the goal was impossible.
Accelerationists (Score:2)
On the r/accelerate Reddit I see that the "accels" ("decels" are banned) are crowing about an METR study showing that the time to do an office task satisfactorily 50% of the time has been dropping, which they whoop about showing "exponential improvement" is still on track. So hitting a low bar on the easiest office tasks faster is for them they proof that AGI is around the corner and that skeptics have been disproved.
Re: Didn't see that one coming (Score:2)
Re: (Score:2)
But you don't. You didn't even read the paper. You're just an asshole.
Re: (Score:2)
Re: (Score:2)
So you're basically mad that MIT isn't the Thought Police for all their researchers, and they don't impose a dictatorial Single Opinion on them? And worse, you're mad they don't impose a Single Result!
I don't know if you've considered this, but perhaps the actual problem is that you're already an idiot, and trying to use AI like a brain prosthetic so you can pretend you're smarht?