Morgan Stanley Says Its AI Tool Processed 9 Million Lines of Legacy Code This Year And Saved 280,000 Developer Hours (msn.com) 87
Morgan Stanley has deployed an in-house AI tool called DevGen.AI that has reviewed nine million lines of legacy code this year, saving the investment bank's developers an estimated 280,000 hours by translating outdated programming languages into plain English specifications that can be rewritten in modern code.
The tool, built on OpenAI's GPT models and launched in January, addresses what Mike Pizzi, the company's global head of technology and operations, calls one of enterprise software's biggest pain points -- modernizing decades-old code that weakens security and slows new technology adoption. While commercial AI coding tools excel at writing new code, they lack expertise in older or company-specific programming languages like Cobol, prompting Morgan Stanley to train its own system on its proprietary codebase.
The tool's primary strength, the bank said, lies in creating English specifications that map what legacy code does, enabling any of the company's 15,000 developers worldwide to rewrite it in modern programming languages rather than relying on a dwindling pool of specialists familiar with antiquated coding systems.
The tool, built on OpenAI's GPT models and launched in January, addresses what Mike Pizzi, the company's global head of technology and operations, calls one of enterprise software's biggest pain points -- modernizing decades-old code that weakens security and slows new technology adoption. While commercial AI coding tools excel at writing new code, they lack expertise in older or company-specific programming languages like Cobol, prompting Morgan Stanley to train its own system on its proprietary codebase.
The tool's primary strength, the bank said, lies in creating English specifications that map what legacy code does, enabling any of the company's 15,000 developers worldwide to rewrite it in modern programming languages rather than relying on a dwindling pool of specialists familiar with antiquated coding systems.
When this low quality AI bubble bursts. (Score:4, Interesting)
Salt the Internet (Score:3)
Do you think someone could post lots of garbage code on the internet, the AI bot would Hoover it up to train its model, and then the AI would emit garbage?
Re: (Score:2)
Re: (Score:1)
Re: (Score:2)
Evaluation is a lot less computationally complex than generation; I would expect anything that gets scraped by a larger AI company goes through at least a two pass system 1) is this even relevant/compiling code? 2) does this meet a minimum standard? and probably for largest companies building a library of quality content 3) is this high enough quality to include in the library for future training? Steps 1/2 can probably be done with something like a 30b or 70b model fairly quickly and step 3 is probably eva
"plain English specifications" (Score:5, Funny)
"plain English specifications that can be rewritten in modern code."
So just the easy bit left to :)
Re:"plain English specifications" (Score:5, Insightful)
Re: (Score:2)
Re:"plain English specifications" (Score:5, Insightful)
An LLM is the only tool that can really do this right now.
If you had a bunch of poorly commented code and didn't know what it did, then having "AI" read your code and just describe it would help immensely with finding out where you need to go to work on a problem. You'd still be the one understanding the code while doing the work, writing the new code, etc.
Re: "plain English specifications" (Score:2)
I just see that they performed a low hanging fruit activity which still could preserve architectural errors and implementation bugs that have been benign so far.
In many cases there aren't any performance optimizations either. Take going from Cobol and single indexed files to joined SQL tables - something that could improve performance or kill it.
A lot of people have no idea about how to optimize and just throws more CPU or other tech at the problem.
Re:"plain English specifications" (Score:5, Insightful)
Remember they don't have to replace all of us, if they replace 20% of us it will create massive amounts of unemployment and cause wages to plummet because of labor oversupply.
We need to pull our heads out of the sand right now. We need to get over the shit we learned when we were 12. This is happening and it's going to affect us badly.
And there is no economic solution to this. We can't all just go start new businesses because nobody's going to give us any capital and even if they did bigger companies would just run us out of business.
And we can't all be plumbers and welders and HVAC. The world just doesn't need that many of them and beyond that we need white collar guys hiring those blue car guys to fix stuff.
You can't take 20% maybe even 30% of White collar employees out of the equation and still have a functioning economy in 2025. Keep in mind we spent the last 50 years automating every factory we could so there are a hell of a lot less Blue collar jobs and then they're used to be.
You're generalising from the specific (Score:2)
Yes, a lot of programming jobs are probably on the line. But that doesn't mean that 20% of ALL white collar jobs are. Think doctors, accountants, lawyers, teachers, engineers (as in designers) etc, etc.
A friend in accountancy reckons that the quality of accountants has fallen in recent years - probably because the possible candidates have become IT staff. If AI does what it promises, a lot of intelligent people will be released to start serving their communities in new ways. Ok, that the optimistic spin, an
Re: (Score:2)
Not to sound really stupid here, but what do accountants do that a whole set of traditional programs haven't already automated away that LLMs that can replace coders can't?
I have limited understanding of being an accountant, but it seems like there's the already replaced manual version of double entry accounting, and then there's reading and applying rules to digital data. Then there's forensic accounting, looking for patterns in data and the LLMs claim to be real good at that too.
Fair question (Score:2)
Most of what can be automated in accountancy has been, though you might be right about some parts of forensic. However the auditing role, creating management accounts and projections, finding optimal tax strategies, and, at its most basic, checking expenses claims, are not easily automated. Auditing is especially hard to automate; it's about checking physical stock are correct, claims for income, invoices etc are conforming to the rules, and just asking management hard questions about obscure corners.
Re: (Score:3)
You can't take 20% maybe even 30% of White collar employees out of the equation and still have a functioning economy in 2025. Keep in mind we spent the last 50 years automating every factory we could so there are a hell of a lot less Blue collar jobs and then they're used to be.
I think that for the parasite class that's a feature, not a bug.
Everything the current administration is doing seems directed at both reducing the population and more tightly controlling average citizens:
- Cutbacks to medicare and medicaid
- Cutbacks to Social Security, while simultaneously making it much harder to actually collect
- Cutbacks at NOAA such that after-hours severe-weather warnings don't go out
- Putting Kennedy in charge of health
All of the above, and about a zillion other government action
Re: (Score:2)
"plain English specifications that can be rewritten in modern code."
So just the easy bit left to :)
That's an important bit. When I have to work on a file of old spaghetti code, I open Github Copilot and type "Explain this code", and it gives me a well written summary of everything going on in the file. That is very useful. Anybody not using AI coding tools is really missing out. I feel like Copilot has doubled my productivity. I can paste a screenshot from a requirements document and enter, "Make an Angular component that looks like this", and it does a remarkable job. It's not perfect, but what it
Re: "plain English specifications" (Score:2)
Re: (Score:2)
Same reason I don't wash dishes by hand. I have a machine that does it for me.
And reading code? Ugh. Everybody's code sucks except for mine. ;)
Seriously, though, a couple paragraphs of summary before diving into somebody else's work really does help. It's not like there's ever documentation or comments.
Re: "plain English specifications" (Score:2)
round down all Fractions of an penny and place the (Score:2)
round down all Fractions of an penny and place them in my account.
Re: (Score:2)
Re: (Score:2)
Can somebody please up the the tree swing comic with Al translating each individual frame to a plain English specification. Because it looks like that the code you have is also what it should have been doing.
Re: (Score:3)
Wasn't COBOL written to make it such that a manager could read the code as plain English and understand what it does?
In any case, the issue isn't understanding what the old COBOL code did. It is that when the senior executive's job is on the line, can you answer the question: Does the new code work the same way as the old code? How do you know?
Your putting alot of faith in an LLM and its ability to write a plain English explanation. Not to mention that (a) the code fragment being analyzed may not actua
Re: (Score:2)
Re: (Score:2)
This is step one : analyze the existing architecture. If you can automate this you are MILES ahead. It is a very fatiguing process. A real grind.
Step two is to interview users and determine their pain points (manual work-arounds for system limitations, or seemingly redundant or repetitive steps that can be merged).
Step three is to design a replacement system.
Then you can begin building and testing the new system.
It's somehow more official from an LLM (Score:3)
"No fucking clue what this code block does, but without it the program runs backwards."
Sure. (Score:5, Insightful)
Sure. Sounds plausible, and there's nothing missing from the story.
Like for example why they transcribed it into an ambiguous human language needing further reinterpretation as opposed to translating it directly into code in the target language.
Re: (Score:3)
I presume the thought is to force human review of business critical logic.
In terms of the relative value, it might be helpful, but probably would have just as well been subjecting snippets of original code as needed to the LLM to help clarify.
I wager that the human doing the final implementation will have both the original source and the 'plain english' specification. I wouldn't be too shocked if they tend to look at the original source more than the specification.
So I'm not thinking this is necessarily a t
Re: (Score:2)
That process sounds like it would be extremely useful if translating, say, COBOL into Java. (I hate reading COBOL, even without bothering to understand it.)
Re: (Score:3)
Perhaps useful, but perhaps a bit wasteful compared to saying "developer can submit code they are trying to port to the LLM on-demand".
The step of proactively submitting 9 million lines of code and getting a bunch of text that has mostly has avery high chance of never being referenced smells of management doing something to look impressive at significant cost to highlight how "in tune" they are with the AI hype in the face of a broader reluctant organization that wants to keep them from doing anything that
Re: (Score:2)
The step of proactively submitting 9 million lines of code and getting a bunch of text that has mostly has avery high chance of never being referenced smells of management doing something to look impressive at significant cost to highlight how "in tune" they are with the AI hype in the face of a broader reluctant organization that wants to keep them from doing anything that might carry a whiff of actual risk.
They probably also spent a pile of money on their "AI" tools and need to justify the expense. Hell, some of the management probably have a stake in the "AI" company that supplied the tools. An endorsement from a big company like Morgan Stanley would increase the "value" of those tools, netting even more money for management.
Re: (Score:3)
x = y; ===> "Set X equal to Y"
There, saved an hour of developer time.
The beauty here is that the output does nothing and must be reviewed completely by programmers for it to be useful in any capacity. It may have taken developers enormous effort to do that work, but so what? It's not clear that anything of value was produced and that 9 million lines must still be available to developers to complete the translations.
"...as opposed to translating it directly into code in the target language."
Yes, and th
Re: (Score:2)
Re: (Score:2)
Legacy systems, especially if they are more than like 20 years old, typically don't have a spec or don't match their spec because of all the tweaks and modifications that have gone into them over the years. If you have the LLM make a spec that you can browbeat the business with it. Otherwise they will just say "make it do what the old one did, only make it secure and standards compliant", which is not helpful at all, and LLMs are not yet even close to the level where they could do a task as large and vague
Have you ever had to rewrite a code base (Score:3)
Most of the time was spent figuring out what individual functions and libraries did. The actual number of lines of code I had to change for the update was probably less than 100.
If I had something that did all that grunt work for me up front I could then go in and pick out the hundred or so lines that needed changing and the whole thing would have taken a couple hours instead of a couple weeks.
That is
Re: (Score:2)
The problem is that you still need to review and check the thousands and thousands of LOC that the bot has generated. Are you going to push the whole bot vomit in production without a second look? And to take that second look, it takes a lot of time anyway.
Re: (Score:2)
The problem is that you still need to review and check the thousands and thousands of LOC that the bot has generated.
False. In this scenario the bot has written 0 lines of code.
Re: (Score:2)
The bot is still doing a lot of work it's just doing analysis work instead of writing code. But that analysis is a huge part of programming.
Re: (Score:2)
"So the bot rewrites your code like this?
# echo "social panic"
# echo "gay panic"
echo "trans panic""
So which one was this AC? I think it's YOU who are stalking HIM. YOU who appear to be deranged and incoherent. YOU who are spewing hate and bullshit. Why don't YOU just stop it? Seek help?
Re: (Score:2)
Why don't YOU just stop it? Seek help?
The anti-rsilvergun troll is the GNAA and the Penis Bird of our generation at once. I only hope that he actually turns out to be all three of them so we can have a lol over it instead of the truth being as sad as it appears.
Re: (Score:2)
Re: Have you ever had to rewrite a code base (Score:2)
Yeah, I obviously got that by the way I was able to explain the same point to the guy who replied to you, thanks
The point isn't to get the bot to write the code (Score:2)
That means instead of going through hundreds and hundreds of lines of code and understand to get all so I can make that one little change I need I just get pointed where I need to by the AI.
Then I can do some testing and if it doesn't work I can double check the AI work but I can do that in a test mode.
You need to stop thinking in black and white terms. That's the
Re: (Score:2)
I'm guessing that what they claim they're doing is vastly exaggerated. AI can't possibly magically provide an accurate 'decompile' of code back into specs of what this code is supposed to do. If nothing else, the code may have bugs and oversights that mean that original specs weren't correctly implemented in the code - in which case there may be absolutely no way to figure out what the code was supposed to do as opposed to what it actually does. More likely, the spec is 'make X do Y', and the programmer wen
Re: (Score:2)
Even more likely, some bright corporate spark decided that Cobol programmers that can maintain the old code are too rare and too expensive. So instead they can use the magic of AI to give them a complete translation of what the old code does, then hire a bunch of much cheaper Javascript programmers, give them the AI-generated specs and boom, everything works just like before, but is now much cheaper to maintain. Profit! Some way down the line they'll realise that nothing actually works, but they probably wo
Entirely feasible (Score:3)
AI is great at analyzing existing code and explaining it, and should also be able to generate unit tests for that code. Then someone can review the explanations and tweak them if necessary. Feed them back in as prompts for new code generation, have the AI generate unit tests for it, and compare the results. A significant time saver.
Re: Entirely feasible (Score:3)
It also gets things very wrong sometimes. My favorite part of LLMs is asking it about code it wrote itself - sometimes it cannot justify why they did it or what it does.
Re: (Score:1)
>> sometimes it cannot justify why they did
The AI I'm using (GPT-4.1) gives a very detailed explanation of what it did and why. Not unusual for the explanation to be longer than the code it generated.
Re: (Score:2)
Re: (Score:3)
Another description would be "incremental development", not unlike what you would be doing on your own.
AI is the new Consultant (Score:1)
The thing you use to justify, rationalize, and give credence to an agenda you already wanted to push through.
The expensive thing you pay to do the same thing you already have staff for.
The thing that implements the hot new business school buzzword to show youâ(TM)re a bold leader, not afraid of the cutting edge.
The thing you can take credit for the successes of without worrying about shame.
The thing you can assign blame to if it doesnâ(TM)t work out.
"Street" Pricing. (Score:3)
..has reviewed nine million lines of legacy code this year, saving the investment bank's developers an estimated 280,000 hours..
This reads like those drug bust headlines where they confiscate a fanny pack full of coke, and then claim they saved 517 million lives with it. With a street value of eleventy-seven billion.
Re: (Score:1)
Bullshit (Score:3, Insightful)
Re: (Score:2)
Thank you for that reminder (Score:2)
The question is when will they come and take us back home...
Re: (Score:3)
The thing is that it actually isn't important whether what they got out of the LLM was accurate, it was enough text for some manager to take credit for generating a huge specification that purports to describe the business logic of all their code. That's some fine job meeting some big KPIs there.
It could be that the specification is wildly inaccurate and utterly useless, but on the other hand at least it's in good company with almost every single "English Specification" I have ever seen made by a human.
I suspect ... (Score:2)
You are the real Sherlock Holmes, and I claim by $5!
Perceptions of history (Score:5, Informative)
"It can technically rewrite code from an old language like Perl in a new one like Python".
Both languages are from the same vintage. Python is from the early 90s and Perl late 80s. Reminiscent of persistent belief JSON is new yet XML is old.
Re: (Score:2)
"It can technically rewrite code from an old language like Perl in a new one like Python".
Both languages are from the same vintage. Python is from the early 90s and Perl late 80s. Reminiscent of persistent belief JSON is new yet XML is old.
True, but Perl isn't used for many new projects these days. Python developers are much easier to find than Perl developers, and probably cheaper, which is what this exercise is really about.
Re: (Score:2)
Ten years are like a hundred in computing. More importantly though, the vintage isn't actually relevant. Python is literally based around a concept from the seventies - significant indentation, from SASL. (Punch cards had indentation too, but it was to specific columns, and not for control flow.)
Re: (Score:1)
> Ten years are like a hundred in computing.
Only because most "innovation" is just recycled and repackaged ideas from the past. The fad cycle is getting faster, like seeing bell-bottom jeans every 4 years instead of every 15.
Re: (Score:1)
Python is probably the wrong language. A compiled language like Java or C# seems a better fit for a staid company.
Good to know which bank I won't use (Score:1)
my money in the hands of a bot created slop? no thanks.
Yes, but... (Score:2)
How much value that AI added to the processed code?
Death of COBOL? (Score:3)
One reason COBOL exists at all today is the massive amount of legacy code, mostly in the financial arena. After all, the BOL is Business Oriented Language.
Although I'm sad to see a language go, I don't feel too bad about this one. I learned COBOL using card punch and batch processing. The COBOL decks were bigger than equivalent programs in Fortran.
I think that this is a pretty good use of AI. Within the limited world of COBOL coding it can be good at figuring out the routine and translating it. I'd love to see the output from this AI. Sadly the article doesn't link to any of it.
Back in the 90s, my then girlfriend was working in MIS at the headquarters of a large insurance company. They were trying to migrate legacy data that was packed in single lines of variable data length and with possible middle initials and other variances. They were used to set formatting. Apparently they had never done parsing of strings. I wrote out how to go character by character and make decisions based on what they had. She took that to them and they were able to translate the data. (They were MIS people not CS people)
Mythical man (Score:1)
Ahh, the "Mythical Man Three Decades"
I am deeply, deeply skeptical. (Score:2)
... and it has nothing to do with AI per se. My experience with legacy code that there is stuff there that is absolutely impossible to figure out by just looking at the code alone. These parts usually encode weird edge cases of external systems, workarounds for even earlier data, "temporary" hotfixes, etc. Even well designed systems have a few corners like these. And even if there are comments, comments tend to assume a certain shared context. I find it unlikely that an AI or even a Homo Sapiens can properl
So should we be shorting Morgan Stanley stock (Score:2)
On the belief that the AI specifications will be wrong in some crucial detail that causes the whole bank to fail?
COBOL to English (Score:2)
COBOL was designed to allow non-programmers to write code that maps closely to the English language.
So now we let AI take that code and translate it to English.
Hm, that's probably something that could have been done without AI at all. But without having actually seen the generated specifications and the code they were generated from, that's difficult to say.
Nonsense (Score:2)
"translating outdated programming languages into plain English specifications"
This is insane and impossible and only a non-technical manager could believe it. English, or any other natural language, is nowhere near precise enough to capture every detail of complex software. At best, it can represent an approximate summary.
I can imagine an AI that analyzes old code and builds a model of its function that could be used to find bugs and translate into another programming language, but translating into English
Deriving specifications from existing code (Score:2)
is always a bad idea. That old code does all kinds of things, both good and bad, both current and obsolete. If you don't have experts in your company that know what the code _should_ do, you are screwed. "Whatever the old code does" is never a good target.
Re: Deriving specifications from existing code (Score:2)
Re: (Score:2)
Indeed, lots of consultants. My bet is that it won't result in a net savings, rather, an apparent savings on the requirements gathering side, and a much inflated development cost built on the inflated AI-generated requirements, where 80% of the requirements are meaningless or obsolete or irrelevant.
Re: (Score:2)
Assuming that work was valuable work (Score:2)
Sure, this AI did some work, and spit out some specs. Would humans have done this work at all? Are the specs that were generated, worth the bytes they occupy?
This is what I suspected would happen (Score:1)
THIS NEVER HAPPENED (Score:3, Funny)
15,000 devs working on 9M lines of code for one company that does nothing different than the other bazillion investment companies.
But CHATGPT SAVED THEM 280,000 hours. My division skillz suggests that is two days' work for those "15,000" supposed devs.
Wow, it saved two days.
Quick write up a PR puff piece. /smdh
Headline: "ChatGPT saved two workdays."
or
Headline "Devs told to take Saturday AND Sunday off. ChatGPT got this one, bro."
Enlightenment (Score:1)
Good luck on that one!