Will Unpredictable 'Franken-Algorithms' Have Deadly Consequences and Make Programmers Obsolete? (theguardian.com) 96
Zorro (Slashdot reader #15,797) summarizes a new article in the Guardian:
The death of a woman hit by a self-driving car highlights an unfolding technological crisis, as code piled on code creates "a universe no one fully understands."
"In some ways we've lost agency. When programs pass into code and code passes into algorithms and then algorithms start to create new algorithms, it gets farther and farther from human agency. Software is released into a code universe which no one can fully understand."
The author dubs these man-made monsters "franken-algos," since "After a time in the wild, we no longer know what they are: they have the potential to become erratic." Self-learning algorithms are already part of the "new all-machine phase" of Wall Street trading, leading to what science historian George Dyson believes are rules "where nobody knows what the rules are: the algorithms create their own rules -- you let them evolve the same way nature evolves organisms."
Where does it end? There's already a robotic sharpshooter policing the demilitarized zone between North and South Korea, and "swarms of coordinated, weaponized drones" already being developed by three different countries. The article suggests re-thinking our legal system to assign blame for any badly malfunctioning algorithms, noting that the Association for Computing Machinery recently updated its code of ethics "along the lines of medicine's Hippocratic oath, to instruct computing professionals to do no harm and consider the wider impacts of their work.... Solutions exist or can be found for most of the problems described here, but not without incentivizing big tech to place the health of society on a par with their bottom lines.
"More serious in the long term is growing conjecture that current programming methods are no longer fit for purpose given the size, complexity and interdependency of the algorithmic systems we increasingly rely on." Toby Walsh, a professor of artificial intelligence at the University of New South Wales, even says "We will eventually give up writing algorithms altogether... "because the machines will be able to do it far better than we ever could. Software engineering is in that sense perhaps a dying profession."
"In some ways we've lost agency. When programs pass into code and code passes into algorithms and then algorithms start to create new algorithms, it gets farther and farther from human agency. Software is released into a code universe which no one can fully understand."
The author dubs these man-made monsters "franken-algos," since "After a time in the wild, we no longer know what they are: they have the potential to become erratic." Self-learning algorithms are already part of the "new all-machine phase" of Wall Street trading, leading to what science historian George Dyson believes are rules "where nobody knows what the rules are: the algorithms create their own rules -- you let them evolve the same way nature evolves organisms."
Where does it end? There's already a robotic sharpshooter policing the demilitarized zone between North and South Korea, and "swarms of coordinated, weaponized drones" already being developed by three different countries. The article suggests re-thinking our legal system to assign blame for any badly malfunctioning algorithms, noting that the Association for Computing Machinery recently updated its code of ethics "along the lines of medicine's Hippocratic oath, to instruct computing professionals to do no harm and consider the wider impacts of their work.... Solutions exist or can be found for most of the problems described here, but not without incentivizing big tech to place the health of society on a par with their bottom lines.
"More serious in the long term is growing conjecture that current programming methods are no longer fit for purpose given the size, complexity and interdependency of the algorithmic systems we increasingly rely on." Toby Walsh, a professor of artificial intelligence at the University of New South Wales, even says "We will eventually give up writing algorithms altogether... "because the machines will be able to do it far better than we ever could. Software engineering is in that sense perhaps a dying profession."
No (Score:2, Insightful)
Someone's got to fix it when it goes wrong.
Typical click bait.
Re: (Score:3)
The only specific example given in TFA is the pedestrian killed by Uber, which was NOT an algorithm failure, it was a policy failure. The policy decision to toss control back to an inattentive human when a collision was imminent was made by humans, not a machine.
Programmers will not be obsolete (Score:4, Insightful)
Re:Programmers will not be obsolete (Score:5, Insightful)
What do you call the person who gives the franken-algorithm a specific list of instructions to accomplish your goal? A programmer.
Yep. The never ending dream of management, but impossible.
I remember the last time I saw it in action. Yeah, Marketing could do more stuff themselves, after acquiring new toys. But they turned into (rather bad) "programmers", using non-standard clunky tools with none of the helpful components of actual programming environments. And spent their time thereafter playing with the toys, instead of marketing. What a win!
Re: (Score:2)
Re: (Score:2)
At some point AI might become self-aware enough to create even better AI.
Not with the current excuse for AI there won't be. It doesn't 'think', has zero capacity to do that (because we don't even understand how *we* do that), is not self-aware (again, because we don't even understand how that happens in human brains) therefore you can't write code or build a machine that does that. You're engaging in 'magical thinking' if you're thinking just adding more and more hardware will make it just suddenly happen all on it's own.
Re: (Score:1)
Nope. A programmer understand exactly what their code is doing and why at every step of the process. If you don't then you aren't a programmer, at best you're a script kiddie.
Re: (Score:3)
Re: (Score:2)
Understanding what code does and predicting what it will do are not quite the same thing. Understanding how a neural network works does not necessarily mean you can predict how it will behave for all inputs.
Re: (Score:2)
Sounds like Vernor Vinge's Programmer-at-arms or possibly Programmer-archaeologist position to me.
Sam
Re: (Score:3, Informative)
Might want to look up the facts of this case. Uber had disengaged the "stop if something's in front of you" aspects of the self-driving car's code, and the woman who was responsible for taking over in such a situation was watching a reality show on her phone at the time of the incident. Nobody even tried to stop this car.
Re: (Score:2)
Question more what you read
Re: (Score:1)
I was about to post this exact sentiment, but with more swearing.
Re: (Score:1)
Unavoidable (Score:2)
Re: (Score:3)
This seems right that complexity like this is likely unavoidable. Evolution has a fairly reliable way to deal with this. Complexity is allowed to grow unchecked but every generation of a species is required to be able to survive and reproduce or it goes extinct. That seems to be pretty much what is happening with software.
There seems to be a dream that there is a simple theory of everything that humans can understand and that will allow us to do what we want in our complex world and still understand i
Re: (Score:2)
All we can do is move from point A to point B across the surface of the planet
Airplanes are largely free of obstacles and we can't operate them without air traffic controllers and a mountain of regulations and safety checks; what makes you think we can drive cars through thousands of constantly moving obstacles a day?
Re: (Score:1)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
The Machine (Score:1)
Are we talking about a program, ... ...or the legal system?
I swear -- I read this, and I can't help but think about the legal matrix we live in.
For sure!
That's exactly wha
Re: (Score:2)
Poor planning, poor QA. (Score:3, Insightful)
There's a real threat these days of big companies and naive developers pushing machine learning as a panacea. Really it's more like a potent mutagen escaped from lab containment into your other work. People trying to obsolete expertise with poorly thought out and even more poorly tested applications of machine learning are definitely going to continue to cause deadly consequences in and around self-driving vehicles.
Someone has to program it (Score:3, Funny)
Someone has to program the initial version.
who better than Java programmers?
forget all those slow, bug-laden, inefficient, crappy languages like C/C++ or ASSembly. Java is where it's at.
Java is fast, and each iteration of a function or program makes it faster!
Forget stupid stuff like IEEE-754 compliance, ditch the hard stuff and focus on performance!
Java is where it's at.
Definitely make programmers obsolete (Score:2)
Re: (Score:1)
You'll always need programmers unless we can teach computers to write code that humans can interact with. For programmers to completely go away you'd have to create a world in which applications don't exist and computers can't be seen. To make a computer that knows the limits of human understanding and can work within those limits would require human testers which would severely reduce testing speed.
What might happen is we'll be left with only UI programmers and all the backend algorithms and architecture w
Re: (Score:2)
No way will that work out (Score:2)
Pretty sure monkeys all prefer tabs. Are you really going to attach that hot mess to crucial systems?
Yes and no (Score:5, Informative)
Will Unpredictable 'Franken-Algorithms' Have Deadly Consequences
Probably yes.
and Make Programmers Obsolete?
Almost certainly not.
We are creating algorithms where the result can not be explained in human terms. Nobody can truly understand why AlphaGo thinks a move is good, it's a neural network of weights we don't understand. It's about as useless as trying to get a chess grandmaster to articulate why a particular move is good, it's subtleties you can't record and put in a rule book. Which is fine for AlphaGo since the worst it'll do is lose a game. If it's Watson totally misdiagnosing your cancer or Waymo's car T-boning a school bus it matters a lot.
That is why I think developers will always be busy implementing guard rails. Like if you're trying to minimize humanity's environmental impact then the divide by zero solution is obviously superior. It's not a practically feasible solution in the real world though.
The actual question should be (Score:1)
"Do you try to implement the code contained in the first link google gives you for your search?"
I'm guessing about 80% of the code tossed into franken-vehicles is directly downloaded by shoddy/H1-B/lazy programmers, who haven't ever written a line of code or documentation for a production envuronment (no marter what their CV says.)
Yes (Score:2)
The answer is, "Yes, there will be deadly consequences and programmers will be obsolete." There's nothing you can do about it. Now make yourself a nice cold drink with some rum and fruit juice or something and go grill a piece of meat.
You can't change things, so you might as well enjoy your holiday weekend. Maybe things will look different on Tuesday, but for now, don't sweat it and go outside. It's nice outs
Chant Against Malfunction (Score:2)
Toll the Great Bell Once!
Pull the Lever forward to engage the Piston and Pump...
Toll the Great Bell Twice!
With push of Button fire the Engine And spark Turbine into life...
Toll the Great Bell Thrice!
Sing Praise to the God of All Machines
Re: (Score:2)
He was replaced by Tina Smith, we have 2 women senators from MN now
Whatever. (Score:2)
It's the best of all possible worlds. - Leibniz/Pangloss.
Not obsolete... (Score:2)
The referenced strategies only come into play to accomplish tasks that were pretty much out of reach of traditional programming. Essentially a last resort. For the problem set that has been feasible for programmers to tackle, it almost always remains the better way.
Further, it's really about bringing taming complex, chaotic, unstructured data into a structure so that programmers can address it. Generating these routines never speaks to how to apply the approach to solve a problem. Human's are required.
The real danger is BELIEVING ALL THE HYPE (Score:4, Interesting)
Suddenly, the gifted programmer— (Score:1)
—employs a rarely seen strategy of “code reuse”.
“Don’t fly on payday.” — Wally
Predictions (Score:1)
Which would free us up to vacation in our flying car and date Rosy the Robot.
Re: (Score:1)
But she's rusty in bed.
Eventually? (Score:3)
We will eventually give up writing algorithms altogether...
Automated code generation. Automated test generation. Automated test coverage. Been there, done that. Twenty years ago. The people who say 'eventually' are the ones with a vested interest in selling meat sack coding labor to customers.
You think it's unpredictable and not to be trusted? Better not fly in a modern airplane.
Human error vs machine error (Score:2)
We should not "incentivizing" big tech to pass the risks on to the public. QA is the boring stuff that involves regulation, redundancy and statistical modeling, of course the fancy Internet companies want to pass the responsibility for QA to end users with eternal Beta v
Only if... (Score:2)
Only if by Franken algorithm they mean some doomsday device with autonomous killing robots that drive humans into extinction. Programmers may some day become obsolete, but the death of programmers will be nothing so vague as the article.
Comment removed (Score:3)
Re: (Score:2)
not with current management (Score:2)
Can you imagine even the smartest speculative fiction AI dealing the typical "I don't know exactly what I want, but it's something like this: and I need the demo ready in a couple of days for a sales presentation."
If anything sets off the AI revolt, it would be dealing with today's mangers and marketing morons.
Humans are stupid (Score:2)
What's with all the Betteridge's law titles lately (Score:2)
The answer is and was always 'NO'!
Franken-Journalism (Score:1)
When a journalist writes about stuff he doesn't understand.
Unpredictable? (Score:2)
No, not unpredictable, unless you don't understand how AI and neural networks work!
Are there edge cases where input generates unexpected results? Absolutely! And that's no different from any kind of programming since the computer was invented.
For that matter, when we used to ride horses everywhere, there were edge cases that caused horse brains to freak out, such as gun shots nearby, or the sight of a wild animal. That didn't stop people from making safe use of horses for transportation, with accidents occu
Law Enforcement and the Court System (Score:2)
Franken-Algorithms? Just wait until they are used by law enforcement and the court system which is already starting.