Is There Tension Between Developers and Security Professionals? (zdnet.com) 146
"Everyone knows security needs to be baked into the development lifecycle, but that doesn't mean it is," writes ZDNet, reporting on a new survey they say showed that "long-standing friction between security and development teams remain."
The results came from GitLab's "2019 Global Developer Report: DevSecOps" survey of over 4,000 software professionals. Nearly half of security pros surveyed, 49%, said they struggle to get developers to make remediation of vulnerabilities a priority. Worse still, 68% of security professionals feel fewer than half of developers can spot security vulnerabilities later in the life cycle. Roughly half of security professionals said they most often found bugs after code is merged in a test environment.
At the same time, nearly 70% of developers said that while they are expected to write secure code, they get little guidance or help. One disgruntled programmer said, "It's a mess, no standardization, most of my work has never had a security scan." Another problem is it seems many companies don't take security seriously enough. Nearly 44% of those surveyed reported that they're not judged on their security vulnerabilities.
ZDNet also cites Linus Torvalds' remarks on the Linux kernel mailing list in 2017, complaining about how security people celebrate when code is hardened against an invalid access. "[F]rom a developer standpoint, things really are not done. Not even close. From a developer standpoint, the bad access was just a symptom, and it needs to be reported, and debugged, and fixed, so that the bug actually gets corrected. So from a developer standpoint, the end point of hardening is just the starting point, and when you think you're done, we're really only getting started."
Torvalds then pointed out that the user community also has a third set of entirely different expectations, adding that "the number one rule of kernel development is that 'we don't break users'. Because without users, your program is pointless, and all the development work you've done over decades is pointless... and security is pointless too, in the end." Juggling the interest of users and developers, Torvalds suggests security people should adopt "do no harm" as their mantra, and "when adding hardening features, the first step should *ALWAYS* be 'just report it'. Not killing things, not even stopping the access. Report it. Nothing else."
The results came from GitLab's "2019 Global Developer Report: DevSecOps" survey of over 4,000 software professionals. Nearly half of security pros surveyed, 49%, said they struggle to get developers to make remediation of vulnerabilities a priority. Worse still, 68% of security professionals feel fewer than half of developers can spot security vulnerabilities later in the life cycle. Roughly half of security professionals said they most often found bugs after code is merged in a test environment.
At the same time, nearly 70% of developers said that while they are expected to write secure code, they get little guidance or help. One disgruntled programmer said, "It's a mess, no standardization, most of my work has never had a security scan." Another problem is it seems many companies don't take security seriously enough. Nearly 44% of those surveyed reported that they're not judged on their security vulnerabilities.
ZDNet also cites Linus Torvalds' remarks on the Linux kernel mailing list in 2017, complaining about how security people celebrate when code is hardened against an invalid access. "[F]rom a developer standpoint, things really are not done. Not even close. From a developer standpoint, the bad access was just a symptom, and it needs to be reported, and debugged, and fixed, so that the bug actually gets corrected. So from a developer standpoint, the end point of hardening is just the starting point, and when you think you're done, we're really only getting started."
Torvalds then pointed out that the user community also has a third set of entirely different expectations, adding that "the number one rule of kernel development is that 'we don't break users'. Because without users, your program is pointless, and all the development work you've done over decades is pointless... and security is pointless too, in the end." Juggling the interest of users and developers, Torvalds suggests security people should adopt "do no harm" as their mantra, and "when adding hardening features, the first step should *ALWAYS* be 'just report it'. Not killing things, not even stopping the access. Report it. Nothing else."
Yes (Score:3, Insightful)
The problem is that real security experts are rare and expensive.
In a large organisation when you have developers who actually know a thing or two about security and an army of "cyber security" professionals that just enforce generic security policy and does not understand enough about the actual underlying impact, the tension is real.
There are a few really solid security experts I know in my organisation, but they are the security architects or is not directly in the enforcement line, so day to day devs ju
Re:Yes (Score:5, Insightful)
As a security consultant that can also code and do system and network administration, I can confirm that. But it gets worse, while developers usually do not understand security, they can at least code. The "security" people you describe, cannot code, have no clue about networking or system administration, and, in fact, do not really understand security either. Hence they get on everybodies nerve requiring people to follow all sorts of seemingly nonsensical requirements, but they are never helpful, because they cannot be. At the same time, what they do does not actually make things more secure, because security policies and technical security usually have very little overlap.
It is high time to remove the policy-enforcers and bring in actual real IT security experts. At the very least these need to be knowledgeable and experienced in technical security, some real coding, system administration and computer networking. Not for all platforms or languages or application types, of course, but enough that they have a real understanding of all involved technologies. And yes, these people will be really expensive. But they will also be really helpful and it will be a lot cheaper to have them than not. The present situation often serves to decrease security. It is like large organizations think they can make IT secure by making policies requiring that everything has to be secure and then enforcing them. That does not work and cannot work.
Nobody is hiring real security experts (Score:1)
I'm a sys/netadmin, I can code down to assembly level, I've done the system integration thing, even some release management, and I know my way around well enough that cracking a game and helping writing an exploit are things I've done (a long time ago, but I did them). And every time I've tried to sell myself as being multi-faceted, no matter the role, eyes glaze over and people look for someone else.
I've more or less stopped looking before"devops" became a fad but my impression is that it boils down to "ca
Re: (Score:2)
Indeed. You did exactly what the market urgently needs. But the market is too stupid to see that. It will probably take somebody like Gartner to describe the blatantly obvious get it into the thick skulls of the C-level people.
Re:No (Score:5, Insightful)
Better question (Score:5, Insightful)
While we're on the topic, should social media companies with major data a breaches (especially intentional ones like Facebook with Cambridge Analytica) be subject to the corporate death penalty of confiscation of assets and being broken up? The shareholders should be equally culpable since it is, after all, their company.
If this happens often enough, investors might consider ethical investing.
Understand the need for funding. (Score:5, Insightful)
Running a software development business is hard. One thing that is hard about it is finding the right amount to spend on security.
If you spend too much on security (which includes all the code scanning, penetration testing, human code-reviewing, re-writing, etc., to find the security holes and make sure they are fixed, AS WELL as training up developers on secure coding practices, ensuring that they include such practices in their designs from the get-go, etc.,) your project can very easily spiral out of control and get cancelled. And even if that isn't its fate, your project will take so long to complete that a competitor will get theirs on the market first, and steal your clients. And even if there are still clients by the time you get to market, your product will be so expensive, including and especially what you much now charge for ongoing maintenance, that you have priced yourself right out of the market.
These are not academic, pie-in-the-sky concerns. These are real-world practical realities that investors and managers must face every day. These concerns constitute a critical component of the success of a software development business.
A startup cannot afford to invest much in security. The funding is simply not there, and the business will utterly fail if it doesn't get a user-pleasing product out the door quickly. All successful businesses started out by skimping on security, because those that tried not to skimp were defeated in the market. It is an unpleasant reality, but it is a reality nonetheless.
Very mature businesses, like Microsoft et al, can afford to properly invest in security. They must, of course, find some way of fixing their old insecure software without breaking it for their existing user base, and then they must continually invest in the ongoing necessities of maintaining secure software. It is why big businesses get the reputation for frustrating development environments where every line of code has to be reviewed by six people before you are even allowed to type it, and then six more after that.
If there is tension between people who specialize in security and people who don't, that is probably just a symptom of this underlying issue. Security is expensive, and hard.
Re:Understand the need for funding. (Score:4, Interesting)
Not just funding. In one place I have worked the security team was mandated to get *everything* they asked for if at all possible. This resulted in just a bad product.
In the market, I saw a product lost handily because it was damn near impossible to use for all the inconveniences mandated by security team. Forced interactive password/passphrase prompts at every use of a password or private key destroyed automation. Requiring certificate validation and not allowing customer registration of their own CA certificates meant most customers couldn't use it at all. A requirement to use OS supplied SSL implementations combined with Cipher suites so picky that half of the customers were not able to even connect.
Of course, the competency of that security team is certainly an issue and the fact that the dev team also lacked competency to push back and counter with secure proposals that would still provide the capabilities needed to compete in the market or escalated the flat-out dumb requirements. However the competing product won by doing things pretty much exactly the way the security team had forbidden.
Re:Understand the need for funding. (Score:4, Insightful)
There's one simple test for telling whether you are dealing with a real security team, or with procedure-following yokels.
When you are initially signing up and are prompted for a new password, will the highly random password created by your favorite password manager be accepted, or will it make you add a certain number of capitals, numbers and special characters?
Re: (Score:3)
Oh yeah, this x 1,000
While working at one of the a largest US banks, they rolled out their *standard* security package on one of the systems my group used. We went from variable-length (8-16), all characters, all numbers, all symbols, to the "Your-password-must-be-8-characters-including-one-upper-case,one-lower-case,one-number,and-one-special-character(!@#$)'".
And the a$$hat from security telling us how this IMPROVED security.
What was really scary was I called him on that utter BS. And then had to explain
Re:Understand the need for funding. (Score:4, Interesting)
The goal of first to market today is not to get any sort of first mover advantage, but to get bought out before the competition comes in, leaving your acquirer holding the bag for all the early development and market-proving costs. See Slak v Teamview or whatever they call themselves.
Or go back a bit - Lotus 1-2-3 killed off VisiCalc and was utterly destroyed by Excel. WordPerfect got crushed by Word, a vastly inferior product backed by vastly more wads of cash. Borland at one time held 2/3 of the entire pc programming market. They were the first with an ATT-c++ compliant compiler - Borland c++ 3.1. They made serious moves to take over the office, buying dBASE, WordPerfect, and coming out with their own spreadsheet.
Their Delphi was a potential Visual Basic killer. Then they took the eye off the ball, killing off the fictional Frank Borland and becoming Inprise, the "enterprise software life-cycle company."
So much for first mover advantage.
Re: (Score:2)
Is there a conflict of goals between developers and clueless management who hire clueless UX "designers "? And when a project fails, shouldn't the management be the ones paying with their jobs?
That would be an excellent start. And then fire all the clueless security "experts" that can do nothing except read and write policies.
Re: (Score:1)
You are a brexiteer? Should have guessed. The stupidity literally drips from most of your postings.
Re: (Score:2, Insightful)
Hum, I call out the dim whit. This is a specific reference to a quote by Michael Gove (a leading Brexiteer) who said " I think the people in this country have had enough of experts", repeatedly in an interview on Sky News on 3rd June 2016.
The implication being that a portion of the electorate listened to the likes of Gove ignored the "experts" and lead us (aka the UK though there is not much united about this kingdom) into the mess we are in today.
Re: (Score:1)
Maybe I do not watch Sky News? And maybe I am not a Brit?
Re: (Score:2)
The shareholders should be equally culpable since it is, after all, their company.
I hope you don't have a 401k, otherwise you're going to JAIL! That's the fundamental problem with the slippery slope of not setting up an upper limit of blame. You've effectively just blamed shareholders for the poor work of a UX designers, or a bug that slipped through a code review. It's quite silly really.
Also Cambridge Analytica was *not* a data breach. If anything the data breach was that they were found out doing this dodgy planned and approved business activity.
Re: (Score:2)
Shareholders ARE to blame by not doing due diligence, throwing money at businesses that give too much voice to UX designers over good design where functionality dictates design decisions.
We're not just blaming software coders for the Boeing 737MAX fiasco - the shareholders are deservedly taking a hit by not insisting the board be more diligent, the FAA has lost the right to sole-certify planes made by American
Re: (Score:2)
While some of what you say is true, there are a minority of developers who are eager to show they know security better then me, and will spend weeks arguing about that they don't need to add the s after http because "we have a firewall!", or arguing that they don't need to patch a library because the execution path isn't one they use (sure, but forever, and what happens when one of the banks who run our software run a scanner tool against it, HONESTLY it's easier just to patch it then to lose deals because
of course there is tension. (Score:4, Insightful)
Re: (Score:1)
your devs are going to dismiss your security team so VERY VERY quickly
Where do you work where the "devs" and the "security team" are different people?
Re: (Score:2)
Where do you work where your security team and the devs are the same?
I have enough trouble just getting Business to approve updating environments to solve technical debt. They're starting to get it but with 90% of our infrastructure end of life, it's slow going.
[John]
Re: (Score:1)
Try translating the problem from Hipster to Business English next time.
Re: (Score:2)
In ten years, hipster will be business English. That's kind of worrying.
Re: (Score:2)
Go and listen to Subterranean Homesick Blues, which was hipster language in the past, and see if it still sounds reasonable.
In clothing and visual arts, fads become normalized and then repeat cyclically. And yet, business clothes from the 80s are only very slightly different. Business English changes even more slowly, for the most part.
Re: (Score:3)
There's a spectrum.
There are totally dysfunctional places where never shall security and developers mix.
There are also places that make it sound good by saying "we have a single dev/security team" but what they really mean is they have a developer team that *maybe* has to review a security powerpoint once a year and no serious securtiy team. I'm sure there are genuinely unified teams as well.
I currently work in a place with distinct teams, but a few people are both on the security team and developers. In
Re: (Score:2)
Prior to that it was utterly hopeless, as the devs couldn't articulate their goals in a way security could understand and security couldn't articulate precisely what the developers should do to properly fix findings.
Well said - this is the root cause of the problem. The obvious solution is to be sure developers are properly trained in security and that the requirements and budget for proper security are in the project.
But there are three problems...
1) Most people who claim to be security experts are not, they just know enough to BS management and bully developers.
2) Most developers don't want to be bothered learning and implementing boring stuff like security.
3) If developers actually did know enough about security to
Re: (Score:2)
Rather than hiring a "dev team" that doesn't know how to do security, and a "security team" who doesn't know how to do development, why not hire people that know both and can actually do their job?
Re: (Score:1)
The security/IT people were flabbergasted when I asked for it.
Boo, that !
A new CoC (Score:1)
Re: A new CoC (Score:2)
Your post is offensive to CoC lovers and therefore in violation of the CoC. Reported to HR!
Re: (Score:2)
True, but you have to wait for them to harass you.
No (Score:5, Insightful)
There is no fucking tension you jackasses.
Developers have been and will always be a group of self aggrandizing morons, but they still get work done. Security Pro's are mostly diva's whining about folks following their bullshit standards written by morons that believe in security theater more than actual security. They tend to create contention with everyone they deal with because of it.
Developers will focus on security when they have been told too by their managers and until that happens they don't give a flying flipping shit about anything other than getting their job done just like everyone else.
As usual, the real problem which is businesses fundamentally not understanding security, is not being talked about. Instead here we yak about a bullshit rivalry that does not exist, never existed, and likely never will exist.
As a practical engineer the problem has always been and will always be... management. When they get paid to give a fuck, everyone else gets paid to give a fuck. Until then, they are paid to not give a fuck.
Re: No (Score:1)
Same here I'm in the mist of fighting a decision with security "policy" breach with no actual security risk or impact.
The security tech experts and architects all agree with the devs that it's not an actual issue but it just does not align with the policy wording and we stuck for months because the process says we have to go thru 5 levels of approvals and via the chain of security risk professionals who know shit about tech security and explain to them why it's not an issue.... the last level of approval is
Re: (Score:2)
Re: (Score:2)
Blah blah, security people suck, engineers can't think for themselves, I hate managers, blah blah.
You've obviously worked in some toxic environments my friend. Your words read like a woman scourned.
Re: (Score:2)
One that isn't misogynistic, where bro culture isn't a thing, where crunch time is banned, and so is PowerPoint?
You mean, developers are not professionals? (Score:1)
So what you're saying is that, in your world, developers do not consider themselves to be professionals, but merely bums on seats, doing the least possible to not get fired. There is no such thing as professional duty for them, no strong personal motivation to do good work. When that work carr
Re: (Score:3)
So what you're saying is that, in your world, developers do not consider themselves to be professionals, but merely bums on seats, doing the least possible to not get fired. There is no such thing as professional duty for them, no strong personal motivation to do good work. When that work carries security implications, being a professional would normally imply being focused on writing secure software, but lacking such professional duty, they don't give a toss?
As a professional developer, everything I do or work on carries security implications. And as a worker who wants to keep his job, my first duty is keeping management happy. So, management's concerns are my concerns. If management worries about delivering business functionality more than security, then I worry more about business functionality than security. Sure, I can attempt to code securely. Sure, I can raise issues that I think could be problems in the future. But ultimately, management signs my paychec
Re: (Score:2)
Re: (Score:2)
Re: (Score:1)
Objectivity is just opinion in disguise.
Re: (Score:2)
What business cares about is income. If it costs $10,000 to run some software through some tests and resolve security issues, not user issues but security issues, they're going to ignore you. It'll be put on the list of things to be updated the next time around assuming there is a next time. We have products that don't have a product manager because they're "done". The product was created, customers signed up for contracts, and it satisfies the requirements of the customer. So all the Changes we submit to u
Re: (Score:2)
I agree.
I think your tone - calling the authors "jackasses" - is unnecessary. Would you say that to their face?
But I agree: developers respond to the work requests and priorities they are given.
In fact, I wonder if we need for the companies that build software to be legally liable for losses when software is penetrated due to a vulnerability. What do you think?
Re: (Score:2)
We've seen so many successful ransom attempts because people don't even want to pay for the most basic of security measures - backing up their shit. If they won't even do that, how much harder is it going to get them to do anything that goes beyond that?
Management is responsible for bad decisions, and whistleblowers need to be protected and encouraged.
Re: Rust will fix it (Score:2)
Security issues are a lot more than programming language based. You can have systematic issues even if you run Ada.
There is no tool today that really can figure out the high level security issues.
Organizational dysfunction (Score:5, Interesting)
So I've been a pen tester for some time, and the biggest issue that repeatedly occurs (which isn't an OWASP or other bug) is simple organizational dysfunction.
Developers/engineers are the meat in the sandwich, and a good infosec consultant will be highly aware of politics and other issues that may impact an organisation's security. Otherwise you could just pay for the next Muppet to run some automated scans. It's the difference.
In almost every report I've written, I have made mention of resourcing issues causing staff to be overwhelmed or make mistakes, but only one company I knew of in the oil and gas sector actually took that onboard and hired more engineers, although only after I took it to the board. One of them hugged me next time I was there.
The profit incentive is countering good security. It's that simple. Companies don't return record profits to shareholders by patching bugs, and the idea that reputation loss affects then is tenuous at best. Try getting Linksys to care about RCE in a product that's only manufactured for 3 months, sold for 12 and had slim margins to begin with.
Re: (Score:2)
At least product bugs are more likely to get addressed if they bubble up high enough. Security issues? Unlikely unless they have extremely high visibility in the world.
[John]
Security is tension! (Score:2)
That camera in the corner is no fun at all.
Re: (Score:2)
That's not a "plumber's crack," that's "security through obscurity."
No More Security Guys. (Score:1)
The idea of a "security guy", at least a PURE security guy is pretty stupid IMO.
The problem is that typically these "security guys" don't understand development one iota. They get trained up with often very little, or poor understanding of software development, or software in general. Many of them have never written a line of code in their lives! One idiot security guy at a former employer insisted that I run "the latest version of Apache" when his scanner said I wasn't running "the latest version". And
Re: (Score:2)
So I've dealt with the *exact* same sort of requirements. Another team caved and shipped their own apache, openssl, and a host of other libraries because CentOS version numbers were flagged as too old.
They had security issues because they were on a six month release cadence and security approval came 2 months before release. By the time it released, it inevitably had applicable CVEs but customer wouldn't be able to update for 6 months or everyone ran around like crazy to produce a "hotfix" for someone els
Re: (Score:2)
I will say that in my case, the security person given citations was happy to sign off and learn something new after being pointed to the distributions resources for vulnerability tracking and added it to his repertoire. and caused him to go back to the code scanning vendor to complain about it's inadequacy at tracking relevance of the target to actual vulnerabilities versus a blind version check. In the end the person was not too ecstatic that the tool would not change and he would have to actually do tedi
yes, cuz management (Score:2)
Duh, There is no penalty for bad security (Score:3)
So no kidding it's hard to get companies and developers to take security seriously. It's an inconvenience, it makes things harder for the end user and it makes support more difficult. Security theatre doesn't have the drawbacks and sells even better than real security.
Re: (Score:2)
I agree with that, I'm transitioning from sysadmin/dev work to more security and advising and the first thing a company wants to do is limit its liability. In the end, no customer wants to pay for security up front, and at this point it is really a roll of the dice, if your source code never gets published, it's very unlikely someone would ever find the gaping holes.
But all the customer wants to do is pay the $5000 for the app and then have the same version sell for at least 10 years and most don't even kno
Yes there is tension, and the security guys lose (Score:1)
Yes, there is tension. As a developer, if I don't get deliverables in, I lose my job, or marketing begs management to have everything offshored. Every day, I have a standup meeting, and if I don't have something done, I either point to someone and say they are blocking me, or I have to have a damn good explanation, or be publicly excoriated in front of all my co-workers.
As a dev, security has no ROI. I don't care if my code requires SELinux to be off, runs as root, runs as SYSTEM in Oracle, and uses full
Management problem. (Score:2)
Developers are being pushed by management to make the latest version of product X functional. Fixing old versions isn't a priority because management doesn't make it a priority ("Where's the profit in that?"). It's only when it can cause bad publicity (which can cause a loss of profits) does management actually allocate time to fix old versions. Management doesn't like that security professionals exist because to them, the point of the product isn't the product itself but rather the money they make from
There is (Score:2)
Generally, management just sets this up for failure by having distinct 'teams', the team that develops but need not be vetted for security competency and another team that only does security but has no sense of practicality.
First off, they have antagonistic professional goals. If something ships but a security problem is there, then the security team will get hit but not the dev team. If nothing ships at all, the dev team gets hit but not the security team. Therefore the safest thing for security team is
Anti-Betteridge (Score:2)
Usually if a headline the answer is 'no'.
But sometimes it's 'duh!'.
Trade-offs are everywhere in life; this is just one of them.
If there is a tension... (Score:2)
All Goes Back to Senior MAnagement (Score:2)
I live this every day. Developers do what they're incented to do. When senior management values features over security and incents developers accordingly, this is inevitable. But management values features over products because they focus too much on revenue growth and not enough on revenue preservation. We constantly have customers that threaten to and sometimes do move to competitive products because of a lack of updates and yet management does next to nothing about it.
To quote the article: (Score:3)
" ...nearly 70% of developers said that while they are expected to write secure code, they get little guidance or help. One disgruntled programmer said, "It's a mess, no standardization, most of my work has never had a security scan.""
All of this information is available: There's books and tutorials and videos on how to hack just about anything. In addition, there's documentation from NIST, CIS, SANS and especially OWASP on what the problems are and how to fix them.
So when some snivelling developer whines that we security people "don't give them guidance or help" maybe that dev needs to get off their ass and learn how to do their jobs. The security researchers are far too busy trying to keep up with the villains out there to stop and spoon feed developers with the simple information they could easily learn by using some of the free resources I mentioned above. And if you are a dev and your app is not getting security scans, why aren't YOU doing them?
Re: (Score:2)
There's books and tutorials and videos on how to hack just about anything.
We do not need guides on how to hack, we need information on how to program securely.
There's documentation from NIST
I have read NIST articles until my eyes wanted to bleed and never come across how to program more securely. Only what must happen and hints at what can help make it happen. I can't speak to the other initiatives you mention.
learn how to do their jobs
It seems that we know how to program. Do you know how to secure what we program?
simple information they could easily learn by using some of the free resources I mentioned above..
Mention, but provide no links to. I own a book on hacking. I own books and have many PDFs that I only have to help me
Security is never in the budget (Score:2)
C-levels don't understand security, department managers don't want to pay for security and project managers just want to get the damn thing tested and signed off on time and on budget. There's immense pressure to "just get it done". Then after a breach, the mahogany suite wants to hold Joe Codemonkey, Jr. personally responsible for "writing insecure code".
You either pay for it up front or you pay even more on the back end. Guess which one management inevitably chooses?