Developers Can't Seem To Stop Exposing Credentials in Publicly Accessible Code (arstechnica.com) 59
Despite more than a decade of reminding, prodding, and downright nagging, a surprising number of developers still can't bring themselves to keep their code free of credentials that provide the keys to their kingdoms to anyone who takes the time to look for them. From a report: The lapse stems from immature coding practices in which developers embed cryptographic keys, security tokens, passwords, and other forms of credentials directly into the source code they write. The credentials make it easy for the underlying program to access databases or cloud services necessary for it to work as intended. [...]
The number of studies published since following the revelations underscored just how common the practice had been and remained in the years immediately following Uber's cautionary tale. Sadly, the negligence continues even now. Researchers from security firm GitGuardian this week reported finding almost 4,000 unique secrets stashed inside a total of 450,000 projects submitted to PyPI, the official code repository for the Python programming language. Nearly 3,000 projects contained at least one unique secret. Many secrets were leaked more than once, bringing the total number of exposed secrets to almost 57,000.
The number of studies published since following the revelations underscored just how common the practice had been and remained in the years immediately following Uber's cautionary tale. Sadly, the negligence continues even now. Researchers from security firm GitGuardian this week reported finding almost 4,000 unique secrets stashed inside a total of 450,000 projects submitted to PyPI, the official code repository for the Python programming language. Nearly 3,000 projects contained at least one unique secret. Many secrets were leaked more than once, bringing the total number of exposed secrets to almost 57,000.
Security? Never Heard of It (Score:3)
Re:Security? Never Heard of It (Score:5, Insightful)
It seems like some devs think of GitHub as their personal diary – "Dear Diary, today I hardcoded my AWS keys in app.js. Shh, it's our little secret!" Spoiler alert: It's not. It's Python's little secret too, and JavaScript’s, and every hacker on the planet thanks to our generous contributions to the 'Open Source Credential Repository' – GitHub's unofficial title, apparently.
I'm sure they're aware, the problem is that security, almost by definition, is a nuisance (ie, it stops folks from doing things).
So if you're developing on more than one machine, you pop the json file with credentials into git. You might not even do it deliberately... but there it ends up and don't bother to investigate when the machine you don't remember having the credentials has them after all.
Or you're figuring out some new API, so just to simplify things you paste the password into the source code directly (just while you're debugging). Then you finally get things working, you forget about your workaround, and now it's pushed up to the central repo.
Security breaches are rarely the result of people not knowing the right thing to do, it's usually because they know the right things to do but don't bother because the abstract threat of a security breach is a much more distant concern than whatever they're currently working on.
Re: (Score:3)
I'm sure they're aware, the problem is that security, almost by definition, is a nuisance (ie, it stops folks from doing things).
Yes, but it doesn't have to be. The real problem is that even though we have cross-platform APIs for things like crypto, etc., we don't have any universally accepted cross-platform APIs for password and key management.
And ne'er the twain shall meet.
If we had a single unif
Winelib uses... (Score:2)
Windows uses Credential Manager/Password Vault.
Linux uses... a pile of different and incompatible APIs, ranging from GNOME keyring to dot files in the user's home directory and everything in between.
What does Wine use when loading a Windows application that uses Credential Manager/Password Vault?
Re: (Score:2)
I couldn't care less since I use anti-obfuscation to protect all the applications I write: all usernames and passwords are set to "123456" and I put it in clear in the code. Somebody looking at the source will obviously think it's obfuscation but in reality it's a new pattern I created and I call it "anti-obfuscation". The fact and the truth is that nobody will ever try those credentials even if they are the real ones and really work in reality. They just think I obfuscated it in the code. This has never
Re: (Score:3)
I'm sure they're aware, the problem is that security, almost by definition, is a nuisance (ie, it stops folks from doing things).
Yes, but it doesn't have to be. The real problem is that even though we have cross-platform APIs for things like crypto, etc., we don't have any universally accepted cross-platform APIs for password and key management.
And ne'er the twain shall meet.
If we had a single unified cross-platform keychain standard, then you'd just code to that standard, and the key would always be in the user's keychain, and it would "just work", and nobody would need to hard-code keys in their code. But instead, everybody rolled their own key storage solutions, and the end result is that developers don't use them as often as they should.
I like your idea [xkcd.com].
Re: (Score:2)
I'm sure they're aware, the problem is that security, almost by definition, is a nuisance (ie, it stops folks from doing things).
Yes, but it doesn't have to be. The real problem is that even though we have cross-platform APIs for things like crypto, etc., we don't have any universally accepted cross-platform APIs for password and key management.
And ne'er the twain shall meet.
If we had a single unified cross-platform keychain standard, then you'd just code to that standard, and the key would always be in the user's keychain, and it would "just work", and nobody would need to hard-code keys in their code. But instead, everybody rolled their own key storage solutions, and the end result is that developers don't use them as often as they should.
I like your idea [xkcd.com].
Back-end it into the GNOME keyring on Linux, Keychain on macOS and iOS, etc., and just make it a single API that provides a unified, standard interface regardless of platform. Embrace, extend....
Re: (Score:2)
One of the best things about git is that it's default include. All too often important files were missed from svn and similar versioning systems.
The only problem is, if you don't tell it explicitly not to, it will include your private keys.
It might actually be quite useful to have some mechanism to check files for things that look like keys when committing them.
Re: (Score:2)
Re: (Score:2)
As the thinking goes, you can hire 2 inexperienced developers for the price of someone competent. And 4 outsourced developers for the price of one competent person. After all, it's in the name, "developing nation", that means they know how to develop, right?
Re: (Score:3)
Re: (Score:3)
I also include a dozen or more "private" keys in my github repos.
However, they're totally harmless -- they're part of testcases, demos, etc -- e.g. one of them is included verbatim in a script which simulates the server that a shady android app (which I have reverse-engineered and modified) used to call back to.
Anybody could regenerate them anytime they want with a couple of openssl commands and they would serve just as well their purpose.
I wonder how much of the 4000 "unique secrets" they have discovered a
Re: (Score:2)
This is a problem too. :| There is a sample certificate in the RFC that defines JWT encoding. I included that well-known public sample certificate in tests for a PowerShell script that generates JWTs and got flooded by squawks when I checked those tests into git.
Above that, it *is* a problem. I want to be able to give my code away. That means testing the code, which means I have to have secrets *somewhere*. What is the right(TM)(R) way to be able to check sample config files into git and not have your
why are not loading the env from the pipeline? (Score:2)
why are not loading the env from the pipeline?
Re: (Score:2)
By "pipeline" do you mean standard input? If not, what do you mean?
The why is pretty simple (Score:4)
Not embedding secrets requires some form of infrastructure from which to fetch secrets.
Which of course does not exist when you are prototyping and in the early usually iterative design phases of building something.
The alternative is to have things like secret server and policy 'thou shalt use it' but that means you have to let people create things, and than you have to have people own those things or they will be created and abandon and than you have another problem, you can't solve any other way than well nobody can tell me what this but it hasn't be accessed in 90 days so its gets all permissions revoked and we can hold onto it for the rest of year to see if something goes boom.
If that happens Murphy's law states it will 4 am on a Saturday.
The only real way to avoid this is lots more institutional hurdles and design documentation requirements. That means though high costs resources like lead develops hit hard stops on Tuesday afternoon because they *need* something to play with, but nobody from Team X can take a meeting until 10a Wednesday.
K8 has secrets and can be pipeline loaded as helm (Score:2)
K8 has secrets and can be pipeline loaded as helm vars
Re: (Score:2)
Yup, too often security is a last minute concern. Everyone hates security, except for the customers. Once customers know about security they will ask for it. Unfortunately, many customers often just assume there's security and they only learn to demand it once their data has been exposed in a moronic lapse of security common sense.
Re: (Score:2)
My former co-worker Kyle at his startup Agilicus tackled this a while ago - did a "ted-x" on solving via Kustomize and SOPS here is the YT video https://www.youtube.com/watch?... [youtube.com]
How much is due to a lack of GIT Knowledge? (Score:3)
I've seen developers and engineers hard code admin account logins, into software, simply to save time so they don't have to log in. I've seen people hard code SSH keys, so the software can log in to the service, without being asked. Numerous times, I've been told that configuration files are too confusing or time-wasting, so the details should just be built into the software. In most of those case that was later followed with: “Why didn't anyone secure the software??!?!?!?!!”, while I laughed as I always put my foot down and refuse to hard code stuff that should be private. Hell, I'll spend my weekends doing the work, to prevent the hard-coding, because it's not worth the fallout from lazy people and lazy management.
Re: (Score:2)
The majority of times I see a developer or engineer throw or add keys into a project, is because they run git commands that to add everything to the repo. I've been told a number of times, to paraphrase: “It's quicker to add everything, I don't have the time to select the files one by one.”. I would think if we educate people on how to use GIT properly, we'd stop 95% of the leaks, the other 5%, pure opinion, are due to sloppy standards.
I'd argue that the real problem is that they are mixing configuration with code. Sample configuration files in git are fine, but the actual configuration for any tool should always come from some well-known location, whether that's a keyring or a dot file in the user's home directory. Any tool that reads that sort of configuration data from the current directory is a security hole in and of itself, because it means that changing into a hostile directory and running that command could result in unexpected
Re: (Score:2)
Re: (Score:2)
The that asks the question, where were the code reviews? Where were the design reviews? Where was the oversight? These are problems though that happen in most startups - because they generally have lots of employees with scant real-world experience, but also very tight deadlines so get a product appearing to work before funding dries up. Also startups sometimes have experienced developers who never had to deal with as much work as they get in a startup where suddenly they're in charge of 12 different com
Re: (Score:2)
Code and design reviews are important, but if people don't understand safe operation, it doesn't matter. People will spend weeks developing plans that don't and can't work, and that rely on security violations just to function.
Re: (Score:2)
Like I said, "startups". Startups are not normal companies. Most are started by these people called "entrepreneurs." This is a French word meaning "la douche." They will have a grand idea, but not necessarily know how to implement it. So they hire their frat bro friends to work for them. The frat bros don't know how to implement it either, but there are lots of books to buy entitled "How to Do Hard Thing In 12 Days". Then they stop at day 3. Later other people get hired and they cannot fix the earli
Re: (Score:2)
We have the problem that 6 years ago, I didn't have time to write documentation, I was working literally 14-hour days, 6 days a week. Move down the road 3
Re: (Score:2)
I don't have the time to select the files one by one.
Drives me crazy to see that. scaffold the project (e.g., gradle init) if necessary, create gitignore and ignore the junk and push.
Clone the project on another folder or ask a colleague to do it and double-check it's not keeping stuff that's automatically generated.
Create a fixed filename for secrets and gitignore that. Only read from that filename until your project matures to more sophisticated scenarios.
Takes a few minutes.
Re: (Score:2)
If you set up GIT hooks, you can catch a lot of these problems, but even if you have them setup, the best final line of defence is a paranoid DevOps lead. I'v
It's a chicken and egg problem (Score:5, Interesting)
You need a credential to access a database, let's say. Then you might encrypt the data, and need a key to decrypt it. Where do you put this data? You might think of encrypting it. Great idea, but where is the key for that?
There are good situational solutions to these issues, but they are out of scope of your application proper. And, you may find a good solution, but since your application needs this actual data at some point, how do you stop it from being exfiltrated?
The point is that this is not a solved problem as the summary seems to imply. I suppose not embedding them in code is step 1, but that's hardly enough.
Re: (Score:2)
Its a *mostly* but you have to decide how its going to be done top level (as in like CIO) and say we are NOT going to have exceptions.
Then you have pick a specific and all of those are going to offer some combination easy, secure, cheap - for which you get to select two of the three, or you may optional go for only one of the three, but having all three is a not possible.
Most organization will at some point in their lives determine that to be unacceptable, and make an exception, then more exceptions, then a
Re: (Score:2)
Your presumption that you know everything and others are clueless is going to age very well. Promise. - Your future self.
It's because of dirt cheap contractors (Score:3, Interesting)
Re: (Score:2)
Because they're easily abused (Score:2)
H1-B workers face a constant threat of deportation.
Neither worker can unionize. Offshore workers face hostile governments who would use open violence, H1-Bs would get deported.
Neither group is used for short term hiring, and as for "low end" there's no such thing. Work is work. Stop treating people who do
Re: (Score:2)
Re: (Score:2)
I don't why you had to add "offshore" and "H1-B" to your argument.
To be fair, the GP said cheap offshore and H-1B workers. Both offshoring and H-1B visas are heavily abused by contracting companies as a way of hiring labor for below-market wages, which tends to attract workers who aren't good enough to get jobs that pay better. Most of the workers who are good at their jobs quickly move on to something more stable unless they have specific reasons to want to do contracting (e.g. needing to work part time). So the talent pool ends up being a crapshoot, and when you do g
Re: (Score:2)
Quick fix for dev code (Score:2)
Just stick the credentials into a separate file to include and put that file in .gitignore right away so it doesn't end up in your public repo.
Re: (Score:2)
Re: (Score:2)
You don't need a .gitignore.
Here are 4 other options:
(1) It's the same problem as keeping your customer database and other Application data out of Git. /var/run/appname-production/db, for example --- files that belong to the Deployment state, and not the source code used to originally initialize the deployment
Have a system "runtime" directory for data which doesn't contain source code and is never checked into git.
such as
(2) Use an environment var
Re: (Score:2)
I have handled this by using a .gitignore file for my repository in development. However, when the program detects that the credentials file I use is not in place, it will do something similar to the parent, depending on what is available:
1: If there is a PAM like Hashicorp Vault available, have that used.
2: I don't use environment variables because ps can show the data. Instead, I prompt the user.
3: In some cases, I use the onboard HSM or a YubiKey HSM plugged in. At the minimum, I can unlock the sto
Where I use to work (Score:3)
Where I use to work, the admins would start almost all java processes using with these arguments:
-u ID -p password
Nice for me as a developer, a simple ps(1) got me the info. But they were following the company (a fortune 500) standards, the ID/PW was in a file readable only by the process owner, but they would set an env variable and use it as a argument to the java process.
More often then not, the ID had enhanced access, so if I got stuck on permissions when testing any of my code changes, I had a nice way to fix the issue :) This was "standard" was also used on production servers.
BTW, I would never think of hard-coding login info in a program/script. So as you can see from above, it was not just developers doing this. It is everywhere.
Re: (Score:2)
It sounds like the admin for that system needed to at least add the hidepid=2 options to their /proc fstab.
instead of going with the defaults there -- that's just the long-standing best common practice to use that instead of the defaults;
as it prevents non-root users from being able to see other users' PIDs and command lines.
But also if the creds are in the environment, then the program could have been written to check the environment variables, lol. And being that it's Java... Java itself has
Inexperience is Inevitable (Score:3)
Very few software shops are staffed 100% with experts, or even solid developers. Even some solid developers are not specifically experienced in dealing with security.
If this actually matters, you need to either:
1) Have rigorous code reviews prior to anything going into a public facing repository
2) Not provide junior devs the sensitive production credentials in the first place
I tend to lean towards #2. Hide the actual credentials behind whatever your preferred method is (injected secrets, environment vars, whatever). Have a solid dev environment so devs don't need to do much on actual production systems. Ideally your dev environment is automatically blown away routinely and new credentials are generated.
Relying on every developer in your project to always do the right thing is never going to be a good answer.
Re: (Score:2)
I propose, for organizations anyways:
3) Forced expiration of credentials after each version update - new secrets will be created for the updated version, and the old secrets should expire.
Credentials should be strong and random but contain a unique "Magic word" embedded inside.
Companies should then have a system that scans for and flags on commits that contain the Magic word. Upon detection; it should fire off an Alert and audit of the changes to automatically revoke whichever credential someh
I'm too busy developing software (Score:2)
Here let me check in the security tokens to our public Gitlab to make deployment and CI/CD easier. Let's ignore the fact that all these CI/CD systems have a place to put secrets outside of public view, and even has built-in scanners to detect and remove matching token(s) from the log output.
The only explanation I have for the prevalence of this issue in my industry is a combination of naïveté and laziness.
Postman - REST API Tester forced Everything Online (Score:4, Interesting)
Just recently in the last few months, the famous application postman that everybody was using for testing, rest API and doing script work for pre-processing and post-processing has forced everybody into their online fog/cloud system and migrated all of their locally hosted an offline work from their offline scratch pad into their online cloud .
This included all of the credentials that were stored offline and securely locally on everybody's development workstation confined within the corporate work environment by the application now forced everyone online. The documentation even states that all of your work is saved and it is now uploaded up into their cloud, which includes all of the credentials .
So even when normal developers who are security conscious and store all of their work offline and locally on their workstations and laptops securely confined within the local network protected by their cyber security teams when an application forces a mandatory upgrade that then uploads all of their work into the vendor's cloud, including all of the credentials without letting the people know that this is happening and that version this exposes a problem .
This is a huge problem with the postman application and droves of developers that are security conscients are migrating away from it and looking for alternatives like soapUI, thunder client, hopscotch and, anything else that can do similar work.
So even when we are trying to be security conscious, a forced upgrade of the application pushed on us by our local desktop teams ends up moving. All of our secure credentials to a vendor's cloud without the developers themselves are proving this. Because the desktop team says keep all versions up to date of all software packages. So on your next launch all of your data is sent up into the cloud and the fog and it's not even up to you. Without a warning I thought this is the version that's going to screw you the next time. You press the icon to launch the app.
Goodbye Postman, you are nice but now you suck because you forced everyone into your cloud and fog full of insecure storage. Hello something else.
Postman - Forced Online Complaints:
https://www.google.com/search?... [google.com]
https://community.postman.com/... [postman.com]
Alternatives:
https://katalon.com/resources-... [katalon.com]
Re: Postman - REST API Tester forced Everything On (Score:2)
I just want to add this that they lock you out of the data that you had stored locally and securely on your local workstation that you were doing work on and now you can no longer access it unless you actually migrate to their cloud and fog version .
They write some kind of platitudes about being super secure and having all of your stuff. Cryptographically encrypted online but we all know all of those password managers that got hacked or authentication providers like Okta that get hacked regularly.
You know w
Re: (Score:2)
I got my own problems with Postman and decided to leave the sinking ship a while ago. I now use a bunch of shell scripts that call their APIs with a curl command. This way I can have every API call test in my GIT repository and versioned too. Also easier to use in my integration tests.
Systemic flaw? (Score:2)
What's the most common credential-containing file, and how are revision control systems allowing them to be checked in without sounding the alarm? Also, it'd be nice to be able to nuke the first rev when you accidentally did that. Of course, you should change your passwords anyway; but with a little bit of thought we can make it a lot harder to do this. After all, you're developers. You've got the ability to, you know, make computers do things for you. Refusing to check in a .key file or whatever, refu
Re: (Score:2)
Scanning for credentials in code is hard. Fortify can do it, but not reliably and with manual work needed on it due to false positives.
Re: (Score:2)
Oh dang, so it's not even a password file? They're hard-coding their bloody credentials in the code? All of the sudden I feel like some kind of level-10 security master.
Re: (Score:2)
Hehehehehe, yes, it is _that_ pathetic. Well, the morons that check in their config file containing life passwords exist too! Need some variety on stupidity after all.
Of course they can (Score:2)
In fact, they basically never did this stupid thing. These people here are not developers, they are imposters pretending to be developers. Happens when people doing engineering work do have to fulfil only low or no standards and when greedy but stupid (the regular kind) business people think they can do things cheaper than possible because they understand nothing.