Invented-Here Syndrome 158
edA-qa writes: Are you afraid to write code? Does the thought linger in your brain that somewhere out there somebody has already done this? Do you find yourself trapped in an analysis cycle where nothing is getting done? Is your product mutating to accommodate third party components? If yes, then perhaps you are suffering from invented-here syndrome.
Most of use are aware of not-invented-here syndrome, but the opposite problem is perhaps equally troublesome. We can get stuck in the mindset that there must be a product, library, or code sample, that already does what we want. Instead of just writing the code we need a lot of effort is spent testing out modules and trying to accommodate our own code. At some point we need to just say, 'stop!', and write the code ourselves.
Most of use are aware of not-invented-here syndrome, but the opposite problem is perhaps equally troublesome. We can get stuck in the mindset that there must be a product, library, or code sample, that already does what we want. Instead of just writing the code we need a lot of effort is spent testing out modules and trying to accommodate our own code. At some point we need to just say, 'stop!', and write the code ourselves.
Blame the Simpsons (Score:1)
Re: (Score:3, Insightful)
If only we had access to their code base!
About time... (Score:5, Insightful)
It feels like this is worst in the Java (enterprise) community, but that could be my imagination. Sometimes I think those programmers need their 3rd party instantiation taken away from them....
And crap, looks like I have been moved over to slashdot-beta so I will probably never see if I get responses....
Re: (Score:1)
Yep, you see this all the time in the iOS development community too. People coding my sticking together 100 slightly incompatible 3rd party libraries, and writing a bunch of glue code. Then hitting problems and responding to help solving them with "we can't do that, we're required to do it this way because the 3rd party library does it that way".
Re:About time... (Score:5, Interesting)
Re: (Score:3)
Re: (Score:3)
I think there's a form of superstition out there to never write any code that's not glue code. Some of this is from a phobia about reinventing the wheel, so that people will waste time in order to avoid wasting time. But some also comes from thinking that third party code will be correct and so increase overall reliability (no matter how much flimsy duck tape is required to attach it).
I see managers with this thinking, it's as if they are assuming their own staff are a bunch of morons so it's safer to rel
Re: (Score:2)
it's as if they are assuming their own staff are a bunch of morons
Its not just that. Its that they tend to believe something along the lines of the Linux mantra -- with enough eyes, all bugs are shallow.
If you have your team of 1 or 2 or a half dozen programmers, you have up to a dozen eyeballs watching that product (and most of the time, it'll be divided up so that those 6 programmers aren't even examining the full code base.)
There's an assumption out there that because a third party library is in use by other people, those other people are doing a lot of the testing an
Re: (Score:1)
Only if his approach also required the exact same amount of maintenance effort. Typically this kind of 3rd party library integration is very high maintenance overhead.
Re: (Score:2)
Third party code can be harder to maintain. The third party often will not help you out, even if you do have an active support contract the support that you get is substandard. If you're Cisco they might help you out, if you're a tiny company with 50 people they'll ignore you and put minimal resources into resolving your problem.
I swear I am not making this up, but I called support once for one component and told them about the bug and details of the fix I came up with for it. They told me that if I alrea
Re: (Score:3)
Yep, you see this all the time in the iOS development community too. People coding my sticking together 100 slightly incompatible 3rd party libraries, and writing a bunch of glue code. Then hitting problems and responding to help solving them with "we can't do that, we're required to do it this way because the 3rd party library does it that way".
Exactly. I don't mind including a good tool that does its job well and leaves everything else alone. But I've run across at least 2 very sad "syndromes" that add-ons sometimes cause:
The first syndrome is represented by libraries that take over everything and unnecessarily restrict your otherwise legitimate actions, because they ASSUME everything will be done with or through them. 2 great examples come to mind from the Ruby world: Formtastic, and Devise.
I dumped Formtastic because it insisted on genera
Re: (Score:2)
Re:About time... (Score:5, Interesting)
What do you do when some complex, external code has a bug ?
Re:About time... (Score:4, Insightful)
Re:About time... (Score:4, Insightful)
And then it gets fixed right away ?
Re: (Score:2)
And then it gets fixed right away ?
Depends on the product and vendor (using the term to mean any software source, not just commercial ones). Some vendors will ignore you, some will abuse/riducule you, some will put it on the list for next quarter's release and some rare precious few will personally deliver an overnight repair.
At least in open-source, if they don't you can try and fix it yourself.
Re: (Score:2)
*ding*
> At least in open-source, if they don't you can try and fix it yourself.
I've had *excellent* support from OSS projects. I've had lead developers personally upload patched builds to my FTP server.
But I've fixed bugs for myself more. Even having the source is often enough to pin down the problem without havign to patch it.
That's why it's my policy that all things being equal (or even a little unequal) to choose open-source components.
Re: (Score:2)
Most non-open source for embedded systems still comes with source code. It's almost mandatory because you will have to fix it eventually, and they sometimes won't supply object files in a format you can use.
Re: (Score:2)
Depends on the product and vendor
And the customer -- when large enough, they can sway a vendor's maintenance priorities.
Re: (Score:2)
No, probably 6 months to a year before they tell you that they can't reproduce it and close your ticket.
Re:About time... (Score:5, Insightful)
Oh and when I say they work great, I was kind of lying. I have a favorite example. A while back a developer I was working with wrote some Spring/Hibernate code to pull records in from the database and print a billing report. Soon as he handed it off to me, I thought "What happens if I throw 100000 records at this?" Well what happened is that it crashed. So I cut the number in half and it still crashed. Down around 30000 records, it started taking about half an hour and THEN crashing.
Turns out he was using the framework to pull all the records from a couple of different tables and doing the join in Java. The SQL join I wrote to test the code took a couple of minutes to run on a million records and returned the correct output. On a hundred thousand it was neighborhood of 10 seconds.
Now the Spring/Hibernate people will be quick to point out that you can edit some XML file or something and make the framework do a join for you, thus solving that problem. And that is true, if you know a fair bit about the framework. And you'd have to know a fair bit about all the other frameworks they used on that project, too. By the time you got done learning all the frameworks they were using to the level of detail where you could actually be that effective with them, you could have written the application they'd built 10 times over.
Fortunately this story has a happy ending. The team ended up deciding to run the original developer's code against the billing database several times a day so that it would never have so many records to process that it would crash, thus solving the problem once and for all!
Re: (Score:3)
Turns out he was using the framework to pull all the records from a couple of different tables and doing the join in Java
Who gave him direct table access?! They should be fired!
Re: (Score:2)
Or is just complex and unfamiliar. The problem with these frameworks is they work great when they work, but you only ever see them working because they've been published with the most trivial example. When you actually start trying to do things with them, you have to know implementation-level details of the framework in order to make it work for you. By the time you've invested all that time, you may as well have written something less generic that actually does what you want.
Wow, I guessed almost immediately that you were talking about Hibernate. No documentation for complex behaviour or advanced features, or where there is documentation, it's out-dated by several versions, sparse, and occasionally plain wrong. The options for sorting out problems are:
Re: (Score:2)
Re: (Score:2, Interesting)
What if the thing you write that substitutes for the missing external dependency has ten thousand times as many bugs?
People always underestimate the investment in software development. I'm sure most people think someone pulls up a chair, types some code, puts it on a floppy disk and KAPOW it's on the market in every device. Nope. We have testing teams, build specialists, cross-platform planning, packaging requirements, release processes, A/B testing, refactoring, branching, madness.
Any time you write a piec
Re: (Score:1)
Re: (Score:2)
You track it down an fix it which still takes less time that writing the whole complex thing yourself from scratch (where you'd have even more bugs to track down). Then you can submit the fix back to them.
That said, if you start seeing more than a few bugs it's probably better to cut your losses and drop that third party framework. I've done that a few times.
Re: (Score:2)
Re: (Score:3)
I've seen this. Most coders just glue third party stuff together with a bit of business logic. Nobody ever writes anything from bare metal, no complex algorithms, nothing. It just takes too long to test your own stuff. The Java world is so rich in libraries that you can always track down something that does what you need.
And as a certifiable.. I mean certified tester, I'm the person who should be testing shit (and lets face it - Too much of the code out there IS shit)
Let the coders code, Find a person who understands HOW to test and let them test.
Also. If you're going to code using a language like JAVA, Write a freaking JUnit test and pass in Min, Max, Max and a mid range value to your methods/objects. Make 100% certain that it handles the data and errors that inevitably WILL happen. Then you have a quick way to REGRESSI
Re: (Score:1)
A programmer who hasn't thought enough about the problem to test his/her code almost certainly hasn't thought about it enough to implement it or maintain that implementation either.
Re: (Score:2)
Programmers have to take more responsibility and think holistically about what they are building - and integrate testing to validate their assumptions against the hard light of the real world. To be a great programmer, you should know how to test and build tests and test rigs as needed. To be a great tester, you should know how to code - so you can automate what you're testing. I think the lines have to blur - a firewall between the two only leads to silos, and limits what can be done if they were to wor
Re: (Score:2)
JAVA
It's a small point, but a pet-peeve of mine: the language is not, and to my knowledge never was, called "JAVA".
It's "Java".
Re: (Score:2)
Re: (Score:2)
It might not have been too bad to go through and make sure it was just passing everything it used, but it was a lot of code and it kind of all needed to be changed at the same time.
I say this as someone who is generally sold on TDD being the best approach. At first it seem tedious never being able to write more than an handful of code line before having to stop and write a test, but the ultimate freedom it gives you to fearlessly refactor is worth it.
On the other hand I would never (have learned the lessons of trying) attempt to go back and create tests for a software project like the one you describe; and as a general rule anything substantial which does not have them.
It sounds like
Re: (Score:2)
Re: (Score:2)
Re:About time... (Score:4, Interesting)
I've told this story elsewhere, but it applies directly to this issue, so I'll recap in short:
Vender is contracted to create an integrated support application for large sums of money ($millions) over a 6 month period; contractor chooses an obscure commercial java framework to build the system on. The application is delivered and appears to work fine for several months, then starts getting sluggish, then a month later the application locks up - and has to be restarted. This progressively gets worse, and is asymptotic with the growth of the underlying customer base - and soon becomes completely useless - shutting down within minutes of being started with a memory exhaustion error.
The main problem we found was the equivalent of a memory leak in Java. The code would instantiate objects based upon the framework in the main loop, and they would never go out of scope. Furthermore the code imported hundreds of libraries that were never used - further impacting clarity and understanding of what the thing was doing.
To make a long story short, since this was already in production and now there was even more pressure to get a solution in place fast (and all the lawyers threats in the world can't replace a knowledgeable developer) - we rebuilt the whole system using perl in a little over 1 week. That solution is still running today - even as we've scaled orders of magnitude since then.
So - to your point - this stuff really does happen, and wastes godawful amounts of time and money, when a more simpler home grown solution would do just as well, if not better.
Re: (Score:3)
for large sums of money ($millions) over a 6 month period
...
rebuilt the whole system using perl in a little over 1 week
And I thought I was bad at estimates..
Re: (Score:2)
I know this post is supposed to be considered a critisism, but I'm not seeing it. Its exactly what should be done. Minimize the amount of work necessary to complete your work. I am the laziest programmer on earth, and if I can save an hour by dropping in a well tested cleanly interfaced library that meets my requirements, I'm going to do it.
"Nobody ever writes anything from bare metal, no complex algorithms, nothing"
No, people don't re-invent the wheel that already exists because we are too busy doing work
Re: (Score:2)
Some of us aspire to a bit more than shuffling tasks from our in-box to our out-box. Some of us want to produce functional, resilient, maintainable code. Some of us want to actually understand how it works, not just trust that it works. Some of us want to write code, not the compiled equivalent of shell scripts that do nothing themselves but pass inputs and outputs between different external blobs.
I'm paid to build. I don't
Re: (Score:2)
people don't re-invent the wheel
That's the key phrase -- don't reinvent the wheel. But is coding "inventing" or "building"? Because sometimes it's quicker to build your own wheel or whittle your own peg than go out and hunt down one the right size for your needs.
Sociological problem: CYA (Score:5, Insightful)
Part of the problem is the CYA issue.
If you're writing the code, you sound like a laborer ("I have to..."). If it breaks, it's your fault and you're on the hook publicly.
If you present a third-party component in a meeting, you sound like a manager ("I propose that we..."). Once three or four other people in the meeting have concurred, if something breaks it's the third party's fault. A ticket or two are initiated, it's someone else's problem and everybody gets to cast blame somewhere beyond the walls of the company.
Rational behavior, regrettably.
Re: (Score:2)
I would not consider being overly risk averse as being rational behavior.
There are many rational reasons to take risks:
1. Gives you, and by extension your company the opportunity to learn and grow. If you never take risks you stagnate and learn nothing.
2. Real invention occurs through taking risks. If you never take risks you don't innovate.
3. Taking responsibility, and therefore risk is what men and women do. Being overly risk averse is immature slug-like, weasel word behavior.
If your company doe
Re: (Score:2)
if something breaks it's the third party's fault
Until its something critical and then you're back on the hook to fix it -- except you can't fix it because its not your code and you end up making more excuses than progress until either the vendor gets their shit together or you get canned.
Also, your quotes are really badly contexted. You can "propose that we" build a product in house, and you can also "have to" integrate with a third-party library.
Re:About time... (Score:5, Insightful)
It's the penny-wise pound-foolish issue when engineers and developers are forced to be mindful of schedules and business objectives. We in essence become as brain damanged as our managers, and start behaving irrationally.
It's well known we have absolutely no capacity to estimate schedule accurately, but we do have the "gut feel". If your gut says that it will take a day to implement functionality, assume it's a week and just do it, it's trivial. If it says 2 weeks, it's actually 2 months plus three squirrel years and a llama month divided by e^-jwt, maybe spend a day or two evaluating options. If it feels like 6 months, try very hard to find something OTS, because this may become the project you're working on when you retire (which in todays parlance means: you die in your swivel chair of old age).
You could of course be entirely wrong. Last week in fact I decided modifying a script to do what I need would take at least a month, had an epiphany in the shower and had it done in 6 hours. Guts have failed even Homer Simpson. But unless someone comes along who has been-there-done-that with a better option, and who demonstrates he' serious by NOT trying to railroad you in a meeting, but in fact just walks by the cube and says "hey, use this", you're usually better off trying it out yourself. At worst you waste some time but learn the problem, and how to best evaluate other solutions that come up.
Re: (Score:3)
Nothing is better than your own code. But given the choice between my predecessor's hairy ball of custom code and a hairy ball of clue between documented frameworks, I'm not so sure anymore. Because the other side to being generic is "will probably continue to function in a sane fashion if I tweak it a little" while one-off code tends to make a lot of assumptions that may have been true when it was written but falls apart in surprising ways when you try to change it. Unless your predecessor actually made cl
Re: (Score:1)
See, this right here is the thing. Sure, you can implement a library yourself. Sure, your implementation might be more closely fitted with the problem you need to solve. But is your code cleaner? Is it more extensible? Is it well-documented? If you have all those things, you can *begin* to compete with a framework that does the same thing. The framework still has a major advantage: people outside of *you* can come into the project knowing the framework. Everyone will have to learn the thing you wrote from s
Re: (Score:2)
Nothing is better than your own code.
Go review code you wrote 2 years ago. Its almost certain your code isn't all that great either when you get to look at it from afar. But like most forms of filth in the world, you don't really notice the smell while you're stewing in it.
Re: (Score:2)
It feels like this is worst in the Java (enterprise) community, but that could be my imagination. Sometimes I think those programmers need their 3rd party instantiation taken away from them....
I once had someone rip out a stream copy I'd written (int r; byte[] buf=new byte[1024]; while((r=in.read(buf)) >= 0) { out.write(buf,0,r); }) and replace it with a third-party library, because "we shouldn't reinvent the wheel."
Granted, I sort of agree, it's ridiculous that such a common thing isn't part of the standard Java library, but it isn't, and we didn't really need to add another 1MB of library dependencies just to do that...
Re: (Score:2)
Meh, having the same number of code lines (highly dubious) then realizing that you implemented something wrong is just a waste of time for everyone. Why so quickly dismiss the expertese of the people writing (and maintaining) these libraries who in all likelyhood have much higher expertese in that one area of development? Instead, you write an in-house job that takes significantly longer (even if the LOC -may- comparible) then you realize it doesn't work. Your opinions are to bite the bullet and replace usi
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Well there must be a balance between code reuse and custom solutions. To use the trusty car analogy: a car manufacturer doesn't create a new battery for every vehicle (or, indeed, make batteries at all). Unless it's top end, they don't create a different engine for every car, or a different transmission.
If you're making top-end software, then sure, spare no expense. But most projects will suffice just fine using existing libraries. Knowing when to go third party and when to stay in-house is a skillset t
Quality of the solution. (Score:5, Insightful)
It comes down to the quality of the solution, versus the added headaches of managing dependencies. It's not a religious issue, but one of experience and engineering. It's also a good idea to make sure that if you're inventing something that someone else might eventually provide, you at least loosely couple it, so it can be swapped out later.
With those thoughts in mind, don't get stuck in analysis paralysis. Use judgment and move on.
Re: (Score:2)
whatever happened to the time when code stood on its own with the ONLY dependency being an interpreter?
Or am I just that OLD??
Re: (Score:2)
whatever happened to the time when code stood on its own with the ONLY dependency being an interpreter?
Or am I just that OLD??
Whatever happened to the time when code stood on its own with the ONLY dependency being the underlying CPU and OS? And yes, we're both getting old(er) - but that's better than the alternative.
Re: (Score:3)
An OS? God, you're such a young lazy punk. Get the hell off my lawn!
Re: (Score:2)
In modern development one can go through an entire project without using any of the language's standard libraries or even, in extreme cases, the language itself. If you ever want to see a frightening example of this gone too far, take a look at over uses of Apache Camel or even Spring, where you can write entire systems using code embedded in XML files and that weird 'conditional logic as constructor arguments' camel allows for.
Re:Quality of the solution. (Score:4, Interesting)
Do you want to keep up with the world around you?
I was having a discussion with a programmer who is 50 (I'm 38) the other day about the time when you used to be able to write an OS entirely by yourself, and how we both miss that time. It wasn't trivial, but the OS was that small, it could be done. Remember, Linux is just a kernel, not 'an OS', and Linux hasn't been the work of one make for 20 years at this point (Not trying to discount what Linus has done, but he has help :).
Today, if you want to be able to produce useful software with sufficient features for most purposes, you can't write it all yourself. Well you can, but you'll be the last person to bring your 'whatever' to market/public view and they'll be 20 others that have more features and more shiny than you because they relied on some other libraries. A single person isn't making Windows 3.1 or newer in any realisticly useful time frame. Windows 95 is well beyond the scope of one person.
Even a good modern text editor isn't going to be written from scratch, you've just not got the time to write all the basic editor features and things like a regexp engine. Yes, you CAN, but not while staying relevant.
Sure, there are places where you can get by without dependancies or MUST have no dependencies. Embedded work and Cryptography are two things I have experience with where re-implementing the wheel isn't uncommon due to various constraints placed on you. Note: Not reinventing, reimplement
With that said ... I'm writing an OS by myself. It will never do anything impressive and no one will ever use it, but it will exist and has already taught me why I hate the x86 processor line in about 18,000 different ways :)
Re: (Score:3)
It will never do anything impressive and no one will ever use it, but it will exist and has already taught me why I hate the x86 processor line in about 18,000 different ways :)
it worries me somewhat that you hadn't figured out that last bit before you started the project, but I wish you luck anyway.
Re: (Score:2)
Re: (Score:2)
AKA "feature creep".
Re: (Score:2)
Re: (Score:2)
This is why I tend to point to Java's dependency injection as an example of the pendulum swinging too far the other way. Much of it was developed as a reaction to what w
Lockdown (Score:3)
At some point we need to just say, 'stop!', and write the code ourselves.
I wonder how much of "invented here" syndrome is related with frustration with curation on the popular curated platforms (iOS, Windows Phone, Windows RT, and game consoles). Cryptographic lockdown applied by the operating system publisher blocks end users from writing their own applications or writing a mod for an existing application. Because people are unwilling to go through the organizational overhead of becoming a licensed developer, they stick with the vanilla version of whatever they can get from the platform's official app store.
Re: (Score:2)
Just FYI: Side-loading is perfectly possible on WindowsRT and Windows Phone. Doesn't cost a penny and is not too difficult.
Not-Good-Enough Syndrome (Score:5, Insightful)
I'd say part of the cause of "invented-here syndrome" can be "not-good-enough syndrome." I'm often comparing my programming skills to people I see online - people whose skills far outpace my own. So when it comes time to access my programming skills, I'll understate how good I am because I'm simply not as good as those "coding superstars." Of course, when you see the online results of code people have written, you don't see the idiotic mistakes they made, the typos they've had to correct, the hours they spent Googling for an answer to a pesky problem. You just see some elegant, amazing looking code. It can be a daily struggle to balance admiring the programming skills of others without trying to compare myself to them (and thus knocking my own skills).
Re:Not-Good-Enough Syndrome (Score:5, Insightful)
Sounds like you've never seen what passes for production code in the bowels of a major corporation. Look at the questions on experts-exchange or stackoverflow. You can safely assume most of them are for paid work. After that, you shouldn't have self esteem issues anymore.
Re: (Score:2)
That's what I keep reminding myself. There's always going to be people better than you and worse than you no matter where you work. The trick is not simply focusing on the people better than you and judging yourself harshly because of this limited view of the talent of others.
Re: (Score:2)
No problem, I have a solution to that. (Score:2)
I'd say part of the cause of "invented-here syndrome" can be "not-good-enough syndrome." I'm often comparing my programming skills to people I see online - people whose skills far outpace my own. So when it comes time to access my programming skills, I'll understate how good I am because I'm simply not as good as those "coding superstars."
I have a solution to that exact problem. Just download any of the most popular web CMSes built in PHP. Wordpress, Drupal, Joomla, or, if you think you're a tough one, Typo
Re: (Score:2)
I remember when I initially began to study programming beyond the introductory console programs nearly all STEM majors learn to write. I sought out communities to participate in because I believed that learning a
Re: (Score:2)
Re: (Score:2)
Developing Skills (Score:2)
While there's definitely a business case to be made for using existing solutions, if you'
Should that be "Invented There syndrome?" (Score:2)
If you're looking for a pre-existing library, then it wasn't written here, it was written there. This should be called invented there syndrome.
the best part of my job (Score:2)
Take idiots that put "GMT" or "MST" at the end of a time when our system (and most others) can't handle that crap.
The data was created by someone else. They most likely had the ability to export it in the correct way, but were too .... inexperienced .... to do it the right way.
Instead of trying to teach/convince them to do it right, I simply write a script to fix whatever bit of stupidity they created.
All the time (Score:1)
I've ran into this ALLL THE TIME! There is some perception that if it's done elsewhere it's good, and if it's done internally it's crap. This exists in not only management but in coworkers. I try to explain all the time the advantages of in-house software
1. Does exactly what we want, how we want it
2. Does it now
3. We update on our own terms, we are not forced to migrate when external migrates
It is a huge waste of time to have to update tons of things internally to work with some refactoring done externally
Write it myself (Score:2)
Honestly I think my problem is that I tend to write my own code for stuff rather than trying to figure out what someone else has done and fit my thought processes into how they were working it through.
Examples:
I wrote a status management program to keep track of the work I do for yearly reviews. Tracks projects, hours, and even builds a timecard.
I wrote an inventory management program to keep track of our systems. We have an official asset tracking tool but it doesn't accommodate virtual machines, I can't s
Re: (Score:2)
We need to address the real underlying problem you are describing right there - code written by different people that does not conform to any standards is hard to manage over its lifecycle - and this goes double for limited frameworks that may get some things right, at the expense of not allowing you to get all things right.
This is one thing that open source has gotten right on occasion - think of the Linux kernel for example, and how many people contribute to that and keep it going.
So really the answe
Re: (Score:2)
Actually I read through several sites a couple of years back looking for templates and ideas on how to implement standards for my code. I created a couple of web pages. One for a Coding Standard page and more recently, a User Interface standard page. I've been pulling out company identifying information and have a Sourceforge site to upload it.
[John]
Often easier to write yourself (Score:2)
I will usually spend an hour or so looking for an existing solution if I expect it to take only a couple days to write the code. Even when I expect it to take a week or two I won't spend more than a couple hours looking around. The problem is that even if I find code that does pretty much what I want to do, it will usually take a day or two of screwing around with it to figure out how to make some required part of it work in my overall project. Maybe 95% will be just the way I need it but there seems to
Solution - Developers need to know code they call (Score:3)
I've seen the end result of this a lot working in systems integration and engineering. The problem is that, yes, most functionality has been written in some library or available through a public API, web service, whatever. Especially with mobile stuff, Apple/Google/Microsoft give the developer huge amounts of pre-built functionality, and encourage its use.
The overall problems with it are, in my mind,
- Developers and systems people not knowing how that huge chunk of functionality they use actually does what it does
- Introducing dependencies on third party applications which may or may not be around later, have spotty support, etc.
- Making applications more complex to deploy and debug -- "is this my bug or the API's bug? Why is a single row database update taking 45 seconds and 100% of a CPU core?"
The reverse problem on the other hand has the potential to be worse. No one should be coding core functionality that has the potential to fail spectacularly or have major security problems. Examples might be writing your own PKI stack instead of relying on the OS/webserver to do it, designing your own file transfer protocols unless you have a _really_ good reason, and many more. So with NIH syndrome, you have to really trust that your developers did everything right. With IH syndrome, you need to install an application, plus the 45 modules it depends on, plus provide it access to public APIs, etc.
I think the "solution", even though there's no right answer for all situations, is to make sure app developers are actually understanding development. It's too easy to write applications by gluing together pieces. With the framework movement, the pieces are much bigger and hide way more from the developer than, say, a library function.
From my side, in systems, we have way too many admins who are scared of scripting. Windows installations are moving towards PowerShell now, and while very useful, PS hides almost everything from the end user. Scripts that used to be 100 lines of loading/parsing/checking code are reduced to a single call to a chain of cmdlets. Very powerful, but the language itself isn't the most intuitive out there and borrows syntax from many languages. This leads to admins finding something on StackOverflow and copying/using it unmodified and unverified, simply because they don't know what it actually does.
The Level of Abstraction (Score:5, Interesting)
You will always have to write some code of your own. Even if you use a CodeIgniter, AngularJS, and every prewritten function on StackExchange, still, you will have to write some code to configure the frameworks and to pull it all together.
You will always use some of someone else's code, too. Aren't you using Linux or something? You didn't write your own OS, did you? You're using a database, like MySQL or PostgreSQL or something, right? You didn't write your own database system, did you? And are you using a web server like Apache or Nginx?
So the question is not, should I write my own code or use someone else's. The question is where to draw the line.
I'm a web programmer, your typical LAMP developer (well, LAPP --- I use PostgreSQL). Like many PHP programmers, I first concentrated too much on the PHP. PHP is not the best language, as many have said, but I don't think it's quite as bad as people make it out to be. Anyway, I never took up any of the PHP frameworks. They seemed like too much trouble to adapt. (I should point out that I started with an intranet with a dozen or so applications already built.) I would research PHP frameworks from time to time, but always rejected them all, and felt a little self-doubt in doing so: "Do I suffer from Not-Invented-Here Syndrome?"
But PHP, and scripting languages in general, provide the right level of abstraction, I think. It takes care of memory management. It provides a bunch of functions for HTTP. It has its own templating syntax [ellislab.com]. It's great if you don't overuse it. In other words, in the MVC pattern, PHP does great for the View and, together with Apache, the Controller. But if you write a lot of your Model in PHP, with all this data processing, checking, calculations, etc. --- well, that's what the database is for, I think.
So, instead of eventually adopting a PHP framework, I learned more and more about Apache and PostgreSQL, and I learned that a lot of the things that I was doing in PHP could be done in SQL or in the Apache configuration, with a lot less typing (a lot more reading but a lot less typing). While most people are busy trying figure out how to write the practically all of the MVC in PHP, I realized that Apache was part of the Controller, PostgreSQL was part of the Model, and the browser was part of the View. I use PHP just to help them out, only when needed.
Re: (Score:2)
I am happy for your enlightenment! It amazes me how many people don't get this, and want to write a PHP "Front Controller" that uses a querystring parameter to select the "page", and then slap a mod_rewrite rule on top to clean up the URL. Instead of—as it sounds like you discovered—putting each page in its own file in the filesystem and letting Apache "route" the request.
Re: (Score:2)
It amazes me how many people don't get this, and want to write a PHP "Front Controller" that uses a querystring parameter to select the "page", and then slap a mod_rewrite rule on top to clean up the URL. Instead of—as it sounds like you discovered—putting each page in its own file in the filesystem and letting Apache "route" the request.
Right. I split the program into more files, instead of mod-rewriting everything through a gigantic PHP controller. Of course any database-backed app must eventually resort to the query string, PATH_INFO, or something. I usually use those to specify just the record ID or search filter, not an app section. Each section gets its own file.
Management problem (Score:2)
Largely, that is.
My time to write a function is accounted entirely differently than the expense to acquire a commercial package that does the same. The bias is to prefer something that can be purchased over that which is written in-house.
Don't even get me started on free software. Management values an application based on how much they paid for it, either in developer-hours or from the expense budget. Stuff that costs zero must be worth zero. Never mind what functions it actually provides.
A many-headed cultural problem (Score:2)
Thanks for bringing this up and giving it a name, first off; it needed one.
Over the years I have seen this mentality a lot, but it's almost always pushed onto a team from management rather than home-grown in the minds of the engineers. I see it as a systemic issue at software companies that the leaders see their own engineers as the least preferable choice to actually build new things. I think this arises from a few factors:
- open-source software has so successfully marketed itself as better than third-part
It's a myth (Score:2)
"Invented here" is somewhat a illusion.
Working with frameworks and other people's code is a engineering problem (design, integration, etc..)
Developing code is a computer programming problem.
Two different problems, and why Software Engineerings are [desired] unicorns in very large software projects.
Re: (Score:2)
Because "re-use" is self-congratulatory nonsense (Score:2)
"I found code that already does this" sounds like you're a better problem solver. It's self-congratulatory nonsense. If the result is very complex, you congratulate yourself even more, for being able to do complex things, rather than punching yourself in the head like you should, for making things more difficult than they need to be.
If you found code that really does make your life easier, that's great. It hardly ever happens.
Or, "Their invention must be right" syndrome. (Score:2)
I see that one a lot. The assumption that because the software vendor did it one way, That's. The. Way. It's. Done.
It's nonsense of course. Our testing system was designed for the most common use case. Maybe 50 dialogs with 5 or 6 controls each. Our system has thousands of dialogs, some of which have as many as 50 controls on a dialog (It's old, legacy, badly designed. I know....). To make that system scale, we had to develop our own abstraction system, an API and a different object mapping system.
In the e
The black box is a trap (Score:2)
The problem with black-box programming is that it's a trap. Far more often than anyone cares to admit, the black box implements functionality in an unreliable or inefficient manner. When you're dealing with code that you wrote yourself, you can correct that behaviour of the "grey" box. But with a third-party black box, all you can do is file a bug report and hope that someone can not only replicate the problem, but that they'll give it high enough priority to fix it before you retire or your project is
This describes modern web development (Score:2)
The word "framework" is often a big read flag. (Score:2)
Re: (Score:2)
you're missing the Point. The point is: Do not RE-invent the wheel.
Re: (Score:3)
...unless you have to redesign your car to make it fit.
I think the point of the article was that that's sometimes the case.
Re: (Score:2)
But then it's totally the point.
Do invent the wheel, if you need it to build a car.
Do not RE-invent the wheel, just because you built a car without wheels and now the usual wheel does not fit. Build your car in a way, it works with usual wheels.
Re: (Score:2)