Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 



Forgot your password?
typodupeerror
×
Programming Bug

'Here Be Dragons': The Seven Most Vexing Problems In Programming (infoworld.com) 497

InfoWorld has identified "seven of the gnarliest corners of the programming world," which Slashdot reader snydeq describes as "worthy of large markers reading, 'Here be dragons.'" Some examples:
  • Multithreading. "It sounded like a good idea," according to the article, but it just leads to a myriad of thread-managing tools, and "When they don't work, it's pure chaos. The data doesn't make sense. The columns don't add up. Money disappears from accounts with a poof. It's all bits in memory. And good luck trying to pin down any of it..."
  • NP-complete problems. "Everyone runs with fear from these problems because they're the perfect example of one of the biggest bogeymen in Silicon Valley: algorithms that won't scale."

The other dangerous corners include closures, security, encryption, and identity management, as well as that moment "when the machine runs out of RAM." What else needs to be on a definitive list of the most dangerous "gotchas" in professional programming?


This discussion has been archived. No new comments can be posted.

'Here Be Dragons': The Seven Most Vexing Problems In Programming

Comments Filter:
  • by NotSoHeavyD3 ( 1400425 ) on Saturday November 12, 2016 @05:36PM (#53272881) Journal
    1 who's my customer 2 What does he or she actually want.
    • Re: (Score:2, Insightful)

      by OrangeTide ( 124937 )

      Cowboy programmers really hate that.

    • by mark-t ( 151149 ) <markt AT nerdflat DOT com> on Saturday November 12, 2016 @05:48PM (#53272943) Journal
      I'd like to second that #2 problem you mentioned. Easily the most annoying problem in programming I face is functional requirements not being fully specced out before the project begins and discovering well into it that I have to redo a month's worth of work when the requirements exceed the demands of how the code was originally written. In most cases, time is usually given for these changes, but sometime's it's not... and man, are those times infuriating.
      • by Anonymous Coward on Saturday November 12, 2016 @05:56PM (#53272981)

        Haha, you should be more Agile! It's not Agile to find out what your customers need first. Just throw a bunch of shit at the wall, make it flatter, and dump it into production to let your customers do your testing and chip away at the things they complain about.

        They'll definitely give you useful feedback and not switch to another software product instead. If there's one thing that's always been true about customers, it's that they're fiercely loyal to their software vendors.

      • by Anonymous Coward

        Reminds me of my last job.

        Boss: this isn't what I wanted, did you read the spec?
        Me: you gave me a ticket that had 3 sentences in it. That's not a spec.
        Boss: that is too a spec. Go back and read it.
        Me: this leaves a ton open to interpretation. This isn't a spec. You need to clarify this with a lot more detail.
        Boss: come on, you knew what I meant.

        And that's partially why I quit.

        • by lgw ( 121541 ) on Sunday November 13, 2016 @12:02AM (#53274247) Journal

          Reminds me of my last job.

          Boss: this isn't what I wanted, did you read the spec?
          Me: you gave me a ticket that had 3 sentences in it. That's not a spec.
          Boss: that is too a spec. Go back and read it.
          Me: this leaves a ton open to interpretation. This isn't a spec. You need to clarify this with a lot more detail.
          Boss: come on, you knew what I meant.

          And that's partially why I quit.

          So you didn't ask your boss what he actually wanted before you started coding? You didn't ask for clarification for each ambiguity you discovered during coding? You just shat something out at random?

          • by Dog-Cow ( 21281 )

            I've worked with lots of people like that boss. If you try and get more detail, you're chided for not working quickly enough, or for bothering them. My policy is always to give such people what they ask for. If they can't or won't learn, that's not my problem. Their spouse/parents/pet can change them. It's not my job.

      • Comment removed (Score:5, Interesting)

        by account_deleted ( 4530225 ) on Saturday November 12, 2016 @09:08PM (#53273743)
        Comment removed based on user account deletion
    • by zwarte piet ( 1023413 ) on Saturday November 12, 2016 @06:06PM (#53273037)
      3 : It has to be ready 2 weeks ago so we'll forward that untested alpha version to our high risk clients who will sue the fuck out of us if anything is not right. 4: Oh can you make a feature that does X for client A. I'll be here for another hour. 5: assumes input from external program is correct by glancing over the xml of 1 example record. 6: we will make that split between a development version and a stable version when all the features on the list are done. (read in 4 years) We want the new features out as quickly as possible. etc....
  • node (Score:5, Insightful)

    by MichaelSmith ( 789609 ) on Saturday November 12, 2016 @05:42PM (#53272911) Homepage Journal

    Oh christ if I get one more node.js developer claim that node is multi threaded...

  • 2 more I've seen (Score:5, Interesting)

    by NotSoHeavyD3 ( 1400425 ) on Saturday November 12, 2016 @05:45PM (#53272925) Journal
    1 Not being careful with floats. (Those can totally bite you including using floats when you should have used an int/uint type)

    2 Developers who decide to reinvent the wheel because "They know best". (Just dealing with code where somebody decided I don't want to use the built in stuff, I'm going to do my own date time stuff which constantly has issue. Makes you pull your hair out.)

    • by ATMAvatar ( 648864 ) on Saturday November 12, 2016 @06:06PM (#53273035) Journal

      1 Not being careful with floats. (Those can totally bite you including using floats when you should have used an int/uint type)

      Some developers, when encountering a problem, say "I know, I'll use floating-point numbers!" Now, they have 1.9999999997 problems.

    • by AmiMoJo ( 196126 )

      It's depressing how much code stores coordinates in floating point format.

  • Closures? (Score:2, Insightful)

    by Anonymous Coward

    C'mon. Threads -- I concur wholeheartedly. But closures? Granted, you have to wrap your head around them, but then they are a docile workhorse.

    As for threads... don't unless you have to. You add the benefits of concurrence and common address space and multiply their curses.

    • Re:Closures? (Score:4, Informative)

      by K. S. Kyosuke ( 729550 ) on Saturday November 12, 2016 @06:27PM (#53273117)
      Not only that; closures are not a problem but actually the ultimate solution to the actual problem of name scoping. Lots of approaches were tried before closures remained as the only sane idea.
      • by hey! ( 33014 )

        I think the problem is the paradigm shift from object oriented to functional. I learned to program in Scheme many, many years ago (early 80s), although I never used it professionally. When javascript came along, people looked at the bizarre object system and thought "This is dumbed down Java." I looked at it and thought, "This is kind of like Scheme."

    • Re:Closures? (Score:4, Interesting)

      by lgw ( 121541 ) on Saturday November 12, 2016 @07:24PM (#53273349) Journal

      C'mon. Threads -- I concur wholeheartedly

      I've never understood why programmers have problems with threads. Almost everything is one of two models:
      * Solve the problem as a series of queues with worker thread pools. Really hard to mess that up.
      * Entire request-to-response workflow in a thread for each request. You have to be a bit careful about locking around shared objects, but usually the answer is avoiding shared objects.

      If you can't get multiple threads on one machine right, how are you going to work on distributed systems?

  • by Harold Halloway ( 1047486 ) on Saturday November 12, 2016 @05:47PM (#53272937)

    Buffers seems to cause a lot problems, particular when they overflow. As a non-programmer (well, a very long time ago programmer) I've never grasped why preventing overflows seems to be so difficult.

    • Re:Buffers (Score:5, Interesting)

      by MichaelSmith ( 789609 ) on Saturday November 12, 2016 @06:11PM (#53273055) Homepage Journal

      Saw this in C:

      some_func(char *data) {
              char *buffer;
              buffer = malloc(200);
              memcpy(buffer, data, 2000);
              send_buffer(buffer);
              free(buffer);
      }

      Occasionally it would cause a crash.

    • Re:Buffers (Score:4, Interesting)

      by Cassini2 ( 956052 ) on Saturday November 12, 2016 @06:15PM (#53273073)

      Microsoft wrote a bunch of code, that in hindsight was a really really bad idea. I'm not really sure that the problems were all Microsoft's fault - they probably didn't invent them. They are Microsoft's fault in that they embedded the concepts into the operating system and then widely popularized them.

      The big gotchas all have to do with limitations of C, and the twists used to optimize Microsoft's programming for the 8086 architecture. I'm thinking of:

      char string[MAX_PATH];
      What can possibly go wrong? Especially before MAX_PATH was invented.

      malloc() new() and friends
      For any non-trivial program, it's almost impossible to get correct. They only really work for something like a C compiler which will allocate a variable amount of memory, do a task, and then end. If you miss a few free() calls along the way - they will be cleaned up when the compiler terminates.

      GlobalAlloc(), LocalAlloc() and friends
      Just in case you couldn't make it work with malloc, try the operating system version of the call. Using OS calls really opened the window to some famous bugs - OpenSSH comes to mind.

      Multi-threading in C
      For any non-trivial problem - it doesn't work. Firstly, for any non-simple piece of code, it is tough to do correctly. Secondly, for a trivial piece of code, like a save operation, you need to somehow make the memory being stored immutable for the duration of the save. For most save operations, this defeats the purpose, as the first thing the user wants to do after save is to change the document. Thirdly, if you really need multi-threading, there is a good chance some of your users need parallel-processing across machines. Multi-threaded code and parallel-processing code are not the same things at all.

      Embedding code in data.
      Firstly, the x86 architecture encouraged this, by not having proper page protections. Secondly, it completely opens the system up to malicious code. ActiveX, JPEG libraries, font libraries, everywhere Microsoft had an API that embedded code in data, it was all exploited.

      • char string[MAX_PATH];
        What can possibly go wrong? Especially before MAX_PATH was invented.

        Start your path with \\?\ and see how MAX_PATH/PATH_MAX works.

        That's why Hurd guys had the right idea to remove these whatsoever, causing lots of buggy software to fail to build instead of having crashes, often exploitable, at runtime.

    • by AmiMoJo ( 196126 )

      Buffer overflows are often the result of not checking the data that is going into the buffer carefully enough, or at all. The spec says the string is max 80 characters, so someone sends 80 characters and some exploit code which runs over the end of the buffer.

  • by rastos1 ( 601318 ) on Saturday November 12, 2016 @05:50PM (#53272949)

    As far as I know, there are four hard problems in programming:

    1. caching
    2. naming (i.e. how do I name that variable/method/class)
    3. off-by-one errors

    • by mbone ( 558574 ) on Saturday November 12, 2016 @06:25PM (#53273107)

      As far as I know, there are four hard problems in programming:

      1. caching

      2. naming (i.e. how do I name that variable/method/class)

      3. off-by-one errors

      Or, as old Fortran programmers would put it, insisting that the first item in a list have an index of 0.

      • by Sique ( 173459 ) on Sunday November 13, 2016 @04:03AM (#53274845) Homepage
        Off-by-one errors are not solved by having the first item in a list indexed by 1. You still have the mathematical feature that all natural numbers have an ordinality and a cardinality. If you insist, that the ordinality of a number equals the cardinality, you have to start with 0 as the first index.

        (For people not much into set theory: The cardinality is the answer to "how many elements are in a given set?", and the ordinality answers "how many predecessors does the number have?")

    • by AmiMoJo ( 196126 ) on Saturday November 12, 2016 @06:46PM (#53273221) Homepage Journal

      The best but if advice I've ever had for variable naming is to put the units in the name. If you are handling values in millimetres, use a _mm suffix. For area use _mm2. For graphics use _px for pixels etc.

      I've seen so many programmers get hopelessly confused because they can't remember what units they have when doing maths or trying to render data on screen. On some systems pointers can use byte or word addressing too.

      • by bidule ( 173941 )

        Ah, true hungarian notation. The other use case being suffixes such as _raw vs _validated.

      • by lgw ( 121541 )

        Pigs are flying, Hell is frozen solid, and I agree with AmiMoJo. Putting units in your names is the best advice. (But I can still argue with you - you "_" using heretic! Eeeeeeevil!)

      • by Xrikcus ( 207545 ) on Saturday November 12, 2016 @07:33PM (#53273383)

        Or, even better, put them in the type. This is one thing that makes std::chrono hard to use incorrectly.

      • The best but if advice I've ever had for variable naming is to put the units in the name. If you are handling values in millimetres, use a _mm suffix. For area use _mm2. For graphics use _px for pixels etc.

        That works and has generally been my go-to method.

        An interesting alternative is where the units are part of the type, and I've been getting to the C++ std::chrono library which makes heavy use of it. It's actually rather nice. It models durations and time points as separate concepts, but not only that, it

    • by vtcodger ( 957785 ) on Saturday November 12, 2016 @07:07PM (#53273299)

      How about?:
        4. Race conditions
          Often very hard to spot using static analysis and not guaranteed to show up during testing.

      The bane of embedded and command-control systems.

      • by Plus1Entropy ( 4481723 ) on Saturday November 12, 2016 @08:32PM (#53273611)

        Agreed that race conditions can be very tricky to spot. But there is one case I've run into a couple of times...

        If you're trying to debug something and you start putting in some print statements or breakpoints, and suddenly the bug goes away... It's probably a race condition, which was inadvertently "fixed" by you slowing down the execution with your breakpoints, etc. I like to think of it as the CS equivalent of the Observer Effect.

  • by melted ( 227442 ) on Saturday November 12, 2016 @05:52PM (#53272961) Homepage

    And it's not that hard. Some people make it hard because they're incompetent. The golden rule of multithreading is: your multithreading set-up has to be very simple. Don't optimize (and for the love of god, unless you really understand cache coherence, don't use atomics), aim for clarity, use mutex locks for shared data, and if you have to use several, make sure you acquire them in the same order. 99.9% of the problems disappear if you follow this simple advice.

    • The most important thing is that you shouldn't even be exposed to these primitives in application programming. You just shouldn't. Likewise, the collective amnesia that makes programming people forget useful concepts invented in the past such as pseudo-time doesn't help any. (I'm pretty sure someone will reinvent that one in near future, with great fanfare!)
    • Yes, and if you follow this golden path, what does it give you? Nothing more than asynchronous behaviour. And there are much better and more robust mechanisms to achieve this (such as co-routines, or their virtual equivalent such as async await in C#, message passing, etc etc). What it doesn't give you is any kind of scalability across multiple cores, and the level of complexity is all out of proportion to the gains. Multithreading particularly in combination with OO, has largely failed, because the paradig
    • by buddyglass ( 925859 ) on Saturday November 12, 2016 @06:43PM (#53273207)
      I'm not willing to chalk it up to a lack of education. Concurrency is legitimately more difficult than straightforward single-threaded code. Some developers can handle it; some can't.
    • For example I worked on code where the guy decided he could use a Boolean (yes in C/C++) for thread synchronization. (Just no, please.)
    • Threading is hard because cyclomatic complexity is increased by an order of magnitude as soon as you add a thread. That's so many more test cases you need to add.
      It's manageable, but wise programmers tend to avoid threads when there is another option.
    • I think you mean "memory consistency".

  • by Lisandro ( 799651 ) on Saturday November 12, 2016 @05:54PM (#53272969)

    Seriously, i spend more time thinking on this than anything else lately.

  • by NFN_NLN ( 633283 ) on Saturday November 12, 2016 @06:02PM (#53273013)

    > NP-complete problems. "Everyone runs with fear from these problems because they're the perfect example of one of the biggest bogeymen in Silicon Valley: algorithms that won't scale."

    Customer: Why is this problem not scaling?
    Programmer: It is NP-hard we are basically just brute forcing it.
    Customer: If you can't solve this "NP-hard" problem, then I'll just find someone who will.
    Programmer: Sure, let me know when that happens.

    • > NP-complete problems. "Everyone runs with fear from these problems because they're the perfect example of one of the biggest bogeymen in Silicon Valley: algorithms that won't scale."

      > Customer: Why is this problem not scaling?
      > Programmer: It is NP-hard we are basically just brute forcing it.
      > Customer: If you can't solve this "NP-hard" problem, then I'll just find someone who will.
      > Programmer: Sure, let me know when that happens.

      Customer: GTFO&DIAF
      Programmer: Well, shit.

  • Multithreading's issues in particular are a matter of education, not technology. The rest are related to programming only vaguely; in particular, NP-completeness is mostly a CS thing and very rarely encountered in software engineering.

    • I think in large programs multithreading can be very hard. Not to make it work, and not even to fix once you've got a good handle on the problem at hand, but you never know how much problems are still lurking there are being newly introduced while the code is evolving and each problem is hard to locate. At the same time CPUs are increasing their number of cores forcing you to make use of multithreading.

      I agree I don't see the problem with NP completeness. Normally you're happy with a good but not perfect so

      • See now, that's what you get for your lack of education.

        Prevention of multithread-related bugs is exactly as dead simple as prevention of bugs related memory addressing, or array indexing. The answer is: only do things that have a defined meaning in terms of the concept space of the bug category at hand. So for array indexing, ensure that your index variable is always known to be in a valid range; for memory access, design the program in such a way that for each pointer it's known what they're supposed to p

    • NP-hard problems are easy to run into everyday problems. Its just a matter of identifying them; most people seem unable to.

  • Modern SAT solvers [toronto.edu] are able to solve SAT problems [wikipedia.org] with millions of variables in seconds. Yes, there are hard problems with some hundred variables that are too hard to solve. But as it turned out, most useful problems are easy to solve. So if you have an NP-complete problem, you should just try to put it into minisat [minisat.se]. If it can't be solved easily, there's still time left to despair.

  • dates (Score:5, Insightful)

    by buddyglass ( 925859 ) on Saturday November 12, 2016 @06:42PM (#53273197)
    Dates, including time zones and daylight savings time. Constant source of bugs.
    • by lgw ( 121541 )

      Dates, including time zones and daylight savings time. Constant source of bugs.

      The only safe way to deal with dates/times is to use a 64-bit int for milliseconds (UTC - always UTC). No time zone nonsense, not DST issues, safe to subtract to get a duration, easy to add a duration to, always the right answer.

  • Writing performant code without obscuring the algorithm or introducing CPU / OS dependencies that will break when you port it.

    Writing code that understands cache coherency and is structured for memory locality.

  • Seriously. I've worked in many departments for many different employers and seen many different reporting implementations. All of which have been different. All of which have been terrible. Layers and layers of impenetrable abstraction over some "generic" "engine". Every. Single. Fucking. Time. I swear it's the same person.
  • Ignorance abounds (Score:5, Insightful)

    by IHTFISP ( 859375 ) on Saturday November 12, 2016 @07:28PM (#53273357)

    Closures are not a ``problem''. Ignorant or incompetent implementations of closures is the problem.
    Just as ignorant or incompetent implementations of garbage collection and memory management is a problem.
    And ignorant or incompetent use of arrays/vectors/buffers and pointer arithmetic and access permissions and ...

    So here is an alternative list:
    1. Low-skilled programmers (``script kiddies'') who write profound amounts of buggy code.
    2. Low-skilled language ``designers'' who re-introduce the known bugs of the past and introduce new innovative bugs as well.
    3. Low-skilled managers who reward high output over high quality, thus ensuring an on-time, under-budget hairball and bugs nest.
    4. Low-skilled educators who teach ``coding'' rather than computer science, thus ensuring another generation of the above.
    5. Low-skilled professional organizations who reward and encourage incompetent industry leaders to unduly influence the field.
    6. Low-skilled investors who reward incompetent technology from dominant, monopolistic companies.
    7. Low-skilled consumers who flock to buy flash-in-the-pan shiny stupid gimmicks but won't invest in sound technical innovation unless it's flashy.

    This is why C students should never be allowed to graduate w/ a degree. They only go on to further muck up the world. Color me bitter. ;-)
    P.S. That `C' above refers to grade level / professional competence, not the language (which should really be named D or D minus minus). *smirk*

  • by engineerErrant ( 759650 ) on Saturday November 12, 2016 @10:50PM (#53274063)

    One of the biggest, most vexing problems in computer science is newbies who equate basic programming to the hardest problems of the field.

    OK, I realize this is arcane. Let me give an equivalent situation in, let's say, agriculture.

    Farmer: "OK, young intern, today we'll learn how to water plants!"
    Intern: "Waaaaah! This is as hard as the manipulation of space-time with thought alone!"
    Farmer: "...Grab a goddamned watering can, moron."

  • dumb stuff (Score:5, Interesting)

    by Spazmania ( 174582 ) on Sunday November 13, 2016 @02:29AM (#53274683) Homepage

    Ooh, dragons.

    1. Multithreading. In some languages (e.g. Java) multithreading is no worse than any kind of parallel processing. Can you be a buffoon and fail? Yes you can. But you can do that in a single-threaded program too.

    In other languages (e.g. C) the chance of errors impacting some completely random data structure are so high that it's frankly important to stay away from anything that would confuse the control flow, such as multithreading.

    2. Closures. Okay, you're talking about Javascript here. javascript is a hideous language which should not be used to the maximum extent practical. Why complain about closers when all of javascript is bad mojo?

    3. Too big data. Also known as lazy-ass programmers who don't want to be bothered to deal with the possibility that their program may be called upon to process a large data set. Solutions for processing large data sets explicitly on the disk (not with virtual memory) are well understood. You just have to choose to use them.

    4. NP-complete. There's no particular dragon there... the elephant in the room is those programmers too ignorant or clueless to bother computing the big-oh running time of their algorithms, so they write software that appears to "lock up" as soon as anyone gives it a useful data set to process.

    5. Security. Correct security implements defense in depth. Break one level so what? Even patched slowly, it's patched before enough layers can be broken to grant access.

    Programmers make two classes of stupid mistake:
    a. They assume they don't need defense in depth because they've written secure software.
    b. They carelessly implement functionality that cuts through all layers of security. Like single-sign on.

    6. Encryption. The odds of a long-surviving conspiracy to hide mathematical breakthroughs which bypass encryption are essentially zero. You can relax on that score. You can also expect one-time-pad encryption to remain unbreakable regardless of mathematical improvements. Because no analysis of extremely long strings of random bits is meaningful.

    You have to be pretty bone-headed to mess up security in this day and age. Yes, many many programmers are bone-headed. But that's not encryption's fault.

    7. Identity management. The only thing which makes this hard is its constant misuse. No authentication is perfect, so correct programming makes authenticated actions reversible instead. The more critical the action, the more critical it be reversible.

    8. Measuring hardness. Now we're in to mumbo-jumbo territory. A simple big-oh evaluation will tell you whether your algorithm is usable. Trying to figure out whether it will be easier or harder to solve one problem or another, neither of which have a known useful algorithm? Silliness! Problems like that are solved when someone finally has the key insight. Inspiration is not a product of work, it's a product of the creative mind.

    9. The authors can't measure. They claimed 7 but listed 8 things.

  • by Anonymous Coward on Sunday November 13, 2016 @02:32AM (#53274695)

    They missed the most vexing problem of all. Hiring a skilled and talented old timer programmer to come solve the problems that fresh faced recruit coders created. Then offering the same rate as they paid the fresh faced recruits, not being able to grasp that the ability to solve, often under tightest of all deadlines, messes left by inexperienced programmers. In some cases less! I have personally had the extreme joy to explain to a physicist that the code he wrote, while it would solve his problem, was written in a manner that the solution was of order O n squared. While the problem could be solved in an manner that mathematically provided the same solution but was order O n. The physicist then said he wanted to test both solutions. His was actually faster on a data set of 100 items. (Lots of fixed constant cost was involved, my solution had more precalculation setup, so for small sets of data, my overhead was more) He proclaimed himself the winner deriding my science. I asked (already knowing) what the normal sample size was they analyzed? What, 100,000 samples? Sure let's use one of those... 20 minutes later mine was finished, I cleverly asked to run first since I lost the prior test. He ran his. It took a while. The next morning his finished. The results were the same. Save the physicist was a mixture of miffed and conciliatory. Very brilliant physicist. Not so great at mathematics. I've even encountered programs implemented in a manner to be non-deterministic when a deterministic solution existed. Vexing is working with some programmer that thinks they know everything. Which really annoys the ones that do.

"May your future be limited only by your dreams." -- Christa McAuliffe

Working...