Forgot your password?
typodupeerror
GNU is Not Unix Programming

Skein Hash... In Bash 90

Posted by Unknown Lamer
from the sixty-four-bits-required dept.
First time accepted submitter Matt16060936 writes "...Last night (err.. 3am this morning) I finished an implementation of the Skein 512-512 hash algorithm (version 1.3). I'm a fan of Skein and hope it wins the SHA-3 competition next year. One of the nice things about Skein is how quickly it's been adopted by many platforms and implemented in many languages. To that end, I present Skein 512-512 implemented in Bash."
This discussion has been archived. No new comments can be posted.

Skein Hash... In Bash

Comments Filter:
  • Ah, Bash (Score:5, Insightful)

    by jeffmeden (135043) on Tuesday September 13, 2011 @03:55PM (#37391032) Homepage Journal

    The cause of, and solution to, all of life's problems. Here's to Bash! If it weren't for you, I would be a terrible sysadmin. Thanks to you, I am a terrible sysadmin!

  • The Wikipedia article states: "The function is optimized for 64-bit processors". On a 32-bit computer, the Bash program will die with an error message "This program is written for 64-bit architectures." So if Skein is adopted as SHA-3, it might run unacceptably slowly on as legacy PCs with pre-x86-64 CPUs and on handheld devices with an ARM CPU.
    • by blueg3 (192743)

      So, just like SHA-2?

      • by tepples (727027)
        What you say is true of the 512-bit variant of SHA-2, but the 256-bit variant of SHA-2 still uses 32-bit words.
    • by butlerm (3112)

      A slightly slower than usual hash algorithm is not likely to be noticeable on a modern client, even a handheld device.

      All else equal, a 64 bit algorithm should get roughly twice the amount of work done per operation as a comparable 32 bit algorithm, so the performance overhead on 32 bit architectures isn't nearly as bad as it looks. Ten to twenty percent maybe.

      And if you are really concerned about performance, you write an implementation in assembly language.

      • by tepples (727027)

        A slightly slower than usual hash algorithm is not likely to be noticeable on a modern client

        Even if the hash is run every time a process is started, in order to check that the executable has not been altered since it was installed?

        • Yes. You could probably run several thousand rounds of Skein on an input each second, and thats being conservative.

        • by butlerm (3112) on Tuesday September 13, 2011 @05:58PM (#37392256)

          On the slowest 32 bit processor tested (32 bit, ARM v4, 75 Mhz), NIST benchmarked Skein using portable C code at just under 1 megabyte / sec. That is about twice as slow as SHA-2 on the same processor, and certainly slow enough that you might notice in the case you mention. On modern 64 bit processor (Intel Q6600, x86_64, 2.4 Ghz), more like 286 MB / sec for Skein, about twice as fast as SHA-2. See here [nist.gov] (pdf).

          The striking difference between 32 bit and 64 bit implementations is much more than I would have guessed, but that may be merely a matter of optimization. For now it looks like a good excuse to use SHA-1 or SHA-2 when doing the sort of thing you describe on slow processors. For something like SSL or IPSEC, you aren't likely to notice the difference, because the bandwidth to a typical mobile device just isn't that high.

      • by drinkypoo (153816)

        All else equal, a 64 bit algorithm should get roughly twice the amount of work done per operation as a comparable 32 bit algorithm, so the performance overhead on 32 bit architectures isn't nearly as bad as it looks. Ten to twenty percent maybe.

        It depends on which processor you're using. x86-64 not only has a crapload more registers but it actually has general purpose registers. x86 instructions often (maybe even usually) expect your operands to be in certain registers and put the results in certain registers, to the extent that x86 really has zero general purpose registers. This is mitigated by register renaming on modern platforms. On a modern machine the cost might not be immense. On an older machine, it will be. Also, longer words mean less lo

    • by Hatta (162192)

      It's going to run really, really, really slow on 8 bit computers.

    • Look at page 25 of the Skien Whitepaper [schneier.com]. Using C implementations, Skien-512 outperforms SHA-512 and Skien-256 is only about 75% slower than SHA-256 on a 32 bit CPU. That isn't unacceptably slow.

  • Damn. Slashcode won't let me post the Skein encrypted version of first post (Filter error: That's an awful long string of letters there.)

    • by Anonymous Coward

      Thank goodness, since hashes aren't encryption.

      • That depends on how well the hash function distributes its hashes, the size of the original message, and how good your rainbow tables are :p

  • Hmmm (Score:2, Insightful)

    by Anonymous Coward

    Amazing how code is always produced in the middle of the night...

  • Why didn't you post this on Github?

    At least it has syntax highlighting, I'm no expert at Bash, but I don't think anyone would appreciate reading it with no syntax highlighting.

    • Re: (Score:2, Interesting)

      by gstoddart (321705)

      At least it has syntax highlighting, I'm no expert at Bash, but I don't think anyone would appreciate reading it with no syntax highlighting.

      People have been doing clever things in bash long before there was syntax highlighting. In fact, it took me years to finally accept that as anything other than clutter. Though, I still can't stand colored output of the "ls" command.

      Having cut my teeth using line editors over 300 baud modems (or terminals wired into the mainframe via serial cables), I am sometimes amu

      • I can totally relate to what you are saying because I am one of the worst offenders... yet I'm a painter. I have all these cool brushes and rollers and pneumatic sprayers and scrapers and paint in all manner of colors and shades. But when I get to the client site, I look like an idiot because I have to resort to scraping off old paint with my fingernails and painting several coats of paint with my fingers. It takes forever and I gotta tell you it gets aggravating. But what am I gonna do? I got so used to al

        • by gstoddart (321705)

          LOL ... funny.

          No, seriously ... when I first saw syntax highlighting I was literally like "WTF is this crap". It was just a jumble of colors on the screen, and I found it quite distracting.

          I was used to working on monochrome VT52s and VT100s ... so the first time I saw syntax highlighting, I turned it off. To this day, I find the color output of "ls" actually conveys less information that knowing that an "@" is a symlink.

          Now, of course, I'll take syntax highlighting any day of the week. But it really did

          • I get where you're coming from... but control is an illusion. That's default in linux. I only first noticed that stuff when I originally jailbroke my iPhone... saurik turned Darwin (BSD) into linux... at least that's what it looked like. After that, I noticed it was like that in linux... just didn't notice it before. You can customize your command line till you're blue in the face... add paths, colors... etc.. but kind of a waste of time. Or you could just drag your .bashrc or .bash_login around with you, a
  • by vlm (69642)

    Come back with the intercal implementation.

  • Sadly the hash of the bash script is only marginally less readable to me than the source.

    • by gknoy (899301)

      A skim of the functions looks like it is a clever implementation of bitshifting left and right (in varying amounts), as well as a block portion. May I suggest Applied Cryptography (http://www.schneier.com/book-applied.html) ? While it may not cover this particular hash algorithm (perhaps recent versions do?), a lot of the actions used here are covered there. The first half of the book (third?) is non-code, and VERY informative to anyone interested in how encryption works.

  • I've had times when I'd have found it useful to have something like base64 or md5 in shell script form to require less dependencies on ancient installations, assuming it could be made work with anything approaching acceptable performance.

    Unfortunately this isn't it. A 194 bytes file took 3 seconds. The skein script (10K) takes 2 minutes and half, and it's ridiculously memory intensive too. The process grew to 150 MB in size.

  • Matt Tomasello for:
    echo 'Usage: cat FILE | skein [ARGS]'

    • 978-0517545164

      • by kwark (512736)

        Looks very entertaining, will make a great fit for an old unix admin/friend and cat lover.

    • by snookums (48954)

      Matt Tomasello for:
      echo 'Usage: cat FILE | skein [ARGS]'

      I find this kind of pedantry a bit dull. When I'm hacking around on things at the command line I almost always use that form. If you're building up a pipeline of various parts to get something done, and you need to insert at the beginning, or reorder components, it's much easier if the input to the pipeline is a clean, separate term.

      Trivial example:
      cat FILE | foo is easily edited to head FILE | foo in a way that foo < FILE is not.

      • by Rysc (136391) *

        Hacking around on the command line is one thing. Writing examples is another. When you save your code for long-term re-use or reference purposes it should be well written.

        • But what's the problem with using cat here? It's not necessary, but it doesn't harm either, does it?
          Also, well written code means maintainable code. Maintainable code means code that is easy to change. Which speaks for cat.

          BTW, I've even found uses for the apparently useless use of cat in cat | command or command | cat (note: no filename on the cat!). That's when using programs which test whether standard input or standard output is a terminal, and I want the non-terminal behaviour even though I'm typing in

          • by Rysc (136391) *

            But what's the problem with using cat here? It's not necessary, but it doesn't harm either, does it?

            In fact it does do harm. Not even counting tangentials like "it encourages people to do this reflexively so they will get bitten when the difference really matters", it execs one more program and uses a good chunk more memory.

            Maintainable code means code that is easy to change. Which speaks for cat.

            I dispute that "cat foo | grep bar" is easier to change than "grep bar BTW, I've even found uses for the apparently useless use of cat in cat | command or command | cat (note: no filename on the cat!). That's when using programs which test whether standard input or standard output is a

  • ...not to be confused with Skin Rash.

  • Newsflash: a known algorithm implemented in a Turing-complete language. Uhm...

    • by zm (257549)
      What is bf?
    • by danlip (737336)

      Newsflash: a known algorithm implemented in a Turing-complete language poorly suited for that algorithm

      fixed that for you.

      I implemented it last night using my abacus and slide rule

      • I always like to challenge people to try it in Brainfuck or with a single instruction computer. Turing completeness is more of a theoretical thing as it ignores difficulty, and polynomial time Turing machine stuff only partly captures that.

        • I've done it on my (purely theoretical) single instruction computer.
          Now, it wasn't exactly difficult because the single instruction of that computer has been well chosen.

          Here's the implementation:

          DWIM

  • Now do the validation suite for it. That's really the hard part.

    • by dfsmith (960400)
      Well, given the hex<—>string routines should probably be printf '%02x' instead of '%x', I'd give it a slim chance of validating.
  • On one of my systems:

    me@here$ uname -m
    i686
    me@here$ getconf LONG_BIT
    32
    me@here$ echo $(( 1 << 32 ))
    4294967296
    me@here$ echo $(( 1 << 64 ))
    1

    Your test would consider my ARCH as 64 when it is clearly not. But then why does left-shifting 64 times not overflow?

Mathematicians stand on each other's shoulders. -- Gauss

Working...