Optimizing Perl 68
An anonymous reader writes "Perl is an incredibly flexible language, but its ease of use can lead to some sloppy and lazy programming habits. We're all guilty of them, but there are some quick steps you can take to improve the performance of your Perl applications. This article looks at the key areas of optimization, which solutions work and which don't, and how to continue to build and extend your applications with optimization and speed in mind."
Error 400 (Score:2, Funny)
Does this mean you can't optimize Perl? =D
Disclaimer: I use Perl almost exclusively for programming.
Re:Error 400 (Score:1)
Good, it's useless for babysitting, or cooking, for example.
FP.
Best PERL Optimization trick ever: (Score:3, Funny)
{ Just Use Python!
Re:Best PERL Optimization trick ever: (Score:2)
Re:Best PERL Optimization trick ever: (Score:1)
(okay the -r is superfluous. so sue.)
Re:Best PERL Optimization trick ever: (Score:3, Funny)
Re:Best PERL Optimization trick ever: (Score:1)
Re:Best PERL Optimization trick ever: (Score:2, Insightful)
You are aware that Python sucks when it comes to speed?
You should look at the python-dev mailing list about how many times people mentioned that Python is slow, especially for regular expressions.
One time someone suggested openning a perl process from within Python.
So think twice before suggesting slow languages.
NOTE: I personally hate Python, so I might not be too objective, but Python being slow compared to Perl is objective!
Re:Best PERL Optimization trick ever: (Score:2)
Re:Best PERL Optimization trick ever: (Score:2, Informative)
this article starts out with the 'some people program in Perl and use terrible habits' point. the problem is, Perl allows you into this bad habit territory, by design of the language.
string handling is just one use for a language. python has plenty of superb string handling libs. its also very difficult to get into the same 'bad habit' territory that you can get into with Perl..
my original post was to make the point that
Re:Best PERL Optimization trick ever: (Score:2)
Re:Best PERL Optimization trick ever: (Score:1)
That said, I'm learning Ruby these days. Ruby continues to impress me
Optimization Rules! (Score:5, Insightful)
The second rule or program optimization (FOR EXPERTS ONLY!): Don't do it yet.
-- fortune
Re:Optimization Rules! (Score:5, Insightful)
Look at his first example, which is concatentating 1 million strings. His "bad" time is 5.2 seconds and the good time is 1.7. Who cares? Nobody uses perl to do high-performance computing. Imagine you are extracting 1M strings from a database and doing something with them. Would you care about a 3 second difference?
Its OK to write good code, but its better to make your code clear and not dependent on clever tricks.
Re:Optimization Rules! (Score:1, Interesting)
Re:Optimization Rules! (Score:2)
Atitiudes like this is a large part of why software sucks so badly. Hardware enjoys a huge speed increase every year, but my computer can't do any more because software has slowed by about that much. Newsflash- I don't care if new computers are faster, the speed of my existing hardware didn't increase.
Performance matters. And just like security, it can't be an afterthought. You need to design for efficiency, trying to sh
Re:Optimization Rules! (Score:3, Insightful)
Performance always matters. A program which computes the monthly payrolls is useless if it takes three months to run. However...
Re:Optimization Rules! (Score:1)
I'm a speed freak, everything I do needs to be fast (I work with computational number theory), but I code almost entirely in C, and only very very rarely resort to assembly language. I often end up with faster code than those that are 100% assembly language, due to investment in your #3 and #6.
And all of my house-keeping tasks (management of lists of candidates, or factors, or whatever) I code in Perl, because of your #1 (and #3).
FP.
Here's My Style Guide (Score:5, Informative)
Here's my style guide, something I developed using Perl for over 5 years now.
Pardon the length, it's unavoidable.
Perl Coding Conventions and Style Guide
By Kevin J. Rice, Kevin@justanyone.com
General conventions:
Read the Perl style guide (http://theoryx5.uwinnipeg.ca/CPAN/perl/pod/perlst yle.html), and follow the conventions therein, especially the following:
4-column indent
Blank lines between chunks that do different things.
Use mnemonic variables- the names must mean something useful. No one character names!
Variable naming conventions:
$ALL_CAPS_HERE constants only (beware clashes with perl vars!);
$Some_Caps_Here package-wide global/static (also prefix 'gv_', see below);
$no_caps_here function scope my() or local() variables;
Be consistent.
Be nice.
Specific Coding Practices:
1. Always do a 'use strict;' at the beginning of every module and script. This catches both subtle errors and bad coding practices.
2. Programs should pass 'use warnings;' with a minimum of warnings before going into production. Note: turning off warnings in production is sometimes required for security or stability purposes. Solve root cause for all warnings if possible; don't just eliminate immediate cause.
3. Turn on Taint checking for all cgi / web enabled scripts. Invoke with "perl -T" or "#!/usr/bin/perl -T".
4. Use spaces for indenting, not tab characters. No file should contain any tab characters. These display differently in various terminals/editors, and mixing spaces and tabs makes code very messy. Most modern editors can be set to automatically insert spaces in place of tabs.
5. Each subroutine should perform one distinct task. Feel free to break down lengthy (i.e., more than 1-2 screenfuls of code) subs. This means almost all subroutines should be 120 lines or less; longer ones should be justified in code review.
6. Code blocks, when more than 1 or 2 lines, should have the block { } at the same indentation level to aid visual clarity of where that block starts/stops. Example:
7. Fully parenthesize stuff like "if ($a >= 5 || $b > 4)" into "if (($a >= 5) || ($b > 4))" so the user has no need to know/get wrong the order-of-operations. This includes one line conditionals like, "if (a) {}" - don't do: "if a {}".
8. Evals: Always use evals when doing system calls. If otherwise using them, always comment/explain why. If you know something might 'die', explain it specifically, since it probably isn't obvious.
9. Explicitly 'return' values at the end of every sub. Don't EVER use the last statement's value as a default return value; someone modifying the code later might not know you're depending on that value.
10. All modules must explicitly end with '1;' to provide a return value for the module.
11. Minimize the use of map() due to its confusing nature.
12. Use parentheses around all function calls, such as sort($a, $b) instead of "sort $a $b;" to make it obvious a function call is occurring. Prefer not to use the Perl subroutine operator, as in "&subroutinename($arg1, $arg2);" just do "subroutinename($arg1, $arg2);".
13. Don't use the 'unless' verb. Instead of, "unless($foo) {...}", code: "if (!$foo) {...}". The 'unless' verb is plenty confusing due to its uniqueness to Perl.
14. Modules and scripts, when over 200 or so lines, should have a logMessage() subroutine that allows for various levels of logging (0=silent, 1=minimal, 4=normal, 8=verbose): logMessage(1, "message");
15. Use a main() sub for all scripts, and include an explicit exit with an exit code appropriate to the platform you're on. Do not
Re:Here's My Style Guide (Score:4, Interesting)
I usually put the CVS $Log: $ at the end of the script, after an __END__, and place a note about it at the top. You can also use the following snippet if you want to make your module's version equivalent to the CVS revision number: use vars qw($VERSION); $VERSION = ('$Revision: $ ' =~ /(\d+\.\d+)/)[0];
Re:Here's My Style Guide (Score:2, Interesting)
use vars qw($VERSION);
$VERSION = ('$Revision: $ ' =~
some comments (Score:3)
6. bullshit. That's personal taste.
13. utter bullshit. What's confusing about 'unless'? It may be unique to perl, but it's a pretty obvious english word.
14. bah
16. good! good that you don't prohibit gotos
18. bullshit. The $_ variable has a nice semantic value. Of course it should be used only in small blocks.
21. hmm.. Sometimes it's n
Re:some comments (Score:5, Insightful)
6. The "bullshit... personal taste" aspect of brace alignment is both true and misleading. Really, it doesn't matter which way you do it, as long as you're consistent. But, with multiple people working on the same code, consistency is difficult. I've always done it with left brace on the left margin so I could easily see what lined up where. If your rule is opposite, fine, but USE ONLY ONE and code looks much nicer.
13. UNLESS (pardon my french) = stronzino (a little piece of shit). It's in the language to assist removal of a single ! 'not'. This can really confuse people. I'm not the smartest guy, nor the dumbest, but sometimes I see it and just go, "huh?". I'm not used to it. Neither have been many other Perl coders I know when we've spoken about it.
14. I take it by "Bah" you don't like scripts to log their actions. I've fought this recently with a 'know-it-all' type who wanted to build something fancy to do logging "when I get around to it". Yuck. Keep it simple, log what's going on so you can trace it later. Simple text files with "just did this, value=12" can help tremendously in debugging production problems. Users never know what they did; error messages never can contain enough info about what happened before.
16. GOTOs are evil. I admit to some brainwashing by CS profs on this, but have dealt with enough spaghetti code to agree with it. Yes, there are times when it's good. But, in my last 100,000 lines of Perl, I haven't had to use it yet. So, it must not be vital. My goal is simplicity of code, not speed, since who cares about speed most of the time anyway, unless it's really bad, in which case there's probably somethign you're doing wrong otherwise.
18. $_ is valuable only until you need to know what's in it. Then, you need a real variable name. You also may need that var to stick around past the next function call. I say, use 'my $request = $_; ' or something to grab $_ and make it obvious.
21. Declaring vars near use is good ONLY in subs. If you have:you'll get an error during parsing due to GV_DEF_ONE not being declared yet.
Regardless, Global vars are hard enough to spot and should be rare, declare them all at the top of the module to make it bloody obvious you're using one.
22. I can sometimes agree to my ($a, $b) = split(',',@inlist); but not disparate vars all crammed together on one line, it's not readable, the vars are hidden, not aligned and initialized, etc.
29. Lines of hashes visually indicate end of file. I can always tell I have the last page of a printout when all my files end with 5 or so rows of hashes. Just convention and a good idea, not a hard-fast rule.
Re:some comments (Score:1, Troll)
Lameness filter encountered.
Your comment violated the "postercomment" compression filter. Try less whitespace and/or less repetition. Comment aborted.
13. There are times the 'unless' fits right in. Like when the action will take place by default, but have some exception. Like
print $dirname unless $dirname =~
16. Spagheti code is evil, not goto. Since Perl have more flexible flow control, I don't remember to have used it yet, but when coding in C, sometimes I use goto to cleanly
Re:some comments (Score:1)
How is "unless" MORE confusing than "if not this is true?"
die "Connection lost" unless $connection->ping;
Thank God for unless.
Re:some comments (Score:1)
Please name an editor or terminal which doesn't treat TAB in the way God intended. Unless the user selects the option "I want my text files to be incompatible with all other tools in the known world". Which, granted, a surprising number of people do ...
Re:some comments (Score:1)
MS-DOS/Windows edit.com doesn't! It converts tabs to spaces on load, and (maybe) converts (some) spaces to tabs on save. Of course, God probably doesn't intend for people to waste their time with edit.com in the first place.
Re:Here's My Style Guide (Score:2)
I've never understood this one. Granted, mixing spaces and tabs makes for messy code, but there are two solutions to that: Use only spaces, or use only tabs.
Why not tabs? They're easier to type when you're deeply nested, just hit
Re:Here's My Style Guide (Score:2)
As I mentioned above in a reply to that post, the varying depth of tabs can really get you in trouble.
My editor (http://ultraedit.com/ [ultraedit.com], when I hit the tab key, insert 4 spaces. Thus the ease of tabbing over to column 20 is indeed 4 keypresses. However, if my coworker does the same thing with his tab settings at 8, he hits tab twice and then puts in 4 spaces to align it. Ug. Or, hits tab 10 times if he's using a tabsize of 2. Yuck again.
Emperically, you want a study that says that mixed use wast
THE EDITOR IS THE PROBLEM (Score:2)
*THIS HITS THE NAIL ON THE HEAD*
You've got it configured to *insert* spaces when you hit tab. I don't recall ultraedit doing that by default (haven't used it in a few years tho). Most editors I use by default will *RENDER* a tab as X spaces, not actually *CREATE* X spaces. If it renders as X, you can easily change the rendering. But once they've become spaces, you can't go back (easily anyway).
Re:Here's My Style Guide (Score:1)
Every editor worth using has a function that makes it automatically insert spaces when you press Tab - but very few have functions that make them automatically insert tabs when you press Space.
Therefore, it's easier to configure your editor to insert the right sort of spacing whichever key you press if you're using spaces rather than tabs. Therefore, using spaces means you're less likely to end up with mi
unless (Score:4, Informative)
And then stress-test with Slashdot ... (Score:5, Interesting)
For instance, flock is your friend ... and as I outline in my slashdot effect analysis [komar.org] you had better be prepared to handle race conditions. Ignoring the web server overload (mod_perl would have helped here), the code actually hung in there fairly well as I've learned from past "mistakes" when I've seen some pretty funky error messages crop up ... but even this time around, there was two minor corner cases I failed to account for (had never been "tickled" before) ... but those are fixed now so I'll be "more" ready if my christmas lights [komar.org] show up on Slashdot again ... but then again, you are never really "ready" for Slashdot! ;-)
Universal Guilt (Score:3, Funny)
Are you accusing me of writing PERL? Come over here and say that again!
Re:No offense (Score:3, Insightful)
Re:No offense (Score:1, Funny)
My approach to sex is the same to programming: have fun, be kinky, but don't be lazy and take proper precautions. I don't blame the girl if I'm a bad lay.
Why the hostility about being a geek and not getting laid? You realize you're arguing with an anonymous programmer on slashdot about programming... not exactly a non-geeky thing to do.
Re:No offense (Score:2)
Orcish maneuvre (Score:2)
Re:Orcish maneuvre (Score:1)
Re:Orcish maneuvre (Score:2)
It's a shorthand operator. If you want to add a number to a variable, you can do this:
Which is effectively this:
||= works the same way, but with the || operator, so
works like
It sets $quux to 'foo' if $quux doesn't already contain something (which evaluates to a "true" condition). If not, it does nothing at all to $quux.
perl optimization vs general optimization (Score:3, Insightful)
But there are some good tips there, too: the part about string handling, references, and the AutoLoader.
Article is seriously flawed (Score:2, Interesting)
First it has a syntactical error with the "x" operator; it puts the number on the left and the string on the right, but the actual syntax it the other way around. If the author had actually tried to run his examples, he would have noticed this.
Then the author says that putting as much text in a single-quoted string as possible better, and says that something like:
print 'aaaaaaaaaaaaaaaaaa',"\n"
is better than:
print "aaa
The example is flawed; the theory is ok (Score:2)
Incidentally, I am getting a slightly better speed on the singlequote example (as claimed). My times a
A better way to Sort than in the article. (Score:5, Informative)
Data object 1: upddate = 111, updtime = 1100, itemid = 200
Data object 2: upddate = 1111, updtime = 100, itemid = 200
So both strings would have a sort value of 111110200, but of course, data object 1 should be sorted before data object 2. Using delimiters in the sprintf statement will ensure that different fields are marked as different, but they will interfere with the sort order.
Another problem is that if your sort string is too long, perl may convert it to a floating point number and thus lose the data from the later fields.
The more correct way to do this sort isThe added benefit of this method is that it definitely won't have overflow problems (which may be the case in the above examples, because "<=>" is the numeric compare operator. Had the author used "cmp", there would then be a quantity of numeric comparisons proportional to the length of the sort string.
The other benefit of my sort is that it is more flexible. you can change the "<=>" operator to a "cmp" operator if one of your fields is string data.
The sort that I propose (one I've been using) may or may not be faster than the "faster" sort proposed by the author, but then again, speed is nothing without correctness.
AKA "How to micro-optimize in perl" (Score:4, Insightful)
Better yet, I would have liked pointers on how to test code snippets for performance (such as illustrating the use of Benchmark or Devel::SmallProf), and then possibly a few pointers like this. (and why was Memoize left out of an article like this?) This sounds like someone writing perl who'd rather be writing assembly code.
In optimizing my (and others') perl scripts, the best tools I've found are the profiler and an understanding of what the code is supposed to do. That, and changing the nature of deployment of the program - from a cgi script to mod_perl, for example. All these little techniques are chasing after grains of sand, when there's a big rock right in front of your face.
Optimizing schmoptimizing (Score:4, Interesting)
They run perl scripts all the time to crunch text files containing lots of data coming in from remote sensors and stuff like that. He told me that the more senior guys have the philosophy is "Optimize? nah, just let it run the extra 20 minutes."
And they're talking about scripts that get run in a cron job DAILY.
Re:Optimizing schmoptimizing (Score:1, Insightful)
Re:Optimizing schmoptimizing (Score:2)
Re:Optimizing schmoptimizing (Score:2)
Then, maybe it was a 'if it's not broken...' case.
Perl is the only language ... (Score:2, Funny)
parse tree (Score:3, Interesting)
My favorite perl joke: (Score:5, Funny)
Manager: How many lines of code did you write today?
Developer: One.
DAY 2:
Manager: How many lines of code did you write today?
Developer: One.
Day 3:
Manager: How many lines of code did you write today?
Developer: One.
Manager: Are you telling me that in three days you've only mangaged to write three lines of code?
Developer: You don't understand -- I've been working on the same line of code all three days.
Manager: (pauses) You're writing in perl again, aren't you?
My 0.02 Euro ;) (Score:4, Interesting)