Forgot your password?
typodupeerror
Perl Books Media Programming Book Reviews

Extending and Embedding Perl 145

Posted by timothy
from the serendipitous-augmentation dept.
habit forming writes "Enjoy using Perl? Ever marvel at how Perl can "do the right thing" but still be written in C? Extending and Embedding Perl aims to take the black magic out of understanding our favorite language. In fact, the authors flat out admit they think it is unfair that only so few of us get to have one foot in Perl and one in C. Tim Jenness and Simon Cozens attempt to break down that barrier with lots of annotated code examples, direct analogies from the structures in Perl to those in C, a fine-grain look at XS and what it takes to robustly use a Perl interpreter in C."
Extending and Embedding Perl
author Tim Jenness and Simon Cozens
pages 375
publisher Manning (http://www.manning.com/)
rating 8.7 out of 10
reviewer habit forming
ISBN 1930110820
summary Get in touch with the inner Perl.

What's that up your sleeve?

It is my experience that many situations require us to "look under the hood" of (thoroughly examine) a solution to understand how to best use it effectively. Perl is no exception. The ability to bring such a force as Perl to a project at the proper time is a valuable skill to possess. However, wading chest-deep into XS and the Perl internals is not for the faint of heart. Jenness and Cozens ease this process by stepping in lightly at first.

What's in it?

The book begins with simple C examples that are then related back to the readers' knowledge of Perl. Then the text seems to throw us a curve by leaping off into building Perl modules. But there is method to the madness: building Perl modules correctly is inextricably linked to XS. Light introductions to XS are performed and the reader is well on his/her way to building .so extensions to any .pm.

After building a very specific foundation of simple C examples, module building, and some XS, the text returns to C to introduce pointers, arrays, file I/O and memory management. With these new skills, we begin to explore the structure and implementation of Perl variable types. Chapter 4 provides many useful diagrams of how Perl variables "look" and what C structures they translate into.

Still following a logical and constant order, we explore the Perl 5 API, learning how to post and retrieve information to the variable types explored in the previous chapter. As much as it might seem, this is not a rehash of the perlapi doc. It is consistent with the perlapi doc, but Jenness and Cozens provide extensively annotated C code examples.

Casting deeper still, we add the advanced C of pointers, arrays, file I/O and memory management to our knowledge of XS. At this point we have everything we need to effectively extend Perl, but the text continues deeper still by exploring how XSUB interfaces to Perl's internals. It is only the clearly documented, step-by-step explanations of this chapter that make it manageable for an average user like myself. Chapter 7 ends our stint with XS by discussing some alternative XS (or equivalent code) generation suites.

Switching gears entirely, we grab libperl.a and stuff into a C program. Chapter 8 begins the task of embedding Perl into a C program. Jenness and Cozens continue the embedded discussion through a Case Study in Chapter 9 and end with a look through the Perl internals in Chapter 10.

The final chapter (Chapter 11) details some of Perl's history, its development process, how we could become involved and what the future of Perl and Perl 6 may entail.

Final Thought

This book was indispensable in gaining a good foothold on using Perl in, from, and around C. I found it a good reference guide as well as an easy ,linear read. It is not a replacement for the perlguts, perlapi and perlxs documentation, but then again, it doesn't try to be. The annotated code examples with every line explained make following the book with development of your own solution a lot easier than in some other books. However, the in-depth explanations can be a bit frustrating for the impatient.


You can purchase Extending and Embedding Perl from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

This discussion has been archived. No new comments can be posted.

Extending and Embedding Perl

Comments Filter:
  • by gorilla (36491) on Tuesday October 22, 2002 @12:35PM (#4505155)
    For many applications, you might find it easier to use Inline::C [cpan.org] or SWIG [cpan.org]. Neither gives you the total power that XS does, but they're much easier to get into.
    • by Hornsby (63501)
      And thank god it's not! XS can be a royal PITA. The marriage of Perl and C is a shotgun wedding at best, and it seems that the groom is wasted before the ceremony even starts.

      Python makes it much easier to interact with C libraries, and Ruby has the nicest C library support of all. Also, for embedding a program language into an application, why not use Scheme? It was designed to be embedded from the beginning and should impose much less overhead. Since it's a functional language, it's also very well suited for AI, which makes it a good choice for games and such.

      I'm not knocking Perl. It has a special place in my heart as the first language I really learned; however, it's best used for what it's really good at, and that's scripting.
      • Sorry for that 'I agree' post, but that's just it.

        Damn, I agree. I could have written exactly the same thing. Except for Ruby which I don't really know about, but Scheme is just the best thing to enhance a C program and give it the dynamism it lacks.
    • Chapter 7 is alternatives to XS. He talks about SWIG and Inline::C as well as some other lesser known things like PDL::PP.
  • by NoOneInParticular (221808) on Tuesday October 22, 2002 @12:37PM (#4505165)
    So now we can combine the readability of both C and Perl with the elegant memory protection of C?
    • by Scotch Game (442068) on Tuesday October 22, 2002 @07:11PM (#4508641)
      I know this is a joke, but it's a play on a concept I think it's time to reject, at least in part.

      Far too much has been made of Perl readability. Yawn. Recently I opened up some code I wrote four years ago for a rather obscure application that I hadn't looked at since. After a minute, and only about a minute, I understood it perfectly. Granted, I wrote it, but similarly I rarely have problems reading other programmers' Perl code, if it's well written or at least well written enough.

      Poorly written code is hard to read in any language, but Perl isn't very difficult. Sure, there's the odd regular expression that's difficult to decipher, but overall it's how well you know the language in question.

      It just frequently seems that the readability issue seems to be centered around how well the code can be read by readers not only new to the code but new to the language as well.

      For that matter, far too much has been made of Python's readability. I'll grant that Python isn't difficult to read, but I've seen some pretty goofy Python code, as well as Perl. Python, or any language for that matter, can be nicely obfuscated just by naming variables poorly.

      And as far as memory protection in C is concerned, well, it's no worse than assembly, and C is smokin' fast when written properly.

      In short, you want to go fast, then you're going to be driving Indycar and it's going to be risky for the amateurs. You want safe? Try Java, and buy a Prius.
  • Perl 5 API??? (Score:3, Insightful)

    by mustangdavis (583344) on Tuesday October 22, 2002 @12:41PM (#4505216) Homepage Journal
    "Still following a logical and constant order, we explore the Perl 5 API, learning how to post and retrieve information to the variable types explored in the previous chapter."


    Isn't Perl 6 coming out soon?

    Maybe the author should have held off releasing this book ... he might have been able to make more $$$ selling a Perl 6 book rather than another Perl 5 book (since every good programmer has a half dozen Perl 5 books anyways)

    • Re:Perl 5 API??? (Score:3, Interesting)

      by gorilla (36491)
      Perl 6 is still being designed, and is expected to be at least 18 to 24 months away from the first version. It would then take a period of at least a year before it was production ready.
    • Re:Perl 5 API??? (Score:5, Informative)

      by Reality Master 101 (179095) <RealityMaster101&gmail,com> on Tuesday October 22, 2002 @12:56PM (#4505331) Homepage Journal

      Isn't Perl 6 coming out soon?

      "Soon"? Considering that they haven't even finished [perl.org] deciding the features and changes of Perl 6, I think it's safe to say that a release version is at least a few years off, with 50% adoption being another three years plus after that.

    • Re:Perl 5 API??? (Score:2, Informative)

      by WWWWolf (2428)

      True, Perl 6 is coming, but the shape of the language is still being discussed, the virtual machine isn't doing that much yet, and there's not really anything substantial yet... it may take a year or two until Perl 6 is out (not sure about the developers' actual schedule, though).

      As far as I know, there's not many (if any?) books that discuss the XS or Perl embedding. It sure isn't covered that widely in the Camel or Ram, and the only reference has been "go RTF 'perldoc perlxs'"... =)

      And most importantly, the Perl 6 folks have not said a word about how XS and embedding stuff works in Perl 6. (I suspect that it will be radically different, because of the Parrot...)

      • Re:Perl 5 API??? (Score:3, Informative)

        by bcrowell (177657)
        And most importantly, the Perl 6 folks have not said a word about how XS and embedding stuff works in Perl 6.
        They have said a lot about it: XS is going away completely, and interfacing C to Perl is going to get a lot easier. (The quote I remember from their FAQ was something like "how could it be any harder than it already is?")

        As far as embedding Perl, well, the Perl interpreter is going to be written in Perl starting with Perl 6. Instead of embedding an interpreter, I think you'd just embed a Parrot VM and hook your compiled Parrot bytecode into your program.

        • XS is going away completely, and interfacing C to Perl is going to get a lot easier. ... Instead of embedding an interpreter, I think you'd just embed a Parrot VM ...

          Yeah, I figured it would be something like this =) XS is pretty arcane.

          Thanks for this interesting nugget of information...

    • Re:Perl 5 API??? (Score:3, Informative)

      by MarcoAtWork (28889)
      Some observations:

      Maybe the author will make more $$$ releasing the perl 5 book now, and the 'revised' perl 6 version next year :)

      Also don't forget the sometimes extremely long lead times for book publishing, it is entirely possible that the author finished this book 6+ months ago.

      And last but not least, yeah, perl 6 is going to come out soon, but do you really think I'm going to use it for production code right away? I really don't think so, perl 5 will be the tool of choice for quite a while longer.
    • Re:Perl 5 API??? (Score:5, Informative)

      by hamsterboy (218246) on Tuesday October 22, 2002 @12:59PM (#4505355)
      AFAIK, Perl 6 is a whole other beast. It's a complete rewrite, with changes to the core language.

      You'll still be able to run your Perl 5.x scripts under 6, but not vice-versa. Thus, with all the existing Perl 5.x scripts existing in the wild, having a Perl 5 book around may still be handy.

      If you like analogies: why would you buy a C book when C++ has been around for years?

      -- Hamster
    • Re:Perl 5 API??? (Score:2, Insightful)

      by thelenm (213782)
      Isn't Perl 6 coming out soon?

      We all hope so, but it's not necessarily going to be released even in the next year or two. In the meantime, Perl 5 is more widely used than ever. And even after Perl 6 does come out, it will take a long time for people to switch over, assuming they do decide to switch. Heck, there's still a lot of Perl 4 code out there even though Perl 5 has been out for... like 8 years?
    • Isn't Perl 6 coming out soon?

      Only for certain values of "soon". My recollection is that estimates were that it would begin to be available in some useable form after 2 years (this was probably a year ago). I think it will be much longer before it's production ready, though that's just a guess, and several more before perl5 goes away.
    • Re:Perl 5 API??? (Score:3, Interesting)

      by hondo77 (324058)

      ...every good programmer has a half dozen Perl 5 books anyways.

      I do have a half dozen Perl books and I barely survived my first foray into XS. Maybe this book can help me go back and get it right.

    • In a good number of shops, including the one I work for there is still legacy Perl 4 code in prod.

      The guy that runs our UNIX services says that they're slowly phasing it out, and expect to be done by mid 2004.

      Nobody has touched this old code till' present because, well, they were afraid of screwin up the prod environment.

      My point? Perl 5 is much more widespread than 4. Given that, and Larry's statments that 5 code will run on the 6 interpreter, I think that it's safe to buy Perl5 books. The dialect isn't going to die anytime soon methinks.

  • by Sludge (1234) <slashdot@t3.14ossed.org minus pi> on Tuesday October 22, 2002 @12:42PM (#4505228) Homepage
    I'm interested in using Perl as an interpreter for other applications. Or, more speficially, game scripting logic. I am confident the time has come for a scripting language to be used for NPC interactions, even in high-performance environments. However, I've never worked with Perl in such a place. I would like to. Has anyone got any performance comments about embedded Perl?
    • by Avalonia (169675) on Tuesday October 22, 2002 @01:22PM (#4505491) Homepage
      We use the PerlStream classes to integrate a Perl interpreter into our C++ applications - it was covered in the January 2002 C/C++ Users Journal - with great success.

      Our ASCII file import parsers are written in Perl and the data read into Perl data-structures. The contents of these data structures can then be accessed directly from C++.

      The code is on the web (it has some subtle C++ bugs needed fixes using the base-from-member inititialisation idiom) here [willnolan.com]
    • I'm interested in embedding an interpreted language inside my game engine for scripting and also for my networked distributed objects system.

      Right now my choice is set on Ruby. I plan of using it with my C++ engine replacing traditional ObjectViews by Ruby Objects. I've just got started on this task but already I have good hope on Ruby. dRuby (Distributed Ruby) has a 200lines networked distributed objects system that really kick ass!

      Someone told me I might want to look into Boost::python binding. I also thought of Perl to do this, having done a little bit of perl scripting (mostly with MySQL). Anyone have comments on embedding Ruby with C++ ?
    • by Anonymous Coward
      Perl is not the fastest embeddable interpreted language, but it ain't the slowest (TCL) either. The problem with most embedded interpreters is that you have to constantly convert to/from the native C types in your code. This can have a large performance overhead. If you do decide to embed with Perl (or other interpreted languages for that matter), keep your API as coarse-grained as possible, i.e., do most of your work in C functions and only do high-level stuff in perl.
      • Tcl isn't the slowest thing out there:

        shootout [bagley.org]

        even though it certainly isn't the fastest. One of the nice things about Tcl is that it's much easier to write C code to use from Tcl, or vice versa, which makes it very easy to speed up slow portions of your code with C.

        With a good, clean interface, it's a lot of fun to write two-tiered applications, in addition to being an extremely powerful way of doing things. Look at any large enough app, and you are bound to find a scripting language of some kind, be it VB, Tcl, Lisp, Lua, elastiC, or something else...
    • I find for doing most complicated perl-type stuff (parsing text, dealing with large hashes, etc) perl is as fast as anything you would write in C. Granted, you *could* write faster stuff in C but you usually wouldn't because it takes so long.

      Save yourself 6 months of programming, use perl and buy a slightly faster CPU.
      • I understand this. I've been a big Perl guy for years now. Synchronizing things in a real time environment would be a tricky thing, I would imagine.

        There is also the compilation step. You've felt the overhead when compiling new code, no doubt. This would need to be done during some other load time in the game.

  • Perl is Perl, C is C (Score:4, Interesting)

    by signine (92535) <slashdotNO@SPAMsignine.org> on Tuesday October 22, 2002 @12:43PM (#4505239) Homepage
    While I can think of numerous times embedding perl in my C would have made life easier, I can't think of any real reason to actually do so, it seems like a real waste of time to me. If you want to complete a project in Perl, then just do it in Perl. If you want to use C for a project, just use C. It's bad enough when people start using buggy bloated libraries, but it's worse when they give their compiler an identity crisis.

    It's much like the great masters of Funk, The Parliament once said: "I've been down to the south, saw some great Funk...The Doobie Brothers...but do you want white guys all up in your Funk?"

    In the same way, "I've been down through the source, I saw some great code...Perl...but do you want scripts all up in your C?"
    • by NerdSlayer (300907) on Tuesday October 22, 2002 @12:57PM (#4505339) Homepage
      If you want to complete a project in Perl, then just do it in Perl. If you want to use C for a project, just use C. It's bad enough when people start using buggy bloated libraries, but it's worse when they give their compiler an identity crisis.

      I've done this on serveral occasions... did you ever have a huge project to finish in a small amount of time that had some sort of processor-critical portion that's run alot? Well then, I would suggest writing it in Perl and then optimizing that piece in C. Works for me.

      Often in the "real world" you can't have your cake and eat it, too. That's why people embed C in Perl.
    • by shoppa (464619) on Tuesday October 22, 2002 @01:01PM (#4505369)
      If you want to complete a project in Perl, then just do it in Perl. If you want to use C for a project, just use C.

      But many projects need both low-level access to X/graphics/system libraries and really do benefit from the high-level approach of Perl.

      In many cases, there are already libraries available that link the two together (e.g. Perl/Tk if you're writing in Perl, or associative array libraries for use from C) but it's never been a secret that you can call C code from Perl. You seem to advocate keeping it a secret.

      • That's embedding C in perl, which everybody already does. I'm talking about how foolish it is to take a primarily C project and toss some perl into it. Just use some regexp libraries and stop being so lazy about your code.
    • umm... hello? (Score:4, Informative)

      by Ender Ryan (79406) on Tuesday October 22, 2002 @01:27PM (#4505521) Journal
      There are plenty of reasons to embed Perl in C. The first thing that comes to my mind is using Perl as a scripting language for a game, which I have done. Write all the low level graphics stuff in C, then write the high level game logic in Perl.

      As for C in Perl... Perl is a scripting language, it's simply not fast enough for everything, and you're going to need C to access different things, like joysticks, video, graphics libraries, etc...

    • Well what you could do is use PERL for checking/retrieving data properly and easily through a web page with less of a risk of buffer overflows since PERL has some great extensions to help in this process compared to another language such as C but then use another language such as C to pass that data (once it's been checked) so that the data's interpretation could be done exubrantly fast.

      It's all about the right tool for the right part of the job.

      • by Anonymous Coward

        PERL [cpan.org] doesn't have any extensions! It's only a guestbook and hit counter programming language.

        • From that link, anyone else find this sentence disturbingly true?:

          It is a direct descendent of Perl, a programming language which was used mainly by programmers. However, the original language required too much reading and thinking and so PERL was developed as a language which was more in tune with the requirements of the Internet age.

    • I can't think of any real reason to actually do so

      I agree. Scripting seems best for a quick and dirty approach, while actually writing some C or C++ takes more time and consideration. I can't see why you want to mix the two, unless of course you need some low level access to hardware that Perl doesn't offer directly...

      • by NerdSlayer (300907) on Tuesday October 22, 2002 @04:14PM (#4507148) Homepage
        I agree. Scripting seems best for a quick and dirty approach, while actually writing some C or C++ takes more time and consideration. I can't see why you want to mix the two, unless of course you need some low level access to hardware that Perl doesn't offer directly...

        Spoken like a dumb college student. I work for a small company started by my friend from college. We do everything almost exclusivly in perl (with some PHP). Our application is mostly a CGI interface to a obscenely diesel MySQL server (80 GB+, some tables have over 300 million rows, you dirty mysql nay-sayers). It has some low level functions that I wrote in C. It's extrememly modular. Perl has allowed us to write small daemon programs quickly. As our system grows large and larger, we're easily able to replace to slow performing sections with a optimized C version of the same daemon.

        We took our particular space (I'd rather not get into it) by storm. Me and one other developer worked day and night for 6 months and we now have a product that we license for between $50,000 and $250,000 depending on how the client wants to scale. It is easily in the top 3 of similar programs. And it was written by me and another dude. In Perl. With some C. In 6 months.

        As much as I love C and have mixed emotions about perl, if it was written in C we'd still be core dumping and our company would have probably gone out of business, as opposed to quadrupling in size and increasing our gross income 10 fold. Again, I hate perl, even, but there's no replacement for it. I, like you, was once a dumb college student who swore I would never write anything but C. But, then I graduated and realized I needed 22" dubs on my escalade. (okay, I have neither 22" dubs, or an escalade, but you can imagine if I did, right?).

        C and C++ do take more time and consideration, and when you're part of a lean and mean company you do what you need to get ahead. If it wasn't for perl I'm 100% sure that we would not be where we are today.
        • In summary, you are saying that you made a quick (but not entirely dirty) solution in Perl to gain market postion, but it won't scale. To make it scale you need a more considered solution written in C (or any other lower level language), which you intend to implement in pieces - starting at the bottle necks.

          So actually, you probably don't want to mix C and Perl at all (you even say that you hate Perl), but were forced to by market pressure and probably a lack of capital (and skill from the sound of it) to make the whole system in C in the first place. I think you are agreeing with me here afterall - thanks.

          Oh, and I'm not a college student, or dumb; what is dumb is your attitude.

          • Well, after reading your previous comment, I was a bit unsure about NerdSlayers harsh language. On the other hand, you have no proven that you are in fact dumb.

            What you are effectively advocating is the waterfall model, a software methodology that hardly anyone considers a good idea anymore (it's only kept alive by stupid companies, I've yet to find a developer who prefer it).

            If you have no idea what I'm talking about, the waterfall model is the old method of producing software: analyze, design, write, test, deliver. You will have no idea of whether your product actually works before you are in the test phase, and then it's too late to back out, so your company will have to spend a lot to pay for the delayed product (because problems will occur). By the time you actually deliver the product, the requirements will have changed, and it can no longer be used.

            Modern software development methodologies advocate an iterative approach. You analyse, design and write a small part of the whole system, preferably the part that's most critical (thus you find out early if the product is feasible), test it, revise your plan, and then do the next part. This way, you will always have a working tested system, and you can focus on stability long before you have all the features in place. And, just as important, when requirements change, you can change the direction you are going in, without scrapping everything you've done so far.

            Another epitome of good software engineering practice was best said by Edsger Dijkstra: "Premature optimization is the root of all evil". While C will give you better performance if you have the time for it, and assembler even better, it's much better to first write something that works, and then optimize the bottlenecks. This way, you will only have unmaintainable code in the few critical parts that needs it (90% of the time is spent in 10% of the code).

            You should also read Richard P. Gabriels classic paper "Worse is Better [naggum.no] for a better explanation of why this approach is sensible. After having done that, you can then consider rereading the part that says "if it was written in C we'd still be core dumping and our company would have probably gone out of business", and see if it makes more sense to you now.

            PS: I've just made some dumb waterfall-like mistakes myself, so I know what I'm speaking of :-)

    • by hondo77 (324058)
      In my case, I wanted to use a third-party's binary-only C API from Perl. Gotta use XS.
    • In the same way, "I've been down through the source, I saw some great code...Perl...but do you want scripts all up in your C?"

      Dude, I don't know what you're talking about, but that sounds really rude to me.
    • In the interest of upholding the tradition of pedantic bastardry here on slashdot, the original lyrics are: Then I was down south and I heard some funk with some main ingredients Like Doobie Brothers, Blue Magic, David Bowie. It was cool, But can you imagine Doobiein' your funk? Ho!
  • XS Mechanics (Score:5, Informative)

    by swm (171547) <swmcd@world.std.com> on Tuesday October 22, 2002 @12:44PM (#4505251) Homepage
    For an overview of XS, see

    XS Mechanics

  • by sammy baby (14909) on Tuesday October 22, 2002 @12:51PM (#4505303) Journal
    Look for a companion volume to be published by Microsoft Press [microsoft.com] entitled, "Embracing and Extending Perl."
  • My $0.02 (Score:4, Informative)

    by dknj (441802) on Tuesday October 22, 2002 @12:54PM (#4505316) Journal
    This book explains how to expand the functionality and usefulness of the Perl programming language. This guide delves into the complex issues of using real code examples from the Perl source. Detailed is how to use Perl from C programs, such as writing interfaces to C libraries, implementing Perl callbacks for C libraries, and passing Perl hashes and arrays between Perl and C. Additionally, developers are provided with an API reference for the internal C interface to Perl and a reference on the typemap system.

    It's amazing how much this book covers: Not only does Sam Tregar show how object-oriented Perl modules are architected, how to write regression test suites, how to extend Perl modules with C code, but he gets also the community aspects right -- how does your module get really popular? You can tell that Sam is a successful Perl module author himself.

    -dk
  • by Anonymous Coward on Tuesday October 22, 2002 @01:08PM (#4505411)
    An "8.7". Not as bad as the totally garbage "8" books, but well below the standard "9".
  • by truth_revealed (593493) on Tuesday October 22, 2002 @01:11PM (#4505433)
    Don't worry about about Perl5/XS becoming obsoleted - it will be with us for a long time since the Parrot (perl 6 VM) project seems to be going nowhere fast. Parrot is suffering from kitchen sink/second system syndrome.

    Recently, the DotGNU have made an overture [develooper.com] to try to use the Parrot runtime for their C# compiler but found that Parrot needs a lot of work to get to the point where they could use it.

    Some Parrot VM problems:

    no calling conventions yet for subroutines. There is no hope of offering mixed language support unless they do this.

    no conversion opcodes for various builtin types (float, char, short, int)

    non-perl languages expected to provide additional support in the form of C code libraries for their opcodes. This would nix any hope of having a single standard universal virtual machine.

    no way to call out to C code

    no equivalent of Java's jar file or CLR's assemblies for parrot library distribution

    way too many registers: their register based VM (32 int registers, 32 double registers, 32 string registers, 32 PMC registers plus various stacks) requires a sophisticated compiler to do proper register allocation and needlessly complicates their VM.

    no consideration of threads in their design. How will they handle synchronization, for example?

    The points above are not coding issues, but issues of design. It seems that Parrot is too hung up on making the VM efficient and are not seeing the bigger picture - to get the features in place first so that high-level languages can work. Or perhaps they should simply concentrate on getting Perl6 to work first. They need more focus. The project tries to be all things to all people, but ends up satisfiying no one.

    • by Elian (10270) on Tuesday October 22, 2002 @01:49PM (#4505722) Homepage
      You missed the mark in just a few places here.
      1. no calling conventions yet for subroutines. Alas not true. They're in place and have been for quite some time.
      2. no conversion opcodes for various builtin types Also incorrect--we've got them and have since the very beginning. What we don't have is low-level support for specific size integer and floats, since our target languages (perl, python, ruby) don't have them. Adding them adds no overhead, though, so they're going in since it'll make base .NET and JVM compatibility simpler.
      3. non-perl languages expected to provide additional support in the form of C code libraries for their opcodes. Once again, incorrect. No external libraries are or will be required. We're turing complete and generally have a richer set of base semantics than .NET or the JVM does. This doesn't mean that someone can't choose to require an alternate set of opcodes if they want, but the engine doesn't require it.
      4. no way to call out to C code This one's actually true. We've not gotten to that part yet, though it's on the list.
      5. no equivalent of Java's jar file or CLR's assemblies for parrot library distribution Incorrect, bytecode files work just fine for this. That part of the design is somewhat incomplete, though.
      6. way too many registers: their register based VM ... requires a sophisticated compiler to do proper register allocation and needlessly complicates their VM Wrong yet again, sorry. Doesn't requore a sophisticated compiler at all. At best it requires some sophisticated register allocation algorithms. Luckily for us, those algorithms are old, known territory, which is why we've got them implemented already. So what if it makes the VM slightly more complex? (And it is only slightly more complex because of it) We're not writing something for CS101 here.
      7. no consideration of threads in their design. Once again, incorrect, if you'd bothered to read any of the design documents. Threading isn't, at the moment, implemented because other things have been more important, but it has been thought about in the design.
      1.5 for 7. Not too good there. Oh, and this:
      seems that Parrot is too hung up on making the VM efficient
      What drugs are you on? If we piss away efficiency in the design, no amount of clever coding will ever get it back. Maybe you're willing to sacrifice a factor of two in speed for "clarity" but so are nine of your friends. (To misquote the author of make) I, on the other hand, am not.

      Finally

      Or perhaps they should simply concentrate on getting Perl6 to work first. They need more focus. The project tries to be all things to all people, but ends up satisfiying no one.
      If you'll look, you'll notice that perl 6 isn't fully designed yet, but the bits that are have been implemented.

      Just because you can't (or won't, or don't want to) see the focus doesn't mean it's not there. It is, thanks very much, and we're well on track to do what we need and do it well. The design's flexible enough to pick up things like JVM or .NET compatibility without a loss of focus or efficiency, so there's no reason not to.

      • Dude, by simply saying that something is not true does not make it true.
        The Parrot calling convention POD was pulled a while back. I don't call saveall/restoreall a true calling convention. Prove me wrong - show me a doc with the new calling convention.
        Where's Parrot's 64 bit integer type? If you hope to support Java and C# you will need it.
        You *do* need some form of distributable bytecode library format - bytecodes alone do not cut it - all projects cannot be monolithicly compiled in a single bytecode file.
        Threading issues *must* be considered from the very start - it affects every aspect of your design - from garbage collection to memory layout. Look at all the trouble Perl 5 had grafting threads to its design - they only "got it right" in Perl 5.8.
        You're living in a dream world of denial. The project has to address these basic issues if it is to suceed.
        It's history repeating itself all over again. Remember what happened to Topaz: Perl for the 22nd Century [perl.com]?
        • by Elian (10270) on Tuesday October 22, 2002 @02:20PM (#4505988) Homepage
          Nope, but being right makes it true.

          1. The Parrot calling convention POD was pulled a while back. No, it wasn't. PDD 3 is in, and has been for a long time. There have been minor revisions, but that's it--minor revisions.
          2. Where's Parrot's 64 bit integer type? Inside a PMC, at the recommendation of the dotGNU folks. We'll stick the 128 bit, 256 bit, and 512 bit integers in one too, if we need them.
          3. You *do* need some form of distributable bytecode library format No, we don't, not for perl, python, and ruby support. (Who was it whining about lack of focus earlier?) Bytecode files do work just fine for it anyway, however. Nobody said they needed to be structureless.
          4. Threading issues *must* be considered from the very start Yes, I know. If you'd have bothered to do any research, you'd have seen the direction we're going, and you'd have realized that what we've done so far has no ramifications for our chosen threading model.

          The only part of history repeating itself is the naysaying from people who've not done their research. If you're going to criticize, at least get it right.

          • You think PMCs will cure all ills, it seems. I beg to differ. They just defer the hard work to the high-level language implementor. Now the high level language implementor would have to write custom C code to support such things as basic arithmetic - doesn't that defeat the whole purpose of having a generic VM? I don't think many people would be crazy about the idea of linking in shared libraries for Parrot apps to support the PMCs in other languages.
            • by Elian (10270) on Tuesday October 22, 2002 @02:45PM (#4506229) Homepage
              You seem to be missing the point with some regularity. PMCs don't have to be written in C--they will be doable in parrot code if need be. And just because 64 bit integers will be done with PMCs doesn't mean that they won't be part of the core distribution, or a recommended library.

              There's nothing particularly wrong with saying "You must have the X library/module/kit to do Y". Requiring the install of the .NET library to get full .NET functionality will undoubtedly be needed. (We're certainly not going to ship the full .NET core library with Parrot any more than we're going to ship the full Java core library) If we don't ship 64 bit ints as part of the core, they'll either be in the .NET library, or in an extended data type library.

              What's next, will you start complaining next that we're going to require installing Postgres to access Postgres databases? (Or will the next complaint be about the bloated size of the distribution to provide the features that match your expectations?)
              • Parrot lacks key builtin opcodes for universal language support - like support for 64 bit ints. As for the embedded Postres support within the Parrot VM itself, no, a C calling facility from Parrot would do just fine.
                By delaying the decision on how to hook up Parrot to native C APIs you are denying Parrot critical third party support. Like guys who simply want to embed Parrot in their application to control a MP3 player or CAD program or whatever. You need the buy in, you need the additional free testers and proponents. Give them a reason to use Parrot at this stage.
                Actually this talk of C#/dotGNU/Parrot integration is a good thing - it will give Parrot some focus to run a real language. Perl6 is not the greatest target at this because it keeps shifting.
                • by Anonymous Coward
                  x86 lacks key builtin opcodes for universal language support - like support for 64 bit ints.

                  PPC G4 lacks key builtin opcodes for universal language support - like support for 64 bit ints.

                  Java VM lacks key builtin opcodes for universal language support - like support for 64 bit ints.

                  You do understand that Parrot is a VM, don't you?
    • I won't bother with a point by point rebuttal of your trolling, especially since Dan has just countered all of your half-truths much better than I ever could. But one remark in particular stands out & bears a reply:
      non-perl languages expected to provide additional support in the form of C code libraries for their opcodes.

      If I remember the talks Dan has given to Boston.pm correctly, you've got this almost completely backwards. A more accurate reading is that Parrot is designed to act as kind of a superset of the target languages (Perl, Python, and Ruby) and that, in the handful of cases where an abstracted general feature of Parrot doesn't map perfectly back to the languages, any of those languages -- *including Perl* -- may have to adapt to Parrot's needs. In other words, Parrot has certain features because the Python & Ruby folks needed them, and these will require Perl to adapt if it wants to take advantage. (If we're lucky, Dan may clarify &/or add to this -- it's been a few months now and I forget most of the details of the presentation...).

      So let's not have this kind of trolling, please. If you think Parrot is making such fundamental mistakes, help out or fork off your own project to prove yourself. Antagonistically spouting off half-correct technical gibberish on Slashdot isn't helping anyone.

  • Save some money (Score:3, Informative)

    by Anonymous Coward on Tuesday October 22, 2002 @01:20PM (#4505487)
    bn.com has the book for $35.96 Amazon has it for $31.47 [amazon.com].




    ----
    Associates Link
  • pcre (Score:4, Informative)

    by larry bagina (561269) on Tuesday October 22, 2002 @01:38PM (#4505606) Journal
    Why would you want to embed perl into a C application? Probably to access perl's powerful regular expressions. If that's the case, the Perl Compatible Regular Expressions [pcre.org] library is a more ideal solution. PHP, Python, and Ruby all use it to support perl regexes.
  • by Snake (13761) on Tuesday October 22, 2002 @01:42PM (#4505640)
    Interfacing Perl with C has its uses, depending on what your current project is.

    In my case, I'm part of a large scale C++ project. I have the ownership of a module with clearly defined interfaces with the other modules written in this project.

    Since my module relies heavily on XML and strings, I have always wanted to pair it with the power of Perl for testing purpose.

    Among various possibles solutions (XS, SWIG, etc.), I settled on SWIG because it could handle 'shallow' classes. (allowing to expose my module as a perl object)

    This has been the best decision I have made over the last year: when I get a bug case, I simply write a perl script to try to reproduce the problem, add some loops to get some combinatory, then check the result. This drastically cuts down on the time spent on debugging my module (or the modules used by it, for that matter :)

    Pros:

    • SWIG: Relatively easy generation of stub code (by using interface files)
    • SWIG: It is possible to use the same interface files to generate stub code for Java, Python (though, I didn't test this feature)
    • SWIG: Excellent doc.
    • Perl: you can leverage the CPAN/PPM modules to do some truly magical hackings
    Cons:
    • While SWIG does a good job of hiding the gory parts of Perl Internals, you still need to brush up on it to better understand how it works, if only to avoid memory leaks
    • Perl: I haven't been to handle properly the passing of strings (I managed to do it by using a horrible hack that seems to work). I'm probably not smart enough to understand the documentation or the samples.
    • SWIG: the stub code is dependant on the version of perl used. It is therefore difficult to release it. This is mostly a dev tool.

    Summary: If you are a C/C++ developper and your code can use XML/text files/strings, consider using SWIG or XS for testing purpose.

    PS: if you want to Quantify/Purify your module/Perl script, using ActiveState Perl, you need to recompile Perl with the -DPURIFY option toggled on.

  • to call their Perl extension 'grain of sand'.
  • by Animats (122034) on Tuesday October 22, 2002 @02:12PM (#4505919) Homepage
    If you interface a language with automatic memory management with one with manual memory management, like Java or Perl with C or C++, the low-level stuff has to be very, very carefully written to prevent breaking the memory management system. Most application programmers aren't good enough to write the bulletproof code needed to do that right.

    The result, of course, is undebuggable random crashes in the high-level part of the system. Here's are some typical bug reports from mixed Perl/C work:

    • #1 [www.ecos.de] Okay, it seems to be some kind of conflict between mod_perl/Embperl and PHP and perhaps Apache::DBI. My Embperl stuff works if there's no database access. It also works if I don't load libphp4.so. I guess the best solution is to either build everything statically or run seperate servers for PHP and mod_perl.
    • #2: [openldap.org] Following a large number of updates to our database, slapd is prone to crashing when reading values back. We load a database of about 3800 users with slapadd, then modify a single attribute of every 'person'. Then slapd is likely to crash on reading values back. Restarting slapd seems to make it work again. Just prior to the crash, slapd will give incorrect query results. ... We have a large client site limping along due to this kind of problem ... so any help would be welcome.
    After this, you begin to understand the logic behind Microsoft's C# mixed-language run-time environment. That's ugly, too, but more maintainable, because the toolset has some support for mixed-language work.

    I'd like to see safe inter-language calls across a protection boundary. CORBA is about as good as it gets, but it's slow, because it marshalls the data into a stream and pumps it through a socket to the other side. There are faster approaches (look at Multics protection rings) but they need some hardware support, which we don't have today.

    • I'd like to see safe inter-language calls across a protection boundary. CORBA is about as good as it gets, but it's slow, because it marshalls the data into a stream and pumps it through a socket to the other side. There are faster approaches (look at Multics protection rings) but they need some hardware support, which we don't have today.

      As long as you have two rings, you can emulate an infinate number of rings. x86 CPUs have 4 rings...

      Of course, MS is never going to put in the kernel code to have the MULTICS 32 rings on x86...

      On the other hand, you can use L4 IPC and something that's CORBA-like. But then you'd be limited to native L4 binaries and Linux binaries. (And last I cecked, the latest kernel ported as an L4 server was a 2.2 kernel.)

      • As long as you have two rings, you can emulate an infinate number of rings. x86 CPUs have 4 rings.

        This is a bit offtopic, but to answer the previous question...

        Basically, to do CORBA-like operations fast, you need to pass a portion of the stack across a protection boundary. Part of the stack should be read-write, part should be read-only, and most should be inaccessable. In addition, you need a way to pass descriptors of memory areas (i.e. segment descriptors) across. Seldom are more than one or two such segments required.

        I once looked into using the call gate machinery on x86 machines and the L4 object model to implement safe calling between mutually mistrustful objects along the above lines. It can almost work, but the hardware doesn't quite do the right thing. The ring machinery basically assumes that one side is more trusted than the other. It's not peer to peer. Very frustrating. It's really close to what you need, but not quite there.

        Without hardware support, you need to do copying. And on most popular operating systems, too much copying. What you want is a subroutine call across address spaces, but what the OS usually gives you is an I/O operation. QNX is one of the few major exceptions, and even QNX has to do copying of messages. But QNX has very low overhead for interprocess message passing, and everything else uses that primitive. That's the right approach when you really need it to work.

        Understand, there's no problem doing this stuff either insecurely or slowly. But fast (without a trip through the kernel every time) and secure (one side can't clobber the other's memory) requires some hardware assistance. So few programs are architected that way.

  • Tcl certainly isn't as cool as Perl, but the implementation is very, very beautifully written C, and combining C and Tcl is a pleasure. It's fun, simple and easy. You also have access to a lot of neat internal features.


    Perl, IMO, is the worst of the scripting languages to combine with C... the interface is not pretty. Other languages like Python aren't bad. Lua is good if you want something really small and fast.

  • by DEBEDb (456706) on Tuesday October 22, 2002 @02:30PM (#4506056) Homepage Journal
    People who know C probably will not have
    problems learning Perl, but I always thought
    it's more fun to learn new way of thinking
    that comes with a new language. Thinking
    of a new language in terms of the one you
    know may result in taking the old-language
    style of programming to the new one - so
    what's the point of learning?

    You can probably explain LISP in terms of Basic
    concepts, but that's a waste... :)
  • by Anonymous Coward
    That's right! Perl .NET Edition Service Pack 2 Revision 3 Professional Version! (OEM)
  • <blurb>
    Ever marvel at how Perl can "do the right thing" but still be written in C?</blurb>
    No. I do marvel occasionally at how the Perl5 interpreter has been so constructed that it can handle pedantic, careful, robust, production-quality mannerisms without barfing on quick'n'dirty one-time script shortcuts. But mostly, these days I just take it for granted, and give thanks that a student of human language has built such a flexible language-to-instruct-machines. The underlying langauge of implementation is usually irrelevant, unless there's a need to connect to something that has neglected to provide a Perl API, in which case we can usually find a way via the de-facto machine-independent assembler, "C".

    But for code that calculates my taxes or bank balance, or drives the displays at the operating theatre if I'm brought in for emergency heart surgery, I want implementation languages where there is no question of ambiguity in the interpretation of the semantics of the code.

    --
    Perl, C, C++ (urgh) hacker. Don't tell anyone I can read COBOL.

  • Can anybody explain to me the beauty of a programming language in which it is legal to write obvious nonsense like

    Open(file) OR Die
    ???

    Thanks.
    • Try reading it out loud. It's only slightly more nonsense than English. (Perl's case-sensitive, though, so I'm assuming you've provided pseudo-Perl.)

      • Yes, that's it. To me it appears that Perl encourages you to blur spoken english with its legal syntax. Confusing to someone who's trying to learn it. The example I'm referring to should really be thought of as a boolean expression (since it contains 'or') but it must be interpreted according to how Perl evaluates it..

        So my original question remains: Why is this a good thing?

        • It's a good thing, according to Larry, because Perl more closely maps to spoken language patterns. His theory is that the computer should do extra work to make the life of a programmer easier.

          I don't understand your comment about interpreting that code as a boolean construct. That's exactly how Perl does evaluate it. See B::Deparse for clarification.

          • My objections against open() or die is that (in my opinion) the statement X or Y should not be any different than Y or X. But as Perl interprets it, it really does matter so the binary or-operation is not commutative... which is strange.
            But if you like it, you like it I guess..
            Thanks.
    • Why? This is a wonderfuly readable piece of code. You code, and perl will DWIM. I sat through a wonderful talk with Damian COnway, and one of his arguments against the oft-called for multi-line comments you get in other languages, is that the code should be self-documenting. There's just no way:

      open(FILE, "/path/to/some/file") or die "Sorry, couldn't open file: $!\n";

      ..would need a single comment

  • Frozen Bubble [frozen-bubble.org]
    This game does all the code in perl using the SDL perl bindings [sdlperl.org] (which makes the code as short as possible and easy to change) with special effects written in C (.xs file). Works like a breeze (besides it's addictive).

If you think the system is working, ask someone who's waiting for a prompt.

Working...