Learning About Plug-In Architectures? 34
Pimpbot5000 queries: "I've searched high and low for a book/website/etc to get me up to speed on plugin architectures, but so far the pigeons aren't delivering. Where can a programmer go to learn about the different approaches, and their respective (dis)advantages? Most resources I've come across merely point you in the direction of creating plugins for existing projects, or quickly skip over the design phase and get straight to the 'and now you use dlopen()/dlsym() to...' part." I know quite a few plug-in architectures are language specific, but a resource that went over several schemes for each language would be a valuable thing for every coder's library.
Well, don't... (Score:5, Insightful)
A much better alternative is to offer a standard extension language, such as Python, Tcl, ELK, SIOD, Guile, lua, or even (the horrors) Perl. Now, most anyone of these languages have a plugin architecture, so if for any reason, you need native code speed, it can be done by writing a WHATEVERLANGUAGE-plugin.
The difficulty of creating a plugin architecture yourself over dlopen, is that you will most likely do it wrong at first try. A little bit less wrong on the second try, and so on... But each minor change you make to the plugin-architecture will probably break almost everything. So it's better to hand the problem off to the scripting-language-implementors...
Re:Well, don't... (Score:3, Insightful)
that can take dynamic extension is have to worry about the vaguiaries of some scripting language
since then you not only have to worry about your own bugs but bugs in the language itself. I had
to do something like that with Tcl. Never again!
The interpreter was so ridden with memory leaks
that we had to withdraw our software and rewrite it with tcl removed.
Anyway , so what if different programs have different plugin architectures. Would you expect
to be able to use your netscape flash plugin with MySql or something? I fail to see the issue.
Re:Well, don't... (Score:2)
point that using multiple languages always creates
a maintenance problem. Good design simplifies
code, rather than sucking in a vast body of
unmaintainable spaghetti, and forcing you to hire
botique language experts.
And really, if you write a plugin in SIOD or
Python for MySQL, it's not going to do you any
more good in Netscape than a native code plugin
would do.
Generic object services is what the first poster
is talking about, not plugins. Use IIOP or SOAP
for that sort of thing.
Having said all of that, the state of practice
in plugin architecture does lag the state of
the art by quite a bit, so this thread may shed
some very welcome light on the subject. I
suggest moving it to a development site, such
as advogato.
Re:Well, don't... (Score:1)
Are you serious? I've looked at advogato and that site is weak.They barely have any replies to the few articles that they post. And sure some of the comments are good but the net gain from reading all the comments is a tiny pimple. I wish there was a good site for this purpose but unless you're like arstechnica with Hannibal you're screwed. The basic fact is that you need someone really knowledgeable about the topic.
t.
Runtime aggregation. (Score:2, Interesting)
Question: How many of you have worked on a project that's designed your own damned COM-like system?
Re:Runtime aggregation. (Score:4, Insightful)
Hmm... I don't think I agree.
When I think "plugin," I think of an application loading and executing some code from a DSO or whatever at run-time.
When I think "CORBA," I think of two running applications communicating with each other at run-time.
They're fundamentally different things. If you were to design your plug-in architecture using CORBA, you'd have to have your programmers build little applications that would have to be started and to run alongside your main app, just so your main app could make remote calls to them. Seems kinda silly to me.
Re:Runtime aggregation. (Score:2, Insightful)
Thinking of CORBA or COM in the manner you are thinking of it will severely limit your uses of these technologies.
Re:Runtime aggregation. (Score:2)
It's an issue of address space. If I dlopen() a library and dlsym() a symbol, I can then call the function associated with that symbol and have that function run inside the calling program's address space, using the program code from the DSO I opened. It is, for all purposes, as if that code had been compiled into my program from the beginning. I can do things like pass it pointers and get pointers in return, because we're all running in one big happy address space.
Generally speaking, though, calls over CORBA interfaces (I won't speak about COM because I neither know about, nor are particularly interested in, it) cause code to be executed in another address space. Even if the other program is started at first reference, there's still another program out there, with a separate address space. Calling it requires a context switch, and it also (and more importantly) requires that you serialize and send any relevant data from caller to callee, and back again. If you're a program like Photoshop, and the plugin is going to operate on the image in memory, a requirement that you must pass the whole image to the plug-in process would be pretty dumb.
While it may be possible to use an ORB to broker calls to and from DSOs linked into your application at compile time, I'm not aware of any method for using the ORB to link in objects at run time. Of course there may be one, but I'm not aware of it.
Generally, I'm coming down on the side that says CORBA (and possibly COM, if it's fundamentally similar to CORBA) is great and cool, but utterly and completely inappropriate for writing plug-ins, in the traditional sense.
Re:Runtime aggregation. (Score:2)
Actually, CORBA is bad for plug-ins, but COM is great.
That is beacuse COM has in-proc servers, meaning that it is on the same adress space, and is extremely fast.
Most of the plugins for MS products are COM components that implements a particular interface.
Re:Runtime aggregation. (Score:2)
See? There you go. I didn't know that. I therefore reframe my opinion: CORBA is bad for plugins, and COM is just plain bad.
(I was kidding. Don't post anything about how COM is okay. I have no opinion about COM.)
CORBA for in-process servers and plugins (Score:1)
I wonder why you would choose CORBA if it was to disable all of the features that make it CORBA. Compatibility with other ORBs might be a reason, but I don't think that the kind of objects that you might want to have in-process and so on would be any good on other ORBs.
Some CORBA proponents will often claim that out-of-process is safer anyway, so that if you have a plugin crash, it won't take down your application, but most applications that do it that way becomes more or less confused after a plugin crashed, some to the point of crashing, so I don't see that being *that* much good anyway.
Re:Runtime aggregation. (Score:1)
Look for something like VBA (Score:1)
Integrating VBA is a bit costly, so you may try Microsoft Scripting Runtime (scrrun.dll) instead, its free. You will still be able to use application's object model from vbscript.
What's the purpose? (Score:3, Insightful)
In talking about a 'plug-in' architecture, from what viewpoint or for what purpose are you looking at the problem?
The first possibility I forsee is that you are developing an application which requires extensibility via third-party modules. In such a case that only design I am aware of (or rather, than I can think of at the moment) is to specify one or more APIs which the plug-in must implement, and then have the facility in your main application to register binary libraries. The application queries the libraries for the API(s) supported, and slots it in somewhere for use at the appropriate point.
The second possibility is that you are developing a plug-in architecture for a language, for others to use in a generic fashion. Your best bet is to study the way this is done in existing languages, and (as many other posters have said) don't do another plug-in architecture for a language that has one.
I had a third possibility in mind, but it seems to have been taken by a pigeon ;)
Re:What's the purpose? (Score:1)
The initial step seems to fall nicely within the realm of defining a plugin interface and using dlopen()/dlsym(). The second step isn't quite as clear, my initial thoughts tell me the extension language approach is best, because I don't fancy myself capable of designing a generic API for all debugger functionality. : )
And while direct suggestions and tips are great (and much appreciated!), I prefer to see what all the options on the table are, and do my best to not reinvent the wheel (nor the spokes, nor the inner tube, nor the valve stem). : P I'm fairly surprised this sort of information hasn't been aggregated in some place, because it must have been the cause of many many needlessly repeated hours of thought/effort/frustration/re-implementation.
-Greg
Re:What's the purpose? (Score:2)
architecture, with dynamic typing. Let each plugin
describe its methods with a snippet of XML, and
let the plugins communicate using XML method
invocations. Implement the application as a plugin
which obtains a thread of control from the
container and scripts the other plugins. When the
container starts up, it can load and initialize
all the plugins, including the application logic
plugin. By using XML interface descriptions, it
becomes trivial to implement inheritance between
plugins, and by using XML invokations, it becomes
trivial to make the component architecture network
transparent.
Hmmm.... this sounds familiar....
Bamboo (Score:3, Informative)
Bamboo home page [watsen.net]
Re:Bamboo (Score:1)
Curious...
http://watsen.net/Bamboo/
Early 1999 to Present
In the true spirit of Bamboo, we try to defer as many decisions as possible regarding the application until runtime. Sometime in early 1999 it occurred to us that there was one last assumption we were making: C++. Although we were using other languages with Bamboo (a Java AWT GUI and some Python scripts), their code segments were embedded into C++ modules. That is, is was not possible to have a 100% Java (i.e. platform- independent) module, which is rather troubling for some Java purists. We are now in the process of correcting this omission with a brand new architecture that simultaneously attempts to solve some other issues. For instance, we are replacing ACE with NSPR (the Netscape Portable Runtime), adding automatic code documentation (Doc++), adding printable docs (HyperLatex), a bug-tracking database (Bugzilla), searching of both the online docs and the bug database, and an online SCM (Perforce).
What the hell are you talking about? (Score:2)
Bam. There's your plugin "architecture". What is the problem?
My own design (Score:4, Informative)
The key design element method is to make it object oriented, take the class's interface as your API, and bang, you've got a plugin interface.
A little info on my design. I was working on a driver to interface with a custom communication board. Well, we had an original version of the board that we abandoned, a later version which was really used, a test board for that one, and then one or two more.... Each board did communication with a companion board on another device.
My design was to abstract out the UART control, the low-level protocol, the FPGA control code and a couple of other things into seperate interchangable objects. These were objects, but implemented in C, since it wasn't possible to use C++ in the kernel on this platform. Each of those components were configured and plugged into the system at driver load time.
Now, I could have made this slightly different, by pulling out the individual components into seperate modules, loaded as needed, but in this case, that would have been overkill. But the principle is identical.
Back to your original question. I didn't have any books, or other guides to assist with the design. The design fell out of the original needs, after a reasonable amount of refactoring, and a bit of good architecture skill.
In your case, without knowing more details about the application, it's hard to help much. If this is for a general use application, your API will probably be insufficient the first time, so it would be worth your time to think about how you will handle large scale changes to the api. (Are you going to maintain backward compatibility? Are you going to support multiple APIs at the same time? Are you going to add additional specific interfaces over time?) But the core idea again is an OO design where the plug-in is just an object.
focus on design principles (Score:5, Informative)
You'll find that the more advanced types of plugin mechanisms are usually implemented in Java. This is no coincidence because Java has a few mechanisms built into the language that enable these mechanisms: reflection (i.e. discovering what methods/properties a class has at run-time), classloaders (load a class at run-time and let it run in a sandbox, destroy classloader to unload the class), dynamic linking (classes are resolved at run-time rather than compile time). An architecture that uses all of this is Jini. Often this is seen as a failed Sun project but the design behind Jini is still very cool.
A final word of advice: don't invent your own plugin mechanism but reuse existing ones.
Re:focus on design principles (Score:1)
But then, Java is of the same language family as Smalltalk and Objective-C, only with a C++-style syntax, so I suppose this is no coincidence either (that Java has similar mechanisms to Objective-C and Smalltalk).
Re:focus on design principles (Score:2)
Java was created by people who worked on smalltalk and self in the eighties so it is definately no coincidence that it has a lot in common with those languages.
Some component systems (Score:1)
You have XPLC [sourceforge.net] (shameless plug, that's the one I'm working on), CrystalSpace's SCF [sourceforge.net] ("Shared Class Facility"), theKompany.com's Korelib [thekompany.com], as well as others, including the previously mentioned Bamboo and Bonobo, but others like KDE's KPart/XPart, Mozilla's XPCOM and OpenOffice's UNO. You might even count Microsoft's .NET as well, if you want to.
Clemens Szyperski [microsoft.com]'s book, Component Software - Beyond Object-Oriented Programming [microsoft.com], is an excellent book on that subject.
Re:Some component systems (Score:1)
Thanks for the links. : )
Hopefully to the benefit of everyone, I'll mention a couple sites I found (mostly through Pierre's book link [microsoft.com]) which contain a lot of potentially useful info:
Anyone know what the dealie is with Korelib? The idea looks excellent, and potentially very useful for myself, but the project seems not to have been touched for over a year.
-Greg
Re:Some component systems (Score:1)
If you like Korelib's idea, XPLC is pretty similar, with maybe exception of the UUIDs instead of strings as component identifiers.
XPLC is getting more development time than before, since my employer, Net Integration Technologies (http://www.net-itech.com/) is allowing me to work a full day per week on XPLC, in addition to what I used to do on nights and weekends.
On the other hand, Korelib seems to be somewhat more usable currently (XPLC is generally usable, but dynamic loading of components is not in yet, so it's usefulness is limited), being used in some of theKompany.com's applications.
jEdit (Score:3, Informative)
It's all about control (Score:1, Funny)
Make sure you leave yourself an option open so that you can screw the developers [suck.com] later, if necessary.
Architectures in general (Score:1)
This will give some insight into component architectures in general, and the way that different approaches are applicable to different modes of use.
You need a pattern or two, not an architecture (Score:2)