Designing a Programming Language For Embeddability

CowboyRobot writes "The creators of the Lua language describe the process of designing a new language and the constraints that certain parameters, specifically embeddability, place on the process. 'Many languages (not necessarily scripting languages) support extending through an FFI (foreign function interface). An FFI is not enough to allow a function in the system language to do all that a function in the script can do. Nevertheless, in practice FFI covers most common needs for extending, such as access to external libraries and system calls. Embedding, on the other hand, is harder to support, because it usually demands closer integration between the host program and the script, and an FFI alone does not suffice.'"

Re:Oh good

[pem]

It's 18 years old. If you don't care to know anything about existing languages, why would having newer ones bother you?

Forth

[smoothnorman]

Forth http://en.wikipedia.org/wiki/Forth_(programming_language)/ [wikipedia.org] was designed to run on (albeit early) embedded platforms. Extensible, easy to learn, easy to implement. I guess it's just the RPN that scares folks off? ("no you old fool! modern processors are optimized away from stack intensive use!")

Re:Lisp

[Darinbob]

Because the host language may be a conventional language, designed to compile efficiently, run in small amounts of memory, has speed critical sections, etc. The embedded language on the other hand is used for doing stuff that wasn't thought of in the design phase and where it's simpler to add a small script than to add a new feature to the program. Generally what happens is that you're tying together a high level language with a low level language.

For instance, take a boot loader. It is written to fit into a small area of flash or just a few blocks of mass storage. So it needs to be small and tight. In manufacturing you'd like to run a set of tests and the boot loader provides some simple commands to call that test various features. But maybe you want a particular test that the built in commands won't do, for example a stress test of memory. The default method is to have development write a new command, check it into source code control, have QA run through it, approve it with the release schedule, then release it to manufacturing. Alternately one could just write a quick script and use that instead if the boot loader supported a scripting language, and manufacturing could manage this themselves without begging for R&D resources. Every time a bug occurs in the process someone could write a new regression test using the scripting language to check for the bug and add the script to a growing collection.

Now if we could somehow get fast/efficient/tiny host programs written in Lisp (which is indeed possible) and we manage to find a collection of developers who understand it (which is harder), then it makes sense to have the scripting language the same as the host language. But in the mean time most host programs are written in things like C or C++. Even if you have a high level interpreted language like Java that you wrote your host program in, it is very difficult to parse/interpret textual Java programs.

Even if your host program is in Lisp it will be difficult to isolate any textual Lisp scripts from the host program itself; you don't want a buggy script to cause havoc with the host program. Most Lisp systems I've seen that allow extra scripting basically use "eval" which is not very safe. So you still need to put up a solid wall with a well defined set of methods to allow a script to run in its own sandbox while interacting occasionally with the host program.