Compiler/Interpreter Validation Guidelines for C?

pjdepasq asks: "For my PhD work, I've developing a programming environment for novice users. It basically interprets the C programming language, plus or minus a few things as we teach the novice programming course here at Virginia Tech. In doing so, I'd like to have one of the students working with me use a compiler validation guideline to validate we're covering the key bases. For example, initially, I'd like them to ensure that the operator precendence is working correctly. My advisor tells me that NIST used to have a software product for doing this, but I don't see it there any longer for C. I'd really just be happy with some type of guide that the student can work from. Does anyone know of some type of free/cheap guideline we can utilize? It would likely need to include other types of validation, since we'll expand this effort later in the semester."

Re:compiler verification

[blair1q]

It would exercise, it, but validation suites are expected to be able to help you figure out what went wrong when it goes wrong.

Just throwing the compiler at gcc would break it in strange and unknowable ways.

And before it validates the first feature you implemented, it would break it in 500 strange and unknowable ways that you deliberately did not implement but would have to work around by hand.

Or was that your subtle joke?

--Blair

Language compiler validation services

[Ada95]

are available from EDS. See the following link: http://www.eds-conform.com/languages.html The services aren't free but the link may be useful anyway.

Feed it code from _The C Puzzle Book_

[marklandm]

What I would do in your case is probably feed the tool code snippets with known values. If you really want to stress operator precedence and such, I'd probably use examples from _The C Puzzle Book_ which have the advantage of having answers you can check. After that I'd probably pull examples from other C texts. It's not as pretty as running a validation suite, but it will probably tell you more.

Plum Hall & Others

[Bytenik]

The following are the most prominent commercial validation suites. It might be worth contacting them to see if they will grant you some sort of "educational use" license.

Plum Hall: http://www.plumhall.com/stec.html

Perennial: http://www.peren.com/pages/cvsa.htm

-----

Re:Cheap and cheerful

[aridhol]

If I was teaching a course, though, I would disobey C's precedence rules and implement sensible ones instead.

If you have a language that's "more-or-less C", follow C's rules. If you teach the students a broken set of precedence rules, you're not doing them any favours - you'll break any program that they make following your precedence rules.

Or [how about this] get the compiler to throw out any expressions whose bracketing makes operator precedence important? It's bad style to rely on C operator precedence, and here you have a perfect opportunity to teach that.

This, I agree with. I've maintained code that relies on precedence. When you've been doing it for a few hours, they get screwed up in your head. Nice, obvious parens make everything so much easier, I sometimes wonder why it wasn't a requirement in the first place.

Knuth's work and grammar as requirements

[BobTheWonderMonkey]

If you go perusing through Donald Knuth's work (in this case, Digital Typography), you will see him mention a killer program he wrote for TeX. I'd cite the program name and page number if I had my copy, but I lent it out. In any event, this single TeX program/script/file tested every little nuance of the system abusively. In and of itself it was a gorgeous piece of work.

Shifting gears for a moment, most QA types will tell you that the best time to devise test cases is when a project is in late requirements or early design. You doubtless have a grammar for your "C-plus-or-minus" language--I would call that a great high-level design document!

Now putting this all together, shouldn't it be relatively straightforward to construct a C program (or set of C programs) that goes through every state of your grammar?

Sounds like a fun project for some undergrad students to work on in Compiler Design. (In fact, it might even be fun to generate a C program generator from that grammar to do it for you; but I digress...)

gcc testsuite

[Hugo Graffiti]

Not sure what you mean by "guideline" but if you
mean testsuite then the latest release of gcc has
a testsuite which runs under DejaGnu.