Follow Slashdot stories on Twitter

 


Forgot your password?
Close
typodupeerror
Programming Software IT Technology

Pragmatic Programmers on Designing with Metadata 26

Posted by chrisd from the interesting-writing dept.
Bill Venners writes "This week I've published the fourth installment of my interview with Andy Hunt and Dave Thomas, the authors of the best-selling book, The Pragmatic Programmer. In this installment, Dave and Andy talk about their recommended approach to design in which details are pulled out of the code and stored as metadata. This installment of the interview really made me think. Their focus on metadata sounded non-intuitive when I read their book, but in actually talking to them about it, I got the feeling they might be on to something. Check out: Abstraction and Detail."
This discussion has been archived. No new comments can be posted.

Pragmatic Programmers on Designing with Metadata More

Pragmatic Programmers on Designing with Metadata

Comments Filter:
  • I don't see what is so special about the idea of separating metadata from the rest of the code. I think this is what experienced programmers often do. (I also think that the authors have not really understood XP, as it is not a style of programming, but a style of working in the first place.) Having to think about what to put in "code" and what to put in "metadata" is really deciding what should be "compiled" and what should be "interpretted". The are two reasons why code should be compiled. Firstly, becau

  • I am not sure where they got their definition of metadata, but it sure isn't what I mean when I say it. From the article:

    When people read our advice in the book about metadata , they tend to imagine very complicated architectures with lots of abstraction. But in reality, it could be very simple. If the sales tax rate is currently 7%, I don't put 7% into the code. I put it into a properties file or the database. The sales tax rate is a detail I abstract out of the code and store externally.

    Uh. He's t

    • He's talking about storing data externally. This is his idea of metadata? Sounds more like 'data' to me.

      Well, of course metadata is data. The important thing is to design out as many assumptions as possible so that they are represented by data rather than code. We used to call this "data driven programming".

      Example:
      Years ago, I had a huge backlog of requests for different data sets out a database. I abstracted out the common aspects of the requests, built an engine that retrieved the data based on a

      • Well, of course metadata is data.

        I think the distinction is between data that's to do with the objects the system models (e.g customers & invoices if it's a billing system) rather than data that makes the system behave in a certain way. The former I would call 'operational', the latter 'configuration'. For me, metadata means schemas and suchlike.

        Some have great difficulty eeing the distinction; a rule of thumb is if end users mess with it it's operational, if only specialists/analysts/etc do it's

        • For me, metadata means schemas and suchlike.

          Right -- a relational schema is metadata for a relational model. A state machine description is also a schema for a state machine model.

          But really, I don't see that they're saying anything new, it's all in 'Code Complete' which is v. old now.

          Sure, it's an old idea. HOwever it is not one that has been absorbed by nost programmers.
          • Sure, it's an old idea. HOwever it is not one that has been absorbed by nost programmers.
            Yeah, I totally agree. Common sense isn't that common.

            I find it interesting to see discussions about OO and inheritance, components and all the rest of the reuse armoury, when 80% of programmers never learned how to use includes properly.

            P.S. I like the sig.
    • When someone buys x[a fish tank] apply function[sales tax] with input y[7%].

      That's kinda metadata, more a function, but it's often hard to tell the difference between data that describes data and a function.

      input y is looked up as in input for function sales tax.
      list values for y have metadata that says they are possible inputs for sales tax.

      sales tax has metadata that says it can be applied to a product and/or a product has metadata that says salestax can be applied to it.

      It's important not to hard cod

  • wow (Score:2)

    by GiMP ( 10923 )
    I'd never think about storing data and loading it dynamically by my program instead of hardcoding it.

    I think that storing everything possible in XML or a database is a good thing. Perhaps he is referring to the coding style of outputing everything in XML and use XSLT to convert it as required. This is an incredibly flexible and scalable technique with only slightly more initial development time and effort.

    For webpages, I can have my application do a SQL select and output the data as XML. The XSLT templat
    • But the XSLT is a part of your application. You don't have to go through a complete edit-compile-link circle that way, but you can have that with a lot of other programming languages, too. (Yes, XSLT is a programming language, just a rather specialized one, with very verbose syntax and not too much helpful tools like debuggers or profilers).

      This isn't about strictly separating code and data, it's about using different tools for different layers of your app to blur this distinction, which is a good thing.

    • One technique for programming with meta-deta is data dictionaries [geocities.com]

      It takes a while to hone the usage of such things, especially WRT handling unexpected requirements, but there is a point where they start to pay off handsomely in my experience.

  • Rules, not metadata (Score:3, Informative)

    by Twylite ( 234238 ) <twylite@cryp[ ]o.za ['t.c' in gap]> on Thursday March 27, 2003 @04:49AM (#5605055) Homepage

    Maybe I've got the wrong idea from the interview, but what was discussed was rules, not metadata.

    Business rules are a well known aspect of enterprise software development, especially in light of the many old(er) custom-build systems in which the rules were hard-coded. A business rule is "sales tax is 7%", or "customer pays a 1.5% surcharge is payment is more than 2 days late".

    Metadata is a partner and also an opposite to a business rule. Metadata is quite simply "data about data". The fact that the value "7%" is "sales tax" is metadata; but the fact that the current value of the sales tax is 7% is not. The age-old concept of a "data dictionary" is an embodiment of what metadata is.

    A rules engine is (rather simply) a powerful extension of the practice of declaring constants for significant literals (which are or could be subject to change); quite often one which allows runtime modification of the value rather than requiring a recompile. Rules engines also tend to provide mechanisms for evaluating compliance with the rule, or performing calculations based on rules.

    • We used to do that for e-commerce.
      You could setup differnet rules in the database, the rules were run as SQL or JavaScript, or a call to an external ActiveX Component.

      There were some user configuration parameters and other bits and bobs so that you could configure everything from a whois do domain name registration to complex multi purchase pricing rules.

      It made for east site design, maximum code reuse and the ability for an end user to configure their area of the system.

      Now, if someone applied that to a
    • There is more to it, though.

      The 'Metadata' should include the sales tax algorithm, not just the percentage.

      For instance, the crazy Goods and Services Tax (GST) here in Canada has weird rules in it like: 3 donuts are taxable, but 12 donuts are not. Encoding a plain 7% in the 'metadata' is not sufficient.

      --jeff++

  • To summarize the article.

    1. The meta data is data concerning the business functions encoded, not what is commonly refered to meta data in programming, i.e. data concerning for example object heirarchies and such.
    2. The article simply says: do not encode "known" constants (such as tax levels, etc.) into the code, but put it in an external XML database.

    Any professional programmer somewhat experienced (i.e. anyone who has run into a respecification of a constant occuring in 52 locations thoughout 30000 lines of

    • Bad Summary (Score:3, Interesting)

      by Jerf ( 17166 )
      The article simply says: do not encode "known" constants (such as tax levels, etc.) into the code, but put it in an external XML database.

      No, that's not what it says. It says do not encode known behavior into the code, but put it in some more easily changed external data source. Also, it might be XML but it can also be in the code, just code structured more like data then code. (I often write code-data like that, writing default keymaps in the language itself, for instance. It's easier then writing a cust
  • Andy Hunt: But for me personally, almost every time I've taken that extra care to make a system flexible, it has saved me.

    There's an effect that I've named (what else) "Hollan's Law": The liklihood of something changing is directly proportional to the intensity of the argument that it never will.

    Thus, when we see something in code that looks like it might need to be maliable, it probably will be.

    Looks like these guys have noticed the same thing.

    Ultimately, good programming is about finding the clear

  • This is not a new technique, but not used as much as it should be: I've used code generation in several projects, and I think that superior of metadata and property files/databases for cases where the property file would consist of a large number of fields, that are repetitive. For example: a big finite state machine. The code generation tool is always written by hand, which is the tough part. After the code generator is finished, you can leave much of the work to less experienced programmers, or even non-p
    • Hard coding, storing (meta)data in a database, embedded scripting engines, virtual machines and code generations, are all techniques (each with their own pros and cons) for implementing functionality in an execution environment.

      Most programming is still done in low-level programming languages. IMHO, we really need systems/languages in which we can transform high-level specifications into executable code. This kind of tools would allow you to select any of the above techniques based on what suits you best.

  • I think he is talking about code implementing rules which are likely to change so he decides to implement a state machine for it. At this stage I wonder if it wouldn't be simpler to just change the code than to program transitions of a state machine in a database :)

    These kind of approaches inevitably end with the programmer inventing a new, personal language for expressing the rules and writing a small interpreter for it (which he basically did).

    The same result and much more can probably be achieved by

Slashdot Top Deals

If mathematically you end up with the wrong answer, try multiplying by the page number.

Close