Where Does the Business Logic Belong? 92
logic-Dilemma queries: "I'm currently working in a big project that involves creating tons of reports. These reports require extensively data operation and manipulation in order to be build, and most of that can be handled directly by the DBMS (which would greatly increase performance and implementation time). However, letting all business logic sit in the database implies that we will be extremely attached to one vendor, which kills any attempt at portability. What would you do to tackle this dilemma? Have you ever faced a situation in which the choice between clean design/portability versus performance would change dramatically the whole system design? What have you chosen?"
It begins with OGG support. (Score:1, Funny)
psql, perl, tcl (Score:2, Informative)
Re:psql, perl, tcl (Score:5, Insightful)
I LOVE Postgres, but the stored procedure portability is not its strong point.
To the best of my knowledge, Oracle and MS-SQL server do not support Perl, TCL, etc. I know Oracle supports Java, but I haven't ventured into that territory.
Oracle uses PL/SQL, MS-SQL server uses T-SQL, and Postgres uses PL/PG-SQL. All of these are different enough that you would have no hope in hell of being able to do a straight recompile. Esepcially when it comes to date handling and things like that each database does it differently.
You will have to decide what's more important: speed or portability. If it's speed, choose the native stored procedures in PL/SQL or whatever your database uses. If portability is your main concern, stick as close as possible to the subset of ANSI SQL that most databases support. Check and see if the SQL constructs you're using work in other commonly used databases.
Finally, keep in mind that Oracle does not have the notion of "databases" the way that MySQL, Postgres, and MS-SQL server do. The closest thing is to have a seperate instance of Oracle running in its own memory space with its own "SID" identifier (Oracle is a wonderful but complex beast!) The reason this may be important is because you might find down the road (as we did) that when you try to mix your 8 unique databases into one Oracle instance that you have conflicting table names. We solved this by putting a three-letter prefix and underscore on each table name to describe what logical application it belongs to. Also, Oracle table names can't be longer than 32 characters, so keep that in mind as well.
My recommendation for portability is to use one database with prefixes on the table names, then have a Perl (Java, whatever you're using) API that returns / accepts the appropriate data structures.
Good luck!
Re:psql, perl, tcl (Score:2, Interesting)
It has the notion of "schemas," (aka users) which are analogous to "databases" in mysql and postgres. In fact, from a user's perspective, they're almost identical. There's no need to put a three-letter prefix on your table names- you can just do schemaname.object, just like in other DBs.
Furthermore, you can create synonyms to tables in other schemas, or even PUBLIC synonyms that allow you to access a table from any schema without
Re:psql, perl, tcl (Score:1)
You can go so far as to code your application to execute "ALTER SESSION SET CURRENT_SCHEMA=blah" right up front and from then on out, your "database", i.e. schema is configurable. You can have a schema called BLAH01 with a version 1 database and a schema called BLAH02 with a version 2 database and your client can just access either one as needed. Schemas are MUCH more elegant than "databases" in SQL Server. They force ownership of data to a user account instead of the system. I
SQL Server has this as well (Score:2)
Re:psql, perl, tcl (Score:3, Informative)
J
Re:psql, perl, tcl (Score:1)
Re:psql, perl, tcl (Score:2)
I don't know if you were referring to connectivity or to being able to have the database understand the languages within the database, though.
Re:psql, perl, tcl (Score:3, Insightful)
Procedural sql is reasonably portable.
I don't know about other RDBMs, but postgres supports perl, tcl, and others.
That portable enough for ya?
Very simple (Score:5, Insightful)
Follow the priority list and go from there.
If the priority is minimize development time, that might tell you one thing, if it is to maximize portability that might suggest another.
For example if you are a pure Oracle shop, have been for 10 years, would never port off of it - why should you care that your logic is implemented inside Oracle. If you are prototyping on a new database, and have no history with it (or any other product) you had better come up with a different answer
Re:Very simple (Score:2)
Re:Very simple (Score:3, Insightful)
Now this may be acceptable. The OP is correct that it all depends on priorities. You may decide that having a tight, fast system today is worth a potential (even if unlikely) hair-pulling refactoring if the DBMS must be swapped.
Because Upgrades Break Software! (Score:2)
It's not like a company is never going to upgrade their databases! Once an upgrade happens, the underlying code has changed... This requires you to retest your application.
Would you rather retest reading data from a table and making sure it hasn't translated into Swahili or retesting all the complicated Business Logic of your application?
I prefer the former...
PL/SQL (Score:2, Interesting)
Definately don't put it into the application.
Re:PL/SQL (Score:2, Insightful)
Re:PL/SQL (Score:2)
Ideally, any system should have a Datalayer, the Engine and the Presentation.
Start from the assumption that you can do all this (and thereby do the BL in the Engine instead of the Dlayer), and modify only if it's necessary (e.g. boss won't give resources to write the necessary Engine components.) If all/most of the necessary logic is in the DB already, that's another thing to think about (you can also of course just rip
Re:PL/SQL (Score:4, Insightful)
In some cases it will be easier to maintain in an application than on the database, in other cases vice versa. Also consider which spot gets you the easiest access to version control. The only other guarantee besides change seems to be the reversed decision.
Re:PL/SQL (Score:2)
Putting it in the application and using a generic db interface should make it portable, but in practice it never does because there are inevitably lots of little quirks.
Using a db's procedural language is bound not to be compatible, but is far easier to port, IMHO, than get the bugs out of an application.
I second the concept ;-)
Re:PL/SQL (Score:2, Interesting)
Remember that the business rule you build into a report will probably be needed in an interface one day (ie: taxation on a charge of some kind), and that the tax rules will change much more qui
Re:PL/SQL (Score:4, Insightful)
How is a business rule in an application cleaner and "API-like"? Do you mean creating a library (e.g. perl module) of routines that implement policy? How is that "cleaner" than calling a stored procedure in a package? Have you used stored procedures and databases as back ends to applications?
Regarding your last paragraph, that's the whole reason why this topic is here. Everyone knows not to build business rules into the interface. The question is where to put them instead. That's what we are discussing here.
If I can boil down your post, I think you are saying that you prefer coding business logic in a library of a particular language. What language?
Re:PL/SQL (Score:1)
Of course you mean: The Applications. In many cases, databases get accessed by more than one application. In these cases you have multiple problems when you put the business rules in the applications, not in the database.
Jobs (Score:5, Funny)
Just last week I was offered a job porting a bunch of business rules written in MS SQL stored procedures to Oracle stored procedures at a nice fat billing rate.
Fight unemployment! Use stored procedures!
What is even better is when they break between releases from the same vendor. Ka-Ching!!
Don't worry about portability (Score:5, Informative)
(Bear with me, it's not a troll).
10+ years of experience in the Financial/Banking sector might not tell you much, but it has taught me that business/domain logic changes so frequently so as to make any possibility of portability be remote, in the best case. Stored Procedures just save your life, period. Also, and perhaps more relevantly, your client probably has made a substantial investment on the RDBMS, and they won't even dream of switching DB layers down the line.
If you're confident about the choice of RDBMS vis-à-vis its architectonic permanence (Is the client happy with it? Are you sure it's gonna sustain the load you plan?), you shouldn't worry about portability in, say, mid-termish 3 years after installation. However, if what you want is to re-sell (that is, productize) the code to another potential client who might have a different RDBMS, your design goals should be adjusted accordingly (for instance, you can insist on portability, by building a middle-tier; or you can push for a RDBMS you know you can pitch together with your system to whomever you want to sell it afterwards).
Re:Don't worry about portability (Score:1)
Re:Don't worry about portability (Score:1)
I would almost never believe that (Score:2)
I admit that developing for something portable to begin with (java, parrot, wxwindows, gtk+, or wh
Re:Don't worry about portability (Score:1)
Not only does the back-end change less frequently, I find the front-end changing every now and then, depending on what's the latest you can hire cheap developers for. Often the type of reports and their mode of presentation also changes. Pushing business logic into the back-end allows you to keep up with the most appropriate front-end techno
Ever heard of a DBMS upgrade? (Score:2)
Eventually, an upgrade will break your software. So, portability is a concern even if you're CIO is blowing Larry Ellison in his spare time. Diff
Do you need portability? (Score:5, Interesting)
If you're going to try to market this as a product, then concentrate on devleoping the business logic, after all, that's what businesses want to buy. They'll pay you to port it to their database or just buy a copy of the database you've written it for, if it's valuable enough to them.
I've been working on apps like this for years. Just stick it in the database. It's so much easier than maintaining a bunch of query engine code or mappings so you can keep your precious business logic in a "programming language". If your using Oracle, you could just write it in Java and install it on the database. PL/SQL or whatever you're writing the business logic in will probably be around longer than any app language like
All that being said, this approach works best if you're using a database that has support for stored procedures, embedded code and custom types, either one of the commercial biggies (Oracle, DB2) or PostgreSQL. Firebird (or whatever they're calling it this week) might work too. I wouldn't trust MySQL for this type of work yet though, I don't think it supports code in the database all that well yet.
Personally, I think databases are going to wind up absorbing application servers like J2EE containers and will eventually look like a relational/object hybrid with interfaces to various protocols and container environments. After all, those engines are pretty simple to slap on top of a good database. Oracle and IBM are already moving in this direction somewhat. Oracle more than IBM. I think MS is going to move this way with SQL Server as well, but of course it will only be for
Does that help?
Microsoft ObjectSpaces Initiative? (Score:2)
Personally, I think databases are going to wind up absorbing application servers like J2EE containers and will eventually look like a relational/object hybrid with interfaces to various protocols and container environments.
Does anyone have any experience with Microsoft's new ObjectSpaces persistent object initiative for .NET? Some overview here:
The ["scientific"] data we're generating is very large, and mu
Re:Microsoft ObjectSpaces Initiative? (Score:2)
Oh, wait, I forgot: most database vendors don't care much about giving you user-defined-types. Maybe functions, if they're feeling nice.
I work with Interbase/Firebird (on-topic: plenty of SP, trigger stuff available, though I rarely use it) -- I know they say that, well, basically, you can do this yourself as a BLOB (hey, it's binary data, right?) and massage your own stuff however
Except for one thing... (Score:2)
Relational database theory in no way prevents you from having any types you like in the database (current maintainers of the academic theory would point out that "object/relational" databases are just relational databases working the way they should.)
Except for one thing: SQL [like Java] is a 32-bit language!!!
SQL BLOBs are 32-bit entities!
Java arrays are 32-bit entities!
WE NEED 64-BIT DATA TYPES AND 64-BIT PROGRAMMING LANGUAGES!!!
Re:Except for one thing... (Score:1)
Couple of things (Score:4, Insightful)
Business Intelligence and many widespread corporate reporting tools are often subject to SOX regulations. The Risk Management guys will usually have an opinion about the portability vs performance issue for you. (Also, these guys are way way way too busy this year to artificially create more work for themselves. Every publically traded company in the US is trying to get these guys to sign off before the year end.)
Second, look into some of the XML data portability ideas and solutions out there... I can't name anything off the top of my head, but there are companies out there with enterprise and off the shelf data portability options. Also, if you're a flexible development team you probably have someone who can cobble together something good using XML. Google XML Data Portability and you'll get started on pointers.
Use Hibernate (Score:5, Informative)
Further, Hibernate is battle tested, and used in the real world.
I choose Open Source (Score:2)
Re:I choose Open Source (Score:1)
i like logic in the DBMS (Score:1, Insightful)
The COM middle tier business logic MS was promoting in the 90's was a joke. I found it to just be another layer of abstration with no benifit.
If you place everything in stored procedures you get a very centralized app that any client application can connect to (web, java, etc...)
If you're worried about being locked into one DBMS platform, try to write all your queries and stored procs with standard sql. Avoid database specific features, the
Re:i like logic in the DBMS (Score:2)
You obviously have no friggin idea what good software is. MSFT wasn't the inventor of this architecture. It is a very old concept that was simplified (some would say not so well by COM and I agree...)
Putting everything in the database sets you up for failure in so many ways... First of all, what happens when the DB Server needs to be upgraded???? YOU NEED TO REGRESSION TEST EVERY SINGLE APPLICATION IN YOUR CORPORATION!!!! That's ridiculous!!
You should have Unit tests
Re:i like logic in the DBMS (Score:1)
...until you upgrade the application server, right? And you're sure t
Re:i like logic in the DBMS (Score:2)
Everyone knows the sign of a debater who is confident in his argument is a parting insult... I've done software design for over a decade. I didn't pick this stuff up from a magazine.
When you scale an application by adding "application servers", you re-test all the software on that server before making it live. The same thing is done for u
where do you want control? (Score:2, Insightful)
Tons of reports? (Score:3, Interesting)
Ahh, there's your problem right there.
Now, the reports are static, meaning they are standards, run at specific intervals.
Later, though, and much sooner that you expect, the requirements will change to doing a lot of data manipulation on the fly.
If you want to make a truly workable system, design an interface for user-wuery creation, and allow them to save their queries, in custom/personal folders. Also allow them to share their queries with others.
I know, it's a pain, but it gives you extreme flexibility in the long run.
A plus is to allow the reports to be converted to png graphics on the fly, so that they are very hard to modify.
Another plus is that you can standardize the SQL and thus ensure compatibility across multiple data sources.
Watch out: make sure the presentation language is portable itself and has a long run-life. PHP and PERL are best. Python is good, but it's harder to find programmers. Java is poor for that, but serviceable. Also, SUN isn't in the best shape these days, so Java's future is shaky 5+ years from now. You can also use jsp, but same applies.
Go for Apache on the web server. Can't go wrong there. If Java then Tomcat.
Stay clear of asp and
As far as stored procedures, ok, but be really careful: they are hard as hell to debug after a while.
If you must use stored procedures, document the hell out of them , make nice diagrams, and do separate text documentation explaining the reasoning behind each one. (this in fact applie to all code written by your shop. But you knew that.)
Finally, take a look at openoffice and its db access. It might be that what you are trying to do might be feasible with it.
Re:Tons of reports? (Score:3, Insightful)
Ahh, there's your problem right there. Now, the reports are static, meaning they are standards, run at specific intervals.
When a business asks for "some reports", it's an indicator that they only have an implicit, at best, understanding of their own business process. They treat the computer as a big bin in which to dump "facts" and generate some output in standard formats which they use to do manual analysis and processin
Re:Tons of reports? (Score:2)
Clean `n Portable - At Least to Start (Score:4, Informative)
Have you ever faced a situation in which the choice between clean design/portability versus performance would change dramatically the whole system design?
Yes, I have.
What have you chosen?Clean and portable.
Reasonable performance right now when you first write your code is good enough.
More important is to write maintainable (readable and understandable) and extendable code.
We have high performing applications written 10 years ago that sit on the scrap heap that few people know or love these days because they were tuned too tightly to getting performance in a very specific situation. But situations are always changing. Highly optimized applications are too specific and too fragile to be useful very long.
Meanwhile, an originally clean design that was of only adequate performance has been used and used and used. And it's become bloated and creeping with all kinds of ugly useful features simply because people could easily add them on. Once there was a rewrite to improve performance when resources were available because a lot of people were using the code.
Here's the irony: lots of surgical scars on old code is a testament to its successful conception and to its continued usefulness, as well as being a motivation for replacing it with the next generation code, which is what you usually hear about at the watercooler.
Portability (Score:3, Interesting)
Reports and other visual output generated at the client using simple data structures spit out by a procedure (middle-tier) layer.
Java/Ruby/Perl/Python/C++ (gasp! adjust for efficiency) through XML-RPC or SOAP works well as a middle-tier layer. Export the grunt work of searching and sorting as SQL statements to your database. Give the layer an SQL-pass-through command so you can prototype code in your client easily, then port/move the code into the middle layer.
Do it over HTTP or pipe it through SSH.
The database can be anything, MySQL/Oracle/MS SQL. Don't write any procedures or anything to them. They are not very portable; it's very easy to get blocked in, at least in my experience. I'm sure lots of people are good at not getting blocked in, but it's not automatic.
Re:Portability (Score:2)
Re:Portability (Score:2)
Re:Portability (Score:2)
- build a database abstraction layer into your application. From the application, access the database only through this abstraction layer; that way you'll only need to recode the abstraction layer to support other databases
- make database data visible to users *only* through views and stored procedures. If you ever need to migrate to another database platform, it's generally much easier to recode views and stored procedure
Fascade Pattern (Score:4, Interesting)
It's spelled Facade (Score:2)
Re:It's spelled Facade (Score:2)
depends... (Score:4, Insightful)
1) abstraction. yes, abstracting the data layer at the database means accepting that you're tied to that database, which isn't a bad thing: i grew up with oracle and still swear by it. as a result, you only write one db layer api, not one per supported dbms per target business logic language.
2) depending on which rdbms and how you set up the user access, you can also probably push a large chunk of the connection pooling onto the db engine rather than having to manage it yourself.
3) access control. microsoft mentioned this several times at their devdays convention. i was surprised that it was news to them (hint: fewer doors means fewer ways in). regardless, it's a good idea: i've always seen allowing execution of arbitrary sql statements by joe random as an invitation for bad things.
4) one place to go to for developer support requests.
hope this helps.
There is no "choice" here (Score:4, Interesting)
If you think that application developers are going to implement the business logic in the application, you better think again. I have watched many systems over the years, and NOT ONCE has the application group implemented the business logic in the application and gotten it right.
In one shop (RI not available in the DBMS, thank you to THAT vendor) we had 1 guy whose job it was to fix data errors. I tracked usage, and he was responsible for 1/3 of the resources used day in and day out just fixing errors in the data every day.
Re:There is no "choice" here (Score:2)
Well, I do Software Architecture/Design and implementation for a living (12 years experience) and I would say you have to do it in the Application/Business/Middle Tier(s) because middle tiers can be distributed and scaled much easier than databases. I would use Stored Procs and Views for certain things, but only to join tables and massage the data, I would _NOT_ put
I know where! (Score:5, Funny)
The best place for buiness logic in in Access Macros!!! It's even better when you have access link to Excell tables for it's data!!!!
Don't use Access Visual Basic - clippy will come down hard on you!!!! Macros are where it's at!!! Marcos are even unicode compliant - so localisation is really EZ!!!!!
DONT USE ACCESS 2000/XP/2003 - it's buggy!!! Aceess 2.0 or 97 is where all the stable apps are!!!!
WORD 97 has great macro support too - don't be afraid to put some business logice there.
There's a rounding bug in Access that you can use to get more money out of your customers!!!! It inflates the sales tax - keep the change!!! You deserve it!!!!
Newbie. (Score:1, Redundant)
You'll find it mos
layering (Score:1)
Fetch the data into business objects of some sort (or a local data store), work the data as you need then send it up to the pre
Re:layering (Score:1, Interesting)
Too many people comprehensively fail to understand Relational theory, while mucking about with the ill-d
That Tablizer fellow (Score:2)
Tablizer has an entire OOP-is-mad Web site, and I could never figure out what he was talking about because I do a lot of GUI work where OO is as essential as air (although Tablizer thinks there could be some kind of table-driven GUI, but I think that is reinventing OO in the style of C-level Gnome programming).
I believe the point is that in certain domains, OO is the hammer that thinks every pointy thing sticking up is a nail.
Re:layering (Score:1)
Code a library interface (Score:1)
Separate logical tiers!!!! (Score:1)
It's hard to give a more precise answer without knowing the details of your architecture.
But, in general, I prefer to partition an app that requires extensive business logic into separate logical layers (whether or not these are separate physical layers).
Database-side stored procedures do limit portability, but if you can run similar code at a point in your application that is physically close to the DB server, or at least on the same fast network segment, this gives you better performance than you'd get
portability (Score:3, Interesting)
In the DB IMO (Score:2, Interesting)
Applications that have used all the facilities of the database GENERALLY take less time and have fewer bugs.
Typically we could have rewritten the entire application for a different platform in
If you're using Oracle (Score:3, Interesting)
Other good reasons for externalizing the business logic is if a significant part of the data is external to the database. In that case, you may want to calculate what you can in the database and store it in a table, then use those calculated values with the external data to get the final values.
Re:If you're using Oracle (Score:2)
The point is, you probably don't need (much) code for calculations, outside of your queries. Most
Re:If you're using Oracle (Score:1)
Unhelpful but funny --or-- things BL stands for (Score:4, Interesting)
The first question to ask yourself when deciding where to put something is where did it come from? Often, this is the best place to put it.
I have been part of the whole where-does-the-business-logic-belong process many times, and if your company is anything like the ones I've seen, some middle manager with training in a field unrelated to anything you're dealing with pulled the "business logic" out of someplace, and you might suggest that they stuff it back in the same place when they are finished with it.
In any case, bright light often is bad for business logic, so it should be kept someplace like that anyway.
-- MarkusQ
Where Does the Business Logic Belong? (Score:5, Funny)
In the application server (Score:2)
Business Logic?! (Score:2, Funny)
data warehouse (Score:2)
The reporting logic can be held using multi-dimensional tables in OLAP cubes or similar so the reports then 'pop put' very quickly.
The hard bit is organising the data properly and getting the SQL tuned for the ETL phase.
Definately depends (Score:3, Informative)
You didn't say enough about your requirements to be able to say for sure, but here are some points that may help you:
I tend to do things using application servers, but this is because I have to combine data from different sources, not all of them databases. It makes sense to have the business logic all in one place - including user rights, audit trail/logging etc. I also like the fact that users don't have direct access to the database.
The right answer for you depends more on your requirements than anything else. There is no answer which is always right for all situations.
-- Steve
portability, reports, etc (Score:4, Insightful)
A solution to your performance problem probably lies in creating some views into your database that represent the data in a less transactional way - create the views such that they are not normalized for transactions but rather for the queries you expect, and the report generation speed problems will take care of itself, and you can leave the business logic code portable.
For example, somebody might want a report like "sales by state", and this report takes age to run against your transactional database because it's normalized for inserting a sale across many tables, and a select to retrieve this data might take a bunch of joins across all tables. If you create views, or replicate the data to another database with a different scheme, your select might be "select * from sales where state=''" and return very quickly.
Report logic goes where it runs fastest (Score:2, Interesting)
However, once you get to the reports, the system is usually at a stable stage as far as business rules are concerned, so they won't change much (if at all).
Finally, that whole thing about migrating from one database to another, well... it j
scalabilty (Score:2)
It depends. You haven't given us enough details. (Score:1)
There are many ways to query
You can use embedded-SQL, stored procedures, or perhaps (if you're careful|lucky) pull off similar performance from a prepared statement.
In many cases, the real overhead isn't the query, but merely passing the large amounts of data from the db to the app server. It depends.
THE issue with web based applications**
I run into the portability vs. speed issue all the time. It's THE fundamental issue with web-based architectures. OO Design on a large-scale, web-ba
Upper Tier Decision (Score:1)
My experience is that Shit Flows Downhill so head higher up the hill and ask a person more closely involved with the customer, contractor, etc..
This might be a good time for an evolutionary prototype where you create a business tier class that allows for more portability. You then create some PL/SQL, T-SQL , (whatever RDBMS you are using) code and test the perform