In my project I need to create a business object validation layer that will take my object and run it against a set of rules and return either pass or fail and it's list of failure reasons. I know there are quite a few options out there for accomplishing this.
From Microsoft:
Enterprise Library Validation Application Block
Windows Workflow Foundation Rules Engine
Open Source:
Drools.NET
Simple Rule Engine(SRE)
NxBRE
Has anyone had any particularly great successes or failures with any of these technologies (or any that I didn't list) or any opinions on what they feel is best suited for business rules validation.
Edit: I'm not just asking about generic validations string length < 200, zip code is 5 digits or 5+4 but assume that the rules engine would actually be leveraged.
The code-versus-rules-engine decision is a matter of trade-offs, IMHO. A few examples are:
Advantages of code
Potentially higher performance.
Uses developers' existing skills.
No need for separate tools, run-time engines, etc.
Advantages of rule engine
(Features vary across the various rule engines.)
Rule DSL that is writable (or at least readable) by business users.
Effective- and expiration-date properties that allow automatic scheduling of rules.
Flexible reporting from rule repository supports improved analysis and auditing of system behavior.
Just as data-base engines isolate data content/relationship issues from the rest of the system, rules engines isolate validation and policy from the remainder of the system.
A modified version of the CSLA framework rules.
Many of the other rules engines have the promise that goes like "The end user can modify the rules to fit their needs."
Bahh. Very few users are going to learn the complexities of the rules document format or be able to understand the complexities and ramifications of their changes.
The other promise is you can change the rules without having to change the code. I say so what? Changing a rule even as simple as "this field must not be blank" can have a very negative impact on the application. If those fields where previously allowed to be blank you now have a bunch of invalid data in the data store. Plus modern applications are either web based or distributed/updated via technologies like click=once. So you updating a couple of components is just as easy as updating a rules file.
So, because the developer is going to modify them anyway and because they are core to the Business objects operations just locate them in one place and use the power of modern languages and frameworks.
I didn't really like rule and validation blocks provided by Microsoft (too complex and inflexible) so I had to build mine, based on experience with custom business workflow engines.
After a couple of iterations the project has finally gone Open Source now (BSD license) and has proven helpful in production systems. Primary features of .NET Application Block for Validation and Business Rules:
Simple to get started with
Rules for the domain objects
Rule Reusability
Predefined Validation Rules
Behavior Extensibility
Proper object nesting
Designed for DDD and UI level validation
Multiple reporting levels
Production-proof and active development
Small codebase
Open Source
Here's how a simple binding of rules at the UI level looks like:
Note, that current implementation does not have any DSL at the moment. C# syntax is expressive enough on its own, so there has been no demand to add Boo-based DSL on top.
I have to admit, for really simple validations, I tend to write my own very small, compact rules engine, mostly because I think using someone else's implementation just isn't worth it for a small project.
I've experimented with Workflow Foundation, used EntLib, and written my own rules engine.
In small applications where I only really need to do UI-based validation to ensure invalid data doesn't sneak into the DB, I reach for the EntLib Validation Block. It's easy to use and requires only a minimal amount of code in my domain objects, plus it doesn't mess up NHibernate or anything else in my technology stack.
For complex stuff, domain-layer validation, etc., I'd easily opt to write my own rules engine again. I'd much rather write rules in code, each rule in it's own tiny class, easily testable and very simple to compose complex sets of rules with.
In the large app that I worked on where I wrote this sort of rules engine, we then used FitNesse to test our rule configurations. It was great having that kind of tool to utilize to ensure correctness. We could feed it tables and tables of data and know, with certainty, that our configured rules worked.
If you are interested in rolling your own, read JP Boodhoo's post on Rules processing. Essentially he lays out a straight forward framework for validating domain objects.
Validatiion in the Domain Layer
Validation in the Domain Layer 2
Try
http://rulesengine.codeplex.com
It's lightweight, uses fluent-interfaces to define validation logic, extensible, and Free!
You can even define rules on interfaces, implementors inherit the rules.
No more annoying attribute-style validation - what it you don't own the class you want to valida
It has a plug-in to Asp.Net MVC (server-side only).
There is also another project called Polymod.Net which uses RulesEngine to provide self-validating UI's as shown in the screen-shot!
Enterprise Library Validation Block provides a very AOP like approach and keeps things simple in both 3.1 and 4.1 from my experience.
I recommend using CSLA Framework. Not only for Validation but for other features also.
Related
Found thinking of a title a little tricky, but basically my question is how do you keep a system as generic (right term?) as possible when needing to write customer special code.
My scenario is that we have a standard system that lives in the trunk of subversion. Sometimes customers want to make a change that does not conform to that standard system, so our practice is to usually branch off the code and develop on that branch, but then it gets messy when trunk work needs to be included in a customer special or if an idea a customer wants needs to be in both the branch and trunk.
Is there a better way or standard to deal with such requests because it seems everything coded is very specific?
It's certainly a difficult scenario.
One option might be to have configuration to stipulate whether a feature is turned on or not - so the feature goes into the main branch but you only set the config to true for the customer(s) that require it. It's still a bit messy but I think it's cleaner than having multiple branches per customer.
In more complex scenarios you can develop new functionality as a type of 'plug in' architecture, where you have different implementations implementing the same interface - some type of factory could decide which to load.
I don't think there is any magic bullet for maintaining a generic code base which is subject to differing customer requests.
In terms of improving code re-usability, there are usually several parts:
Language Features
Being able to interchange smaller parts of an application is a core language design concern. Many languages are designed with methods of maximizing re-usability. Here are some examples (some are C# specific):
Interfaces
Generics
Co/Contra-variance
Dependency Injection
Design patterns (specifically, ones related to structure or creation)
They will allow you to reuse your codebase more easily but, on their own, aren't a golden bullet.
Project Design
Even if you exploit every feature available in C#, a insufficiently thought out project will probably be difficult to interchange. I'd reccomend any larger solutions should be split into logical, independant, subunits. Again, an example of a solution structure might be:
MyProgram // Wires up components and subunits.
MyProgram.Core // Contains a core 'kernel' for your application
MyProgram.IO // Contains generic interfaces and implementations for performing IO
MyProgram.UI // Contains UI code that displays and interacts with your core. The core should not depend on UI elements.
MyProgram.Models // Contains structure of databases, data models other components may act on
MyProgram.Models.Serialization // Converts your models for IO etc. Maybe you want MyProgram.IO to be generic enough to reuse.
Helpers // Generic helpers that you tend to use in multiple applications. You will want these to be in a standalone repository so that you can keep the classes validated.
Versioning
Ultimately, you may not be able to handle -every- problem with language features and good project design. If you have an awkward client request, sometimes you might want to pull (not branch) the components you -can- reuse and branch the ones you need to edit.
We develop a distributed system built from components implemented in different programming languages (C++, C# and Python) and communicating one with another across a network.
All the components in the system operate with the same business concepts and communicate one with another also in terms of these concepts.
As a results we heavily struggle with the following two challenges:
Keeping the representation of our business concepts in these three languages in sync
Serialization / deserialization of our business concepts across these languages
A naive solution for this problem would be just to define the same data structures (and the serialization code) three times (for C++, C# and Python).
Unfortunately, this solution has serious drawbacks:
It creates a lot of “code duplication”
It requires a huge amount of cross-language integration tests to keep everything in sync
Another solution we considered is based on the frameworks like ProtoBufs or Thrift. These frameworks have an internal language, in which the business concepts are defined, and then the representation of these concepts in C++, C# and Python (together with the serialization logic) is auto-generated by these frameworks.
While this solution doesn’t have the above problems, it has another drawback: the code generated by these frameworks couples together the data structures representing the underlying business concepts and the code needed to serialize/deserialize these data-structures.
We feel that this pollutes our code base – any code in our system that uses these auto-generated classes is now “familiar” with this serialization/deserialization logic (a serious abstraction leak).
We can work around it by wrapping the auto-generated code by our classes / interfaces, but this returns us back to the drawbacks of the naive solution.
Can anyone recommend a solution that gets around the described problems?
Lev, you may want to look at ICE. It provides object-oriented IDL with mapping to all the languages you use (C++, Python, .NET (all .NET languages, not just C# as far as I understand)). Although ICE is a middle-ware framework, you don't have to follow all its policies.
Specifically in your situation you may want to define the interfaces of your components in ICE IDL and maintain them as part of the code. You can then generate code as part of your build routine and work from there. Or you can use more of the power that ICE gives you.
ICE support C++ STL data structures and it supports inheritance, hence it should give you sufficiently powerful formalism to build your system gradually over time with good degree of maintainability.
Well, once upon a time MS tried to solve this with IDL. Well, actually it tried to solve a bit more than defining data structures, but, anyway, that's all in the past because no one in their right mind would go the COM route these days.
One option to look at is SWIG which is supposed to be able to port data structures as well as actual invocation across languages. I haven't done this myself but there's a chance it won't couple the serialization and data-structures so tightly as protobufs.
However, you should really consider whether the aforementioned coupling is such a bad thing after all. What would be the ideal solution for you? Supposedly it's something that does two things: it generates compatible data structures across multiple languages based on one definition and it also provides the serialization code to stitch them together - but in a separate abstraction layer. The idea being that if one day you decide to use a different serialization method you could just switch out that layer without having to redefine all your data structures. So consider that - how realistic is it really to expect to some day switch out only the serialization code without touching the interfaces at all? In most cases the serialization format is the most permanent design choice, since you usually have issues with backwards compatibility, etc. - so how much are you willing to pay right now in development cost in order to be able to theoretically pull that off in the future?
Now let's assume for a second that such a tool exists which separates data structure generation from serialization. And lets say that after 2 years you decide you need a completely different serialization method. Unless this tool also supports plugable serialization formats you would need to develop that layer anyway in order to stitch your existing structures to the new serialization solution - and that's about as much work as just choosing a new package altogether. So the only real viable solution that would answer your requirements is something that not only support data type definition and code generation across all your languages, and not only be serialization agnostic, but would also have ready made implementation of that future serialization format you would want to switch to - because if it's only agnostic to the serialization format it means you'd still have the task of implementing it on your own - in all languages - which isn't really less work than redefining some data structures.
So my point is that there's a reason serialization and data type definition so often go together - it's simply the most common use case. I would take a long look at what exactly you wish to be able to achieve using the abstraction level you require, think of how much work developing such a solution would entail and if it's worth it. I'm certain that are tools that do this, btw - just probably the expensive proprietary kind that cost $10k per license - the same argument applies there in my opinion - it's probably just over engineering.
All the components in the system operate with the same business concepts and communicate
one with another also in terms of these concepts.
When I got you right, you have split up your system in different parts communicating by well-defined interfaces. But your interfaces share data structures you call "business concepts" (hard to understand without seeing an example), and since those interfaces have to build for all of your three languages, you have problems keeping them "in-sync".
When keeping interfaces in sync gets a problem, then it seems obvious that your interfaces are too broad. There are different possible reasons for that, with different solutions.
Possible Reason 1 - you overgeneralized your interface concept. If that's the case, redesign here: throw generalization over board and create interfaces which are only as broad as they have to be.
Possible reason 2: parts written in different languages are not dealing with separate business cases, you may have a "horizontal" partition between them, but not a vertical. If that's the case, you cannot avoid the broadness of your interfaces.
Code generation may be the right approach here if reason 2 is your problem. If existing code generators don't suffer your needs, why don't you just write your own? Define the interfaces for example as classes in C#, introduce some meta attributes and use reflection in your code generator to extract the information again when generating the according C++, Python and also the "real-to-be-used" C# code. If you need different variants with or without serialization, generate them too. A working generator should not be more effort than a couple of days (YMMV depending on your requirements).
I agree with Tristan Reid (wrapping the business logic).
Actually, some months ago I faced the same problem, and then I incidentally discovered the book "The Art Of Unix Programming" (freely available online). What grabbed my attention was the philosophy of separating policy from mechanism (i.e. interfaces from engines). Modern programming environments such as the NET platform try to integrate everything under a single domain. In those days I was asked for developing a WEB application that had to satisfy the following requirements:
It had to be easily adapted to future trends of User Interfaces without having to change the core algorithms.
It had to be accessible by means of different interfaces: web, command line and desktop GUI.
It had to run on Windows and Linux.
I bet for developing the mechanism (engines) completely in C/C++ and using native OS libraries (POSIX or WinAPI) and good open source libraries (postgresql, xml, etc...). I developed the engine modules as command-line programs and I eventually implemented 2 interfaces: web (with PHP+JQuery framework) and desktop (NET framework). Both interfaces had nothing to do with the mechanisms: they simply launched the core modules executables by calling functions such as CreateProcess() in Windows, or fork() in UNIX, and used pipes to monitor their processes.
I'm not saying UNIX Programming Philosophy is good for all purposes, but I am applying it from then with good results and maybe it will work for you too. Choose a language for implementing the mechanism and then use another that makes interface design easy.
You can wrap your business logic as a web service and call it from all three languages - just a single implementation.
You could model these data structures using tools like a UML modeler (Enterprise Architect comes to mind as it can generate code for all 3.) and then generate code for each language directly from the model.
Though I would look closely at a previous comment about using XSD.
I would accomplish that by using some kind of meta-information about your domain entities (either XML or DSL, depending on complexity) and then go for code generation for each language. That would reduce (manual) code duplication.
In my last question I posted some sample code on how I was trying to achieve separation of concerns. I received some ok advice, but I still just don't "get it" and can't figure out how to design my app to properly separate concerns without designing the next space shuttle.
The site I am working on (slowly converting from old ASP section by section) is moderately sized with several different sections including a store (with ~100 orders per day) and gets a decent amount of traffic (~300k uniques/month). I am the primary developer and there might be at most 2-3 devs that will also work on the system.
With this in mind, I am not sure I need full enterprise level architecture (correct me if i am wrong), but since I will be working on this code for the next few years, I want it to perform well and also be easy to extend as needed. I am learning C# and trying to incorporate best practices from the beginning. The old ASP site was a spaghetti mess and I want to avoid that this time around.
My current stab at doing this ended up being a bunch of DTOs with services that validate and make calls to a DAL layer to persist. It was not intentional, but I think the way it is setup now is a perfect anemic domain model. I have been trying to combat this by turning my BLL to domain objects and only use the DTOs to transfer data between the DAL and BO, but it is just not working. I also had all my dtos/blls split up according to the database tables / functionality (eg - YouTube style app - I have separate DTO/BLL/DAL for segments, videos, files, comments, etc).
From what I have been reading, I need to be at least using repositories and probably interfaces as well. This is great, but I am unsure how to move forward. Please help!
From what I can see you have four points that need addressing:
(1) "With this in mind, I am not sure I need full enterprise level architecture"
Lets deal with the high level fluff first. It depends on what you mean by "full enterprise level architecture", but the short answer is "Yes" you need to address many aspects of the system (and it will depend on the context of the system as to what the main ones are). If nothing else, the keys ones would be Change and Supportability. You need to structure the application in a way that supports changes in the future (logical and physical separation of concerns (Dependency Injection is a great for the latter); modular design, etc).
(2) "How to properly separate concerns in my architecture without designing a spacecraft?"
I like this approach (it's an article I wrote that distilled everything I had learnt up to that point) - but here's the gist:
Looking at this you'll have a minimum of six assemblies - and that's not huge. If you can break your system down (separate concerns) into these large buckets it should go a long way to giving what you need.
(3) Detail
Separating concerns into different layers and classes is great but you need to go further than that if you want to be able to effectively deal with change. Dependency Inversion (DI) is a key tool to use here. When I learnt DI it was a hand-rolled affair (as shown in the previous link) but there are lots of frameworks (etc) for it now. If you're new to DI (and you work in .Net) the article will step you through the basics.
(4) How to move forward
Get a simple vertical slice (UI all the way to the DB) working using DI, etc. As you do this you'll also be building the bones of the framework (sub-systems and major plumbing) that your system will use.
Having got that working start on a second slice; it's at this point that you should uncover any places where you're inadvertently not reusing things you should be - this is the time to change those before you build slices 3,4 and 5 - before there's too much rework.
Updates for Comments:
Do you you think I should completely
drop web forms and take up MVC from
scratch or just with what I know for
now?
I have no idea, but for the answer to be 'yes' you'd need to be able to answer these following questions with 'yes':
We have the required skills and experience to use and support MVC.
We have time to make the change (there is clear benefit in making this change).
We know MVC is better suited for our needs.
Making this change does not put successful delivery at risk.
...do I need to move to projects and
setup each of these layers as a
separate project?
Yes. Projects map 1-to-1 with assemblies, so get the benefits of loose-coupling you'll definitely want to separate things that way, and be careful how you set references.
when you refer to POCOs, are you meaning just DTOs or rich domain objects?
DTO not Rich Domain Object. BUT, people seem yo use the terms POCO and DTO interchangeably when strictly speaking they aren't - if you're from the Martin Fowler school of thought. In his view a DTO would be a bunch of POCO's (or other objects(?)) parcelled together for sending "across the wire" so that you only make one call to some external system and not lots of calls.
Everyone says I should not expose my
data structures to my UI, but I say
why not?
Managing dependencies. What you don't want is for you UI to reference the physical data structure because as soon as that changes (and it will) you'll be (to use the technical term) screwed. This is the whole point of layering. What you want to do is have the UI depend on abstractions - not implementations. In the 5-Layer Architecture the POCOs are safe to use for that because they are an abstract / logical definition of 'some thing' (a business concept) and so they should only change if there is a business reason - so in that sense they are fairly stable and safer to depend on.
If you are in the process of rewriting your eCommerce site, you should atleast consider replacing it with a standard package.
There are many more such packages available today. So although the decision to build the original site may have been correct, it is possible that building a custom app is no longer the correct decision.
There are several eConmmerce platforms listed here: Good e-commerce platform for Java or .NET
It should cost much less than the wages of 2-3 developers.
I'm about to embark on a new project within which we require the ability to re-use validations based on (preferably XML) on both the client and server.
We would setup a service to provide the XML validation configuration data to the client side.
The following is not meant to be inflammatory in any way.
The Enterprise library does have support for the validation of objects to be configured in XML but java developers would not have access to a java reader version of this XML interpretation.
There is also Spring.Net validation but again I think this may be tied too much to .net. Is the Spring.Net validation suite straight ported over from the java spring framework i.e. without changes to the xml config?
Is there any other frameworks for validation which are able to be used in both .Net and Java?
The project will be fully SOA and the validation is one of the last things I have to figure out.
EDIT:
To clarify the validation needs to occur within the language that the receiving client is using, i.e. if the client to the web service is Java then the validation would be read into java and validated within java so that error conditions could be reported to the UI for the user to rectify. Equally if it was a .net client the .net client would be able to read it in and provide the same functionality.
I don't want to validate within the xml, the xml will be a set of rules, i.e. Customer.Name will be a maximum 50 chars long and must be at least 5 chars, and is a required field.
Thanks
Pete
Have a look at DROOLS.
There are .Net and Java versions of the rules engine.
Java Link and .Net Link
I've not user the libraries, so cannot comment on how "seamlessly" the one set of rules could be used in both environments.
How about trying the validation in a scripting language that can be run in both the jvm and by .net.
Scripting languages would be ideal for this kind of logic so maybe:
Ruby - http://www.ironruby.net/ and http://www.jruby.org/
or Perl.
This approach would allow use to the exact same code for validation and then call this from Java or .net.
Using jruby wouldn't be much of a performance overhead and it can integrate very closely with java. I've less experience with Ironruby but from what I've read once the code has been loaded and is running the performance is ok and it can be integrated well into the .net code - see: http://www.ironruby.net/Documentation/.NET/Hosting
Not to take away from my answer but regardless of how you do this it will involve introducing a new technology with all the associated overheads - dev environment etc. A better approach may be just to do it in .net and java seperately but maintain a very extensive test suite of examples to ensure that two validations remain in sync.
Not sure what sort of validation your are trying to accomplish. If your business objects are going to be serialized in XML form, then aside from schema validation, you can augment that with additional business rules and checks using Schematron.
Schematron is an ISO standard and provides a way of encoding business rules, restrictions and validation that is not possible in XML Schema.
The Schematron differs in basic
concept from other schema languages in
that it not based on grammars but on
finding tree patterns in the parsed
document. This approach allows many
kinds of structures to be represented
which are inconvenient and difficult
in grammar-based schema languages. If
you know XPath or the XSLT expression
language, you can start to use The
Schematron immediately.
I have been working on an ASP.NET/C# web app for some time and its size has gotten way to large for how it is being programmed. It has become very hard to maintain and getting harder quickly, something that used to take an 1hr to update now takes about 3-4hrs.
I believe that reworking the app to use different layers would help solve many of these problems. However the more I read the more it seems that everyone does it differently, but achieve mostly the same goals.
I have seen layers such as Presentation/UI, DB, Business, Services, ect. It appears that a 3 layer may be the best but I am unsure.
What layers should I have in a web app and what should each include or be limited to?
Words from previous experience are most appreciated.
I believe the common approach is to have 3 layers: presentation, business logic and data access. It should provide for a good foundation.
Having said that I need to point out that division into layers may not help very much with ASP.NET WebForms project. The biggest issue of this framework is its code behind which lures developers into creating monster pages that talk to all layers and services simultaneously to fetch the data to display. The cure is to work out this presentation layer first and let the code only interact with one specific layer (most usually, the business logic). When (and if) this is done, then the project may be successfully refactored.
Your correct in saying everyone does it a little differently. This is how we do it:
Domain
Model - class objects such as Customer, Account, etc.
LookUp - value objects such as AccountType, IndentityType, etc.
Repositories or DataAccessObjects current we use LinqToSql and SQLClient on our older applications.
Service
Services for each of the models or Aggregates. You could also call this the application layer. The idea is we could change to a different UI and it would involve as little as possible code changes.
UI
Currently we are using Asp.net and MVC in our newer applications
The idea is that we can insert different "things" into the layers and not effect the others. ex. If I start using EntityToSql the UI or the Service layer is none the wiser. It just knows it creates an IRepository and calls the FindAll() method. We are also converting older applications to MVC and before we do that we seperate them into these layers so when something else comes out we would like to implement our Service and Domain layers don't have to be changed.
It's important to always be asking your self where should this live? Logic in your UI should really be confined to UI logic. The important thing is to have everyone on your team understand your way and be willing to implement it...
Mainly it is just managing dependencies.
Good examples are Sharp Architecture projects and Arc projects. You may download some open-source example applications for gaining more insight.
Seperation of concern might be a better way of looking at your web application and breaking things up. For say, if you have one page that is doing data access, validation and displaying of items you could take each one of those items, and create it's own class so that you have only one type of thing happening in each class. Then as you go through and start to refactor your code you can then start grouping same type classes into the same type folders, namespaces and projects.
Good luck, you've got a lot of work ahead of you.
Check out the Microsoft MVC model for ASP.NET. The issues you described are very common to ASP.NET applications, and refactoring it to match the MVC model might be the way to go. Don't get me wrong - it'll definitely take some work - but IMHO, it'll be worth it.
I believe it's heavily dependent on the requirements and whats needed. There is many types of design patterns of ways to structure a app. I've found it helpful to fully digest what I'm trying to accomplish from the requirements and find a match to patterns on this website. Give or take unique circumstances.
http://www.blackwasp.co.uk/DesignPatternsArticles.aspx
This site actually breaks it down design pattern by design pattern. Allowing you to make a good match based on your requirements and scalability needs.
Traditional layering is only part of the solution. Since that was your question, though, I know many very large sites that layer by presentation, business logic and data access.
However, there's much more that you could / should do, including:
Tiering and division-of-labor: partition business logic between the web tier and the data tier in a maintainable way (usually implies using stored procedures instead of dynamic SQL)
Refactoring your application. In many sites, pages get large and unmanageable due to insufficient attention to continual refactoring. Use approaches such as custom user controls, Master pages, common page base class, ASP.NET skins, page adapters, control adapters, HttpModules, etc, to move your application-wide logic into centrally manageable locations.
Careful adherence to OO design principles.
Strategic use of events to help ensure consistent separation of concern.