I'm trying to build a web application that let the administrator talk to the database through C# and add new tables and columns to fit his requirements (sort of a very simple database studio) but I'm not trying to just create some spaghetti application.
So I'm trying to figure out how to let those things dynamically (automatically) when he creates a table and use the table to build them :
1- The business objects or entities (the classes, it's objects and properties).
2- The Data access layer (some simple methods that connects to the database and add, update, delete retrieve items (objects)).
Is this possible ? any pointers on how to achieve it ?
EDIT
just opened your link!! .. it's talking about the data bound controls and stuff! .. my question is way more advanced than that!.
when you build an N-Layered application you start with the database schema and implementation and it's easy to do programtically then you start building the DAL classes which (add, edit, etc in other words the CRUD operations) in and form this database
what I want to do is to allow the web administrator to choose add the new table through my application and then -dynamically- the application would take the tables names and columns as parameters and create new classes and define within them the CRUD methods that will implement the SQL CRUD operations
then it would also create dynamically the classes and define within them the variables, properties and methods to call and use the DAL methods .. all this based on the table, column names
NOTE : All this happens on the run-time!
You might want to look into ASP.Net Dynamic Data. It's a RAD tool which very easily gives you CRUD functionality for your entities and more. Check it out.
Sometime back I had also asked similar question on SO. I got only one reply.
Today I was digging some information on MSDN and as I had guessed it, MS CRM entity model works based on metadata. So basically whatever a CRM developer is working against is just metadata, they are not real objects as such. Following is the MSDN link.
Extend MS CRM Metadata and here is the MS CRM 4.0 SDK.
I hope this should get you started.
Update: Recently hit upon Visual Studio LightSwitch. I think this is what we wanted to build. A UI which will pick up table information from DB and then create all CRUD screens. VS LightSwitch is in its Beta1 and has quite a lot of potential. Should be a nice starting point.
First, any man trying to create MS Access is doomed to recreate MS Access. Badly.
You are better off using ASP.NET Dynamic Data (as suggested) or ASP.NET MVC Scaffolding. But runtime-generated playforms that actually make decent applications are really pipe dreams. You will need developer time to do anything complex. Or well.
What you are asking is non-sense. Why? Because the idea behind BLL and n-tier is that you know your data model well, and can create a static class model to represent your data model.
If your data model is dynamic, and changing, then you cannot create a static BLL (which is what a BLL is). What you will have to do dynamically build your queries at run-time. This is not something that any of the traditional methods are designed to handle, so you must do everything yourself.
While it's possible to dynamically generate classes at run-time, this is probably not the approach you want to take, because even if you manage to make your BLL adapt to your dynamic database.. the code that calls the BLL will not know anything about it, thus it will never get called.
This is not a problem you will solve overnight, or by copying any existing solution. You will have to design it from scratch, using low level ADO calls rather than relying on ORM's or any automation.
Related
I have a requirement to build forms directly from a specific dataset format. I've gone one step of being able to create the database from the dataset using F#. Now I want to be able to create CRUD forms programmatically from the database. I've googled and checked for CRUD generators but couldn't find a suitable one for either WPF or asp.net. Websharper could have been a good option but F# is not an option as the employer wants C# even though the database generator was in F#. This I may have to convert to C# as well. Also I checked sharpkit but don't think is a viable solution. The reason for needing such a framework/application is that datasets come and go too quickly and design is not an issue. Having every time to design CRUD forms to be used just for a short while is tedious and time consuming.
Microsoft offers CRUD scaffolding through the Microsoft.EntityFrameworkCore libraries. It can take an existing SQL table and generate the supporting object, controllers, and CRUD classes, or it can take an object class and create a local database.
Here is a good example of the former, generating CRUD classes from an existing database:
https://www.c-sharpcorner.com/article/will-perform-simple-scaffold-crud-operations-in-asp-net-core-using-ef-core/
In trying to separate my domain layers and GUI and looking into all the different ways to do that, one thing that I keep asking is why is this so difficult? Why all the extra code for data obejcts and then all the extra mapping of properties copying values in and out etc. Shouldn't theere be an easier way?
Then I remeembered when i used to wite small littler db app using MS Access and, Access has the concept of a Dynaset, basically a Dynaset is a View, just like an SQL Server View, except it is an updateable view. So, a MS Access form would be based of the View/Dynaset and therefore would not have to know the details of all the individual tables involved. Sounds like the Data objects pattern to me. Now, since Access has had this for 2 decades, shuoldn't there be a similar Dynaset, View, Mapping tools for Entity Framework, one that abstracts away the entities from the presentation? Is there one I am not aware of? 3rd party?
Thoughts on this?
If I understand you correctly, you may be looking for Entity Framework with POCO entities. You can find templates for them in the online gallery for templates (when you Add New Item in the project). Alternatively you can use right-click in your .edmx design view, select "Add code generation item" and pick the Fluent Generator.
These methods create multiple files instead of the default all-in-one EF generated file. One such file is the DbContext (as opposed to ObjectContext), one contains only entities (in the form of regular C# objects, no attributes or anything, just plain objects) and the last contains generated mapping in the form of fluent rules.
In this phase you can de-couple the entities file from its template and move it to another assembly. And voila, you have entities independent on the EF infrastructure. You can just pass the context these entities like you would before, and it'll do mapping by itself.
Alternatively you can use tool like AutoMapper, but you'll have to provide the mapping manually, which is a lot of work, but may be good in some cases.
Good design requires work. If it was easy, everyone would do it automatically. After all, everyone wants to do the least amount of work possible.
All the things you are complaining bout are part of the good design process, and there is no getting around them if you want a good design.
If you want to take shortcuts, then by all means, skip them. It's your code. nothing requires you to do things any specific way.
Access can do a lot of things because it's a desktop application, not a web application. Web applications are fundamentally different from desktop applications in how you design them, how they work, and what issues you face with them. For instance, the fact that you have a stateless environment and cannot keep result set from request to request makes many of the things people take for granted in Access impossible to do in a web app.
Specifically, if you want to use views, you can do so. Views are updateable if they are properly designed, but typically require update statements that only affect one table in the view). EF can work with views as well, but it has a lot of quirks you must deal with.
The data mapper pattern has emerged as a common pattern in web design because it's the easiest and straight forward way to have clean separation of concerns between layers and/or tiers. I suggest you find ways to make them work within your development process.
It may also be that MVC is not the most appropriate framework for you to use. It sounds more like you want to build Web apps the way you did Acceess, in which case Visual Studio Lightswitch may be a better choice for you.
http://msdn.microsoft.com/en-us/library/ff851953.aspx
I've been reviewing examples on the web of 3 layer design and I've noticed that most samples return either datasets or data tables. The thing that is confusing me is what if you would rather return a generic list of type so you can utlize properties or methods from within the type your list is based on? As example using a Name property that concats various fields in a specific way depending on the data, if the List is bound to a control on a form then the Name property can be used as the datafield. If you would want to accomplish the same thing when using a dataset or table, you'd have to return the data from the database to acheive the same (I try not to use datasets or datatables so I'm probably very wrong about this statement. :) )
The part that is really confusing me is about resusing code, to me it seems the only way to reuse code is to retrieve the data into either a dataset or datatable and then loop through the data and add it to a List, is this generally the best practice for 3 layer or is there a way to do this without datasets and datatables.
The example in the link below demonstrates in essence using datasets or tables and then adding it to an object but I'm forced to ask if this is the best practice?
http://www.codeproject.com/Articles/36847/Three-Layer-Architecture-in-C-NET
Thanks
Using DataTables is a specific dotnetism. The reason behind it is that they contain metadata about the structure of the data, which lets DataGrid (and other such components) display the data automatically without using reflection or so. My guess is this is amongst other things a heritage of the MS Access approach to RAD, where the intent was enabling "business people" to create apps by generating the user interface directly from a SQL schema, essentially doing the opposite of a tiered design. This heritage then seems to have leaked into the hivemind.
There's nothing wrong about using "plain" data structures, as long as you're willing to give up the RAD features, and the trend lately seems to have been to get rid of this tradeoff too. (For instance with Web Forms' strongly typed data controls, and MVC's model binding features.)
Also, speaking more generally, Code Project articles from before MVC was established are not really a good source of wisdom on general software architecture.
What you should carry your data on depends entirely on your needs.
If you retrieve data from the DB and bind it to a datagrid, datasets might give you the perfect solution. If you want some other method where data tracks its own update status you should look into Entity Framework. If you retrieve data and send it through a web service for cross platform or cross domain processing you need to load your data onto some other serializable classes of your own.
Take a look at the article below. It is a little old and targeted at EF4 but it summerizes pros and cons of different strategies very well. (There are three articles in the series, I suggest you read them all)
http://msdn.microsoft.com/en-us/magazine/ee335715.aspx
I think the samples you're finding used data tables and datasets because it's a simple way to show 3-tier design. Now days Entity Framework has largely replaced the "data access layer" mentioned in the sample.
Before entity framework when I wrote a data access layer I would return a generic list that I built from the database. To run an update, delete, or insert I would pass an object in as the parameter to the methods, then use the object's properties as the values in the sql statement. I preferred doing it that way for the reasons you mentioned but also because it allowed me to change the object definitions or db schema (or even use a different db all together) independently of each other.
Sorry if this has been asked elsewhere, but I couldn't find a clear answer anywhere.
I have decided to begin learning to use relational databases a bit more, namely SQL. This is a major beginners question but its probably essential to get started on.
I'm basically a little confused the best practice on how to utilize SQL (or other). At college i have accessed databases (using JSON strings) for things such as mobile apps, but i have never actually designed and built a database myself, as my tutor made the mentioned database for us to access himself.
Lets say I have a C# application that holds genealogy information (i.e. families and their members) and i wanted to store each individual on a database. Would I, simply use the structure I already have but save to fields in a database instead of an xml or text document? Or does it work the other way, i.e. do I create a database with required fields then just retrieve this from the database in a c# application and manipulate the data as I so wish, so the application would be entirely different (so the c# application basically doesn't really hold/store any data and just works on whats fed from the database)?
Whats troubling me is that usually where I would store my c# objects in a dictionary or list for example, would i instead just retrieve straight from the database? or retrieve from the and store the data into a normal structure and work from there (surely this would defeat the point of fast-searching from a database)?
I may be over-thinking it slightly. Hope that makes sense. Thanks in advance
Would I, simply use the structure I already...
or
do I create a database with required fields...
I think that is the crux of your question.
Starting from the database
For me, when building an application that uses a backend database, an Entity-Relationship diagram is pretty crucial. I found quite a nice little tutorial for you here: http://www.sum-it.nl/cursus/dbdesign/english/index.php3 but you can easily find one that suits your learning style. The key point is that you are trying to model the problem domain (the real world out there that needs your application) in a way that your application can somehow capture. Once you have an E-R diagram of related tables, it is easier to figure out the details. Using SQL Management Studio for SQL Server 2008 (Express edition) you can create a few basic tables and build the E-R diagram right there and have it generate relationships for you. You can then, at your leisure, examine the SQL used to achieve that and refine accordingly.
Personally, I always start by examining the problem domain, then I build the E-R diagram, then I build the database. I start building the C# application when I'm reasonably confident the database reflects the problem domain.
Starting from your C# application
However, what really matters is that you model the real world in a meaningful and effective way. In your case you already have a starting point in structures you've created in C# and you can use them to give you a starting point to build the E-R diagram. If you find it easier to get a C# application going and then build a database that reflects it, that should be fine. Perhaps you already have an approach that helps you capture the problem domain effectively. It's an iterative process whatever you do: building the C# code might reveal problems with the underlying database design and vice versa.
Diagramming - E-R or UML?
I'm personally convinced that this whole business is so complicated that you really need some diagrams.
to visualise your database, use an E-R diagram
to visualise your C# application use a UML class diagram
As you head towards a working application, you'll see how these 2 diagrams begin to match or at least reflect eash other pretty closely. In both cases, (entities or classes) understanding the relationship between objects will be really important when you query the database because it is crucial to understand relationships between tables (especially using 1-to-many relationships to resolve a complex many-to-many relationship) and various techniques for joining tables in queries (INNER or OUTER joins etc) No matter how clever your C# application is, you will at some point need to understand at least some of the complexities of the SQL language - and it is easier if you can refer to an E-R diagram.
Where to store?
Whats troubling me is that usually where I would store my c# objects in a dictionary or list for example, would i instead just retrieve straight from the database?
In the database, without a doubt. A C# class called Family would have a property FamilyName, say, with a setter method built in. If you discover a spelling mistake and want to change the name, the setter method would open a connection to the database, run an UPDATE query with the specified family name, (and probably the family id) as a parameter, and update the underlying field accordingly. Retrieving data would involve running a SELECT query etc.
Conclusion
Do some tutorials on how to examine a problem domain, create an entity-relationship diagram and build a set of related tables based on the diagram. I'm convinced that way you'll find it much easier to keep track of the C# classes that you build to communicate with the backend database.
Here's an example of a simple E-R diagram for families and their members:
To begin with you might think members and family could be in one table, but then you discover that creates a lot of duplication so you separate that out into family and member table with a one-to-many relationship, but then you realise that, through marriage for instance, people can belong to more than one family and you need to create a many-to-many relationship. I think the E-R diagram is the best place to work out that kind of complexity.
Not knowing what your structures look like or how your DB will be designed this is hard to answer. But you should be able to use existing data structures, and just pipe the data from the database instead of the XML file.
Look into Linq-to-XML, C# has a strong library to interact with SQL. May be a bit confusing at first, but very powerful once you learn it.
If I am right you are asking also if you should retrieve all the records from the database and store them as objects in a collection or retrieve selected records from the database and use the dataset results without placing them in a purpose defined structure.
I tend to select the records I want from the database and then load the results into my purpose defined classes / structures. This allows you to add your manipulation methods to the class holding a record result etc. without needing to take in dataset results to each method. However you will find yourself doing singular updates all the time when a batch update might be more efficient... if that makes sense.
Take a look at entity frameworks code first. If your data structures are classes in your application there are techniques to use that to create your database schema from that. As far as the data. Store it in your database and populate your lists and dictionaries with it. Or populate list of class genealogy individual with it.
If you want to write your own data classes, there's a free tutorial here written by myself. What I would definitely not to is use the data sources in ASP.NET, as these wizards are the Barty Crouches of the ASP.NET world - they appear good, but turn out to be evil, as inevitably you'll want to be able to tweak them and you won't understand how to do this.
I am starting a new ASP.NET MVC project to learn with, and am wondering what's the optimal way to set up the project(s) to connect to a SQL server for the data. For example lets pretend we have a Product table and a product object I want to use to populate data in my view.
I know somewhere in here I should have an interface that gets implemented, etc but I can't wrap my mind around it today :-(
EDIT: Right now (ie: the current, poorly coded version of this app) I am just using plain old SQL server(2000 even) using only stored procedures for data access, but I would not be adverse to adding in an extra layer of flexability for using linq to sql or something.
EDIT #2: One thing I wanted to add was this: I will be writing this against a V1 of the database, and I will need to be able to let our DBA re-work the database and give me a V2 later, so it would be nice to only really have to change a few small things that are not provided via the database now that will be later. Rather than having to re-write a whole new DAL.
It really depends on which data access technology you're using. If you're using Linq To Sql, you might want to abstract away the data access behind some sort of "repository" interface, such as an IProductRepository. The main appeal for this is that you can change out the specific data access implementation at any time (such as when writing unit tests).
I've tried to cover some of this here:
I would check out Rob Conery's videos on his creation of an MVC store front. The series can be found here: MVC Store Front Series
This series dives into all sorts of design related subjects as well as coding/testing practies to use with MVC and other projects.
In my site's solution, I have the MVC web application project and a "common" project that contains my POCOs (plain ol' C# objects), business managers and data access layers.
The DAL classes are tied to SQL Server (I didn't abstract them out) and return POCOs to the business managers that I call from my controllers in the MVC project.
I think that Billy McCafferty's S#arp Architecture is a quite nice example of using ASP.NET MVC with a data access layer (using NHibernate as default), dependency injection (Ninject atm, but there are plans to support the CommonServiceLocator) and test-driven development. The framework is still in development, but I consider it quite good and stable. As of the current release, there should be few breaking changes until there is a final release, so coding against it should be okay.
I have done a few MVC applications and I have found a structure that works very nicely for me. It is based upon Rob Conery's MVC Storefront Series that JPrescottSanders mentioned (although the link he posted is wrong).
So here goes - I usually try to restrict my controllers to only contain view logic. This includes retrieving data to pass on to the views and mapping from data passed back from the view to the domain model. The key is to try and keep business logic out of this layer.
To this end I usually end up with 3 layers in my application. The first is the presentation layer - the controllers. The second is the service layer - this layer is responsible for executing complex queries as well as things like validation. The third layer is the repository layer - this layer is responsible for all access to the database.
So in your products example, this would mean that you would have a ProductRepository with methods such as GetProducts() and SaveProduct(Product product). You would also have a ProductService (which depends on the ProductRepository) with methods such as GetProductsForUser(User user), GetProductsWithCategory(Category category) and SaveProduct(Product product). Things like validation would also happen here. Finally your controller would depend on your service layer for retrieving and storing products.
You can get away with skipping the service layer but you will usually find that your controllers get very fat and tend to do too much. I have tried this architecture quite a few times and it tends to work quite nicely, especially since it supports TDD and automated testing very well.
For our application I plan on using LINQ to Entities, but as it's new to me there is the possiblity that I will want to replace this in the future if it doesn't perform as I would like and use something else like LINQ to SQL or NHibernate, so I'll be abstracting the data access objects into an abstract factory so that the implementation is hidden from the applicaiton.
How you do it is up to you, as long as you choose a proven and well know design pattern for implementation I think your final product will be well supported and robust.
Use LINQ. Create a LINQ to SQL file and drag and drop all the tables and views you need. Then when you call your model all of your CRUD level stuff is created for you automagically.
LINQ is the best thing I have seen in a long long time. Here are some simple samples for grabbing data from Scott Gu's blog.
LINQ Tutorial
I just did my first MVC project and I used a Service-Repository design pattern. There is a good bit of information about it on the net right now. It made my transition from Linq->Sql to Entity Framework effortless. If you think you're going to be changing a lot put in the little extra effort to use Interfaces.
I recommend Entity Framework for your DAL/Repository.
Check out the Code Camp Server for a good reference application that does this very thing and as #haacked stated abstract that goo away to keep them separated.
i think you need a orm.
for example entity framework(code first)
you can create some class for model.
use these models for you logic and view,and mapping them to db(v1).
when dba give you new db(v2),only change the mapping config.(v1 and v2 are all rdb,sql server,mysql,oracel...),if db(v1) is a rdb and db(v2) is a nosql(mongo,redis,couchbase...),that's not work
may be need do some find and replace