In my first c# project, I need to connect to a database server for multiple read only queries. Would anyone share experiences on how to organize the queries into the project? currently I just hardcoded query strings in the c# source files whenever needed. but it is hard to maintain and once something changes on the database server side I am in trouble. Or should I put all query strings in the .config file using appsettings? Are there better ways? I do not have rights to save stored procedures on the server. thanks.
There are different answers with varying levels of sophistication based on your needs. Except in the very smallest of projects, I create two class library projects for database access: one that contains the data model and queries and another test project that exercises the first project's queries. In simple solutions, you use this library in an ASP.NET or other project.
You should strongly consider learning an ORM like NHibernate or VS 2008/.NET 3.5's Linq-To-SQL or Entity Framework. Minimally, you MUST remember to use parameterized queries if you have a web-facing app.
In more sophisticated solutions you will completely encapsulate the database into it's own service, or tier. In my experience I had a data access tier that ran in it's own Windows Communication Foundation service, as a Windows Service, and it was the only service that could talk directly to the database or knew the database's data model. It would do all the interaction with the database, and then transform the data into different data models that are read by the other tiers. I typically create a project called "Contracts" that contains all the interfaces and data models that are communicated from the data tier to the rest of the system. The reason you do this is so that you avoid the pain you have mentioned: you can update the underlying database, ORM layer, and "common data models" and then not change the other tiers at all.
If this is your first project, try to keep thinks simple. If you add too much variables probably you'll end thinking more in technology than in solutions.
That said, if your queries don't expect to change it's parameters, you can use stored procedures. This approach also will help boost your queries as the execution plan will be kept in the database.
Related
I am working on a MVC4 project which will need to use a number of different databases, each with a few stored procedures for searching. The site is an asset search tool which needs to query various existing systems. If I allow the EF to generate models on its own, I will end up with a Model for each procedure I use in each database.
What I would prefer is to have my own POCO model already defined and the EF maps its results to that Model. So regardless of what database the data is taken from it maps back to that same Model. The column names in each database differ slightly so it would really need to be mapping columns to model properties.
There is no writing back to the database, it purely selects data out.
On the 'Edit Function Import' form I can create a model based on the results. There is also an option to view 'Function Import Mapping' but it does not appear to do what I am looking for.
Has anyone else tried this?
Added an image to help explain the issue
The closest to this I have managed so far is to have EDMX1 query 2 databases. This only works because they are on the same Db server. I had to fully qualify the Db names in the stored procedure. I could then use 1 EF Model as a return type for the 2 queries. That Model still is not usable in another EDMX though, so if I need to connect to a different Db server, I still cannot share the Model. So the problem is not solved.
Here is image of current progress.
Function Import Mapping is for mapping stored procedure / function calls to EF code. It's not really relevant here, unless you're using stored procs (which is not the way to go 90% of the time with EF - only use stored procs for more complex procedures).
An EF context, by its very nature, can only have a single database associated with it. You need to create multiple contexts in order to access multiple databases at once.
What I would do in your case is create a database-first schema (.edmx) file for each database, then write a service layer abstraction above it that allows you to flatten the data into your expected model. This is the kind of thing I do all the time, regardless of how many databases I'm working on at once. You've almost outlined this in your first diagram. The service layer may have multiple classes (for example, for a blog website you might have BlogService, UserService, CommentService etc), each of which contain methods that you call from you application layer.
I've put a quick diagram together that might help to explain
http://www.gliffy.com/go/publish/image/4818386/L.png
The service layer does all of your EF work, and your application layer (or business layer, whatever you want to call it) will do all of your business logic.
This setup lends itself well to TDD and Dependency Injection / IoC. Everything is neat and nicely separated.
I have the following scenario:
Server
SQL Server 2008
Core (Entity Framework and business logic)
WCF Service
MVC Web application (for backend management)
Client
Local Database - a simplified model of the main database
WPF Client
Requirements
The client has to work fully offline, and persist data
Changed data should be pulled from the server over WCF service
Client should not change the data, but call a a WCF method (if not available queue the call)
Possible Solutions
Microsoft Sync Framework - I think its an overkill, because I mainly need one way synching, and also the data structure is not the same.
Datasets serialization over WCF, yes, because Datasets support merging and offline scenarios, but isn't It out of date?
Entity Framework? I tried to build a prototype, but EF doesn't seem to support my needs very well (I need to search for an entity, and change it if modified, or add it if not existant)
Question
What, do you think, is the most appropriate approach?
Is SQL Server Compact a good local db?
I am very interested in your thoughts. Thank you!
The Microsoft Sync Framework is in my opinion not appropriate because you have differing schemas. I can also imagine that you have some business rules about what data is allowed to change and how it should be synced.
The choice between DataSets and Entity Framework depends on your needs. An Object-Relation-Mapper comes in to view when you are really using an object model.
If your domain is complex enough and you have the knowledge using a full fledged Domain Model is definitely a nice solution that can scale really well and handle complex projects.
If your project is somewhat simpler and you don't want to build a Domain Model you can choose for DataSets.
Personally, I think that learning the ins and outs of Domain Modeling and the Entity Framework as an ORM is a nice choice for projects. When you have enough experience using these technologies, you will even favor them on small projects.
About the problems you where having with your EF prototype.
Because the data schema of the client and the server is different I would use custom Data Transfer Objects for moving data between the two. This way, you decouple the object models and they can change independent of each other.
The client knows everything there is to know about the data changes. Because it has a local representation of the server data it knows if data is added, changed or deleted. Why don't you add this knowledge to your server call? If you use a field in your DTO that states if the object is Added, Modified or Deleted, the server won't have to detect this.
Please excuse the noob question as I am new to integrating data with my applications. I've tried to find answers on the net, but not there yet.
I have an application I'm developing in C# on VS2010 which requires data in/out from a database. I am trying to figure out if its a DataSet or Entity Data Model I need to use when setting up a data source. My understanding was that it was the EDM which allowed me to treat tables/fields in a database as objects, but somehow it looks like I can do that with a DataSet too.
Some sources explain that a DataSet makes a cached copy of the Database which can then be manipulated.
Essentially my question is which should I use and what are the (dis)advantages of one over the other.
You have several options open to you when it comes to storing and retrieving data to/from a database:
At the very simplest level, use ADO.NET to open a connection to the DB, create a command and execute it. If you expect results back (i.e. SELECT ...) then you could call the command's ExecuteReader(...). Working in this manner results in very quick execution and the minimum of overhead, but you have to do more of the heavy lifting. If your app is simple, this is probably a good way to go. If your app is, or is likely to be more complex, you may want to consider other options...
ADO.NET DataSets are a reasonable DB IO mechanism, particularly for reading data from a DB. However, they can be a little cumbersome when trying to update the DB.
You could use an Object-Relational Mapper (ORM) like nHibernate or Entity Framework, but, frankly, that often results in your learning curve increasing dramatically while you figure out how to plug together the moving parts and make them work well together.
You might also consider a new variant of Entity Framework called Code First (CF): This allows you to pretty much design your code and CF will generate your EDM and handle the majority of the DB operations required for you to build your system. Scott Hanselman wrote up a nice intro into EF CF.
Having used practically every DB API and ORM on Windows over the last 20+ years, I am delighted with how CF is shaping up! EF 4.3 that shipped just a couple of weeks ago includes some key new improvements to CF including migrations which allow you to handle changes to your DB schema as it evolves. I've build 3-4 systems using EF CF over the last couple of months and am very happy - it's my favorite relational database IO mechanism at present.
If you want to really get into EF CF, I strongly recommend Julia Lerman's book EF CF - it's a short, nicely written, very useful guide that should take you no more than a day or two to work through the main sections of.
Hope this helps.
If you add a LocalDB data source to your project (because you want a small local database file) then when the Data Source Configuration Wizard pops up, it explicitly asks you whether you want to use a Dataset or Entity Data Model database model. Is this the situation you were facing? That was the problem I had that brought me to this entry.
There is no question that for an enterprise class application, or a website, you would want to investigate ADO.NET or an ORM, but it doesn't help answer this question, which has to do with what are the differences between choosing Dataset vs Entity Data Model in the wizard.
Essentially, Entity Data Model is the more recent technology. If you are unfamiliar with Dataset, then this is probably not the time to start using it.
If you're asking what are the pros and cons for ADO.NET (DataSet) vs EntityFramework (Entity Data Model) then there is a discussion that may help at ADO.NET Entity Framework or ADO.NET
EF will get you up and running pretty quickly but in my (very limited) experience its been a pain to maintain.
What is it that has determined that these are your only two options? There are far more available to you including many ORMs.
If your application is supporting a business application than queries get complex pretty soon. In such scenario, stored-procedures save a lot of time and are much easier to maintain and they work better with ADO.NET. In almost all scenarios, I would suggest using stored-procedures and ADO.NET. Move as much of the business rules and logic to stored procedures as you can...much easier to maintain this way.
Use Datasets (datatables) only to retrieve and read data. Any data that needs to be saved to database should be directly manipulated in the database ... no point doing it in dataset and then saving the same. In a multi-user environment it is almost always better to save the changes to database as soon as the user has clicked "save".
You may (should) use business objects within the application for business-logic processes.
Let us take a simple example of where you are saving a Contact (name, phone, email, address etc) and then retrieving a list of contacts added today...I would suggest you do it as follows:
1) Adding the contact - Client (web or otherwise) collects data --> data is saved in a Contact business object --> validate Contact object --> Call repository layer to save Contact object (adding a repository layer is useful but not-necessary to keep the data layer abstract from the client) --> Repository calls the data layer to save the contact object (here a simple ADO.NET call, using Command object, can be made to call the stored procedure to save the contact in database). No dataset was used in this use case.
2) Retrieving list of contacts -- Client calls the repository layer to get the list of contacts --> repository layer call the data layer to retrieve the data --> here the list of data is retrieved as a dataset(datatable) --> return the datatable back to the client and let the client read the data directly from datatable while rendering the data. Even a single contact can be retrieved as a dataset.
P.S: ORM is almost always an overkill. It is almost always used because certain developers like to keep everything object-oriented...so an extra layer gets added even though it does nothing useful (IMHO).
But, what if you have business logic (stored procedures) which can be used in many different applications.
So depends: if you make your application for different users with different backend storage, or you make many applications for users which doesn't change backend storage so often.
It is very important to have database integrity and rules independent from application (inner or outsource)
I have probably written the same LINQ to SQL statement 4-5 times across multiple projects. I don't even want to have to paste it. We use DBML files combined with Repository classes. I would like to share the same Library across multiple projects, but I also want to easily update it and ensure it doesn't break any of the projects. What is a good way to do this? It is OK if I have to change my approach, I do not need to be married to LINQ to SQL and DBML.
We have both console apps and MVC web apps accessing the database with their own flavor of the DBML, and there have been times when a major DB update has broken them.
Also, since currently each project accesses the DB from itself, which is sometimes on another server, etc. Would it be possible to eliminate the DB layer from being within each project all together? It might help the problem above and be better for security and data integrity if I could manage all the database access through a centralized application that my other applications could use directly rather than calling the database directly.
Any ideas?
The way I handle this is using WCF Data Services. I have my data models and services in one project and host this on IIS. My other projects (whatever they may be) simply add a service reference to the URI and then access data it needs over the wire. My database stuff happens all on the service, my individual projects don't touch the database at all - they don't even know a database exists.
It's working out pretty well but there are a few "gotchas" with WCF. You can even create "WebGet" methods to expose commonly used methods via the service.
Let me know if you want to see some example code :-)
I'm trying to tackle the problem of disconnected operation for an application with a relatively rich data layer, and it occurs to me that the most natural way to make this work is with a client-side database. I don't want to have to install a separate product, however, and I'm left to wonder if there are any layers out there where you can essentially link a database-like persistence layer into an application. Has anyone had any experience with this? Are there any good frameworks that cover this area?
I would recommend SQLite. It's a full SQL database engine wrapped in a single dll with no installation or maintenance that just ships with your app and runs in-process. There's a great .NET wrapper that integrates nicely and allows you to create custom functions in .NET.
http://sqlite.phxsoftware.com/
If you don't need the power of a relational database and want to simplify translation of your object model for persistence, you should look into DB4O - it's an object database that can run on your client and transparently persist your classes.
You can use NHibernate with sqlite or sqlce database. We use sqlce.
.Net has strongly typed datasets, which work great for this purpose.
http://msdn.microsoft.com/en-us/library/esbykkzb%28VS.71%29.aspx
Even thought you don't want to install another product, you might want to consider SQL Server Compact Edition. Although you do need to install it, it's free, and installs no new Windows services.
The databases themselves are simply a single file per database. LINQ to SQL and LINQ to Entities are still supported, and you can even get a Windows Mobile version.
Are you looking for a database-like persistence layer because you want the query power of a database on the client side, or for persistence between application runs, or both?
If you need both, or just the persistence, then any one of the other answers showcasing integrated DB libraries will do (like this one for SQL Lite).
However, if the only thing you need is the ability to perform complex queries against in-memory data then I would highly recommend using plain-ol LINQ-to-Objects, assuming the option is available to you.