I'm trying to tackle the problem of disconnected operation for an application with a relatively rich data layer, and it occurs to me that the most natural way to make this work is with a client-side database. I don't want to have to install a separate product, however, and I'm left to wonder if there are any layers out there where you can essentially link a database-like persistence layer into an application. Has anyone had any experience with this? Are there any good frameworks that cover this area?
I would recommend SQLite. It's a full SQL database engine wrapped in a single dll with no installation or maintenance that just ships with your app and runs in-process. There's a great .NET wrapper that integrates nicely and allows you to create custom functions in .NET.
http://sqlite.phxsoftware.com/
If you don't need the power of a relational database and want to simplify translation of your object model for persistence, you should look into DB4O - it's an object database that can run on your client and transparently persist your classes.
You can use NHibernate with sqlite or sqlce database. We use sqlce.
.Net has strongly typed datasets, which work great for this purpose.
http://msdn.microsoft.com/en-us/library/esbykkzb%28VS.71%29.aspx
Even thought you don't want to install another product, you might want to consider SQL Server Compact Edition. Although you do need to install it, it's free, and installs no new Windows services.
The databases themselves are simply a single file per database. LINQ to SQL and LINQ to Entities are still supported, and you can even get a Windows Mobile version.
Are you looking for a database-like persistence layer because you want the query power of a database on the client side, or for persistence between application runs, or both?
If you need both, or just the persistence, then any one of the other answers showcasing integrated DB libraries will do (like this one for SQL Lite).
However, if the only thing you need is the ability to perform complex queries against in-memory data then I would highly recommend using plain-ol LINQ-to-Objects, assuming the option is available to you.
Related
I'm a very beginner in .NET and now I'm developing a little project (web API) using NancyFX framework. In my project, I need to use SQL database for some very basic tasks like storing registered users' details or getting some user information. I'd like to know what is the most popular, convenient and modern way of using SQL in .NET for beginners? I mean, should I use LINQ or just pure SQLClient functionality or are there any good libraries for working with SQL on .NET? I've tried to implement LINQ to SQL pattern but ended up with huge chunks of unused auto generated code and even bigger mess in my head...
For a framework to communicate with you're database I would recommend using Entity framework, its very convenient and easy and has the Code first approach which you should read about.
More over i suggest you follow the repository pattern,
https://msdn.microsoft.com/en-us/library/ff649690.aspx
This basically means - each object you save in the db, will have a repository which will contain all the object of its kind and that will be you're entry point to reading/inserting/updatibg/and deleting rows from the db, while abstracting away all details of implementation - in our case I recommend entity framework as I mentioned before.
Good luck
So far, in my applications, I stored all my data via serialization, so I never really needed anything else. But now I am working with more complicated data and I need more than a simple file to store them. Is there some inbuilt storage engine in Windows I can use, which will allow me to pull and edit data with SQL queries? (Since I doubt the user will be willing to install and configure standalone MySQL server just for my application and I don't really want to use 3rd party solutions)
And if there is, how can I access such database engine?
Yes, Windows does have an embedded database engine, which has been there since Windows 2000. It is called Esent.
There is a project called Managed Esent to expose the features of Esent to managed code. There is a NuGet package for it.
I do not know whether it fits your exact need of updates via SQL queries but it does boast a number of features:
ACID transactions with savepoints, lazy commits, and robust crash recovery.
Snapshot isolation.
Highly concurrent database access.
Flexible meta-data (tens of thousands of columns, tables, and indexes are possible).
Indexing support for integer, floating point, ASCII, Unicode, and binary columns.
Update
I have since had some experience with Esent in C# via the nuget package. It does NOT have SQL query capabilities. There is support for indices and basic cursors over them but nothing like the power of SQL. My suggestion if you need a light-weight SQL is definitely SQLite.
SQLite is a software library that implements a self-contained, serverless, zero-configuration, transactional SQL database engine.
http://www.sqlite.org
SQLite will work without requiring your App User to install SQLExpress or any other database engine, making it a good option for desktop applications.
Use SQLite as standalone database and integrated part of your applications.
Main benefits:
data is stored inside a single file, easy to pull/edit
ships as part of your application as dll, no separate
installation/configuration needed
i found some info about the database on the Microsoft website database info
there is a embedded database engine. Esent
If it is small set of user specific data, storing at windows registry is a good option.
No, there is no inbuilt database available.
But there are good alternatives, like the mqsql express server or a SQLITE database, which is basically a single database file which is accessed through a library in your project. So if you use SQLITE, you do not need to instal anything.
I'm building a C#/WPF job search tracking application to keep track of resumes submitted, interviews, followups, etc and am not sure of the best way to store the data. Where/how would YOU store the data? My first thought was XML to keep it simple, but it seems like I should "model" my data since there will be lots of related bits of information. Would SQLite be a better choice? Other recommendations?
Since I assume you want to query and update that data I would even suggest an ORM like Entity Framework - it's easy to get started and the basic stuff like querying and updating will be very straightforward if you have worked with LINQ before - saves you the hassle of writing your own SQL queries. This also will allow you to easily extend your model later on should you decide to do so.
Edit:
There are self-contained light-weight alternatives that would still allow you to use LINQ:
SQL Server Compact:
Microsoft SQL Server Compact is a free
SQL Server embedded database ideal for
building standalone and occasionally
connected applications for mobile
devices, desktops, and Web clients.
Here' an article that describes how to get LINQ to SQL to work on it. Apparently you can also use LINQ to Entities on it but there's quirks (such as design-time support) that you'll have to work around.
SQLLite:
SQLite is a software library that
implements a self-contained,
serverless, zero-configuration,
transactional SQL database engine.
SQLite is the most widely deployed SQL
database engine in the world.
There's a LINQ provider for it called DBLinq
As #Robert Harvey pointed out in his answer there's (almost) full support for EF since there's an ADO.NET provider for SQLLite:
Support for the ADO.NET 3.5 Entity
Framework
Supports nearly all the
entity framework functionality that
Sql Server supports, and passes 99% of
the tests in MS's EFQuerySamples demo
application.
Generally, you want to store the data from your application in a database. For WPF and C#, that database is usually SQL Server or SQL Server Express, because Visual Studio 2008 easily integrates with those.
Do not use XML for this. XML is not intended to be a large scale storage medium; the purpose of XML is to provide a common language for different computer systems to talk to each other.
SQL Server Express is usable on any Windows PC; you just need to deploy the redistributable, using a named instance. See also http://msdn.microsoft.com/en-us/library/dd981032(SQL.100).aspx. If it's a small application and you want to go simpler, you can use SQL Server Compact Edition, or SQLite with the ADO.NET provider.
I'm writing a .NET application and the thought of implementing a data layer from scratch is icky to me. (By data layer I'm referring to the code that talks to the database, not the layer which abstracts the database access into domain objects [sometimes called the data access layer and used interchangeably with data layer].)
I'd like to find an existing generic data layer implementation which provides standard crud functionality, error handling, connection management - the works. I'll be talking to SQL Server only.
It doesn't matter to me if the library is in C# or VB.NET and I don't care if it's LINQ or ADO.NET. As long as it works.
** I want to emphasize that I'm not looking for data access technologies or mechanisms (e.g. LINQ, ORM tools, etc.) but rather existing libraries.)
If you are talking to only SQL Server the Linq to SQL is your best option. It is pretty easy to get up and running. You will get both the Data Layer and the Abstraction. All you have to do is provide a connection string to Linq to SQL and it will handle the rest.
If you are going to connect to other database than SQL you would want to with NHibernate.
NHibernate takes a little more work than Linq to SQL to get up and running. MS provided in Visual Studio a nice tool that can get you reading from a SQL database pretty quick.
Honestly as much of a fan as I've always been with NHibernate. With the latest release of Enterprise Library 5 Data Access Block that they added in the dynamic mapping support natively. I would have to strongly consider not using NHibernate on a project and instead use a forward database generation tool from my domain objects to create my database (perhaps even use NHibernate solely for the scheme export) or something like CodeSmith and use EntLib.
You can use easyobjects has a very small learning curve, and is very extensible.
From their web:
EasyObjects.NET is a powerful data-access architecture for the .NET Framework. When used in combination with code generation, you can create, from scratch, a complete data layer for your application in minutes.
I'd like to find an existing generic data layer implementation which provides standard crud functionality, error handling, connection management - the works. I'll be talking to SQL Server only.
Might want to check out Subsonic. Though I personally find it quite limited, it's certainly not an ORM, but a "query tool." It will make CRUD operations easy and straightforward, and it generates partial POCO classes for every table in your database, rather than trying to map from a database to a domain layer.
Microsoft's Entity Framework might be what you are looking for to releave you from writing "the code that talks to the database".
The best things are that it already ships with Visual Studio and - depending on your requirements - you can use most functionality out-of-the box or manually adjust it to your custom business logic via T4 templates.
You can use it for forward and reverse engeneering and being a microsoft technology it integrates well with other MS products like SQL server.
I started using it 3 months ago in my current project at work which is composed of several windows and WCF services to convert third party data into our own database scheme. From the experiences we made with it, we'll be using the EF in future project a lot more.
What would you expect this framework to do with your exceptions? If it can't connect to your database, what should it do - crash the application, show an error message (winforms or WPF or ASP)... the questions are endless.
An ORM such as those suggested elsewhere in these answers is likely to be the closest you're going to get. Expecting a third party framework to provide all your exception handling isn't realistic - how would a third party know how your application is supposed to behave?
The direct answer to your question asking for "an existing generic data layer implementation which provides standard crud functionality, error handling, connection management - the works" is simple: use ADO.NET. The answers everyone else have provided actually go beyond that functionality, but your responses suggest that you think that there's something even further beyond - something that implements your data layer for you. My suggestion is that what you're looking for probably doesn't exist.
(EDIT: I made it a community wiki as it is more suited to a collaborative format.)
There are a plethora of ways to access SQL Server and other databases from .NET. All have their pros and cons and it will never be a simple question of which is "best" - the answer will always be "it depends".
However, I am looking for a comparison at a high level of the different approaches and frameworks in the context of different levels of systems. For example, I would imagine that for a quick-and-dirty Web 2.0 application the answer would be very different from an in-house Enterprise-level CRUD application.
I am aware that there are numerous questions on Stack Overflow dealing with subsets of this question, but I think it would be useful to try to build a summary comparison. I will endeavour to update the question with corrections and clarifications as we go.
So far, this is my understanding at a high level - but I am sure it is wrong...
I am primarily focusing on the Microsoft approaches to keep this focused.
ADO.NET Entity Framework
Database agnostic
Good because it allows swapping backends in and out
Bad because it can hit performance and database vendors are not too happy about it
Seems to be MS's preferred route for the future
Complicated to learn (though, see 267357)
It is accessed through LINQ to Entities so provides ORM, thus allowing abstraction in your code
LINQ to SQL
Uncertain future (see Is LINQ to SQL truly dead?)
Easy to learn (?)
Only works with MS SQL Server
See also Pros and cons of LINQ
"Standard" ADO.NET
No ORM
No abstraction so you are back to "roll your own" and play with dynamically generated SQL
Direct access, allows potentially better performance
This ties in to the age-old debate of whether to focus on objects or relational data, to which the answer of course is "it depends on where the bulk of the work is" and since that is an unanswerable question hopefully we don't have to go in to that too much. IMHO, if your application is primarily manipulating large amounts of data, it does not make sense to abstract it too much into objects in the front-end code, you are better off using stored procedures and dynamic SQL to do as much of the work as possible on the back-end. Whereas, if you primarily have user interaction which causes database interaction at the level of tens or hundreds of rows then ORM makes complete sense. So, I guess my argument for good old-fashioned ADO.NET would be in the case where you manipulate and modify large datasets, in which case you will benefit from the direct access to the backend.
Another case, of course, is where you have to access a legacy database that is already guarded by stored procedures.
ASP.NET Data Source Controls
Are these something altogether different or just a layer over standard ADO.NET?
- Would you really use these if you had a DAL or if you implemented LINQ or Entities?
NHibernate
Seems to be a very powerful and powerful ORM?
Open source
Some other relevant links;
NHibernate or LINQ to SQL
Entity Framework vs LINQ to SQL
I think LINQ to SQL is good for projects targeted for SQL Server.
ADO.NET Entity Framework is better if we are targeting different databases. Currently I think a lot of providers are available for ADO.NET Entity Framework, Provider for PostgreSQL, MySQL, esql, Oracle and many other (check http://blogs.msdn.com/adonet/default.aspx).
I don't want to use standard ADO.NET anymore because it's a waste of time. I always go for ORM.
Having worked on 20+ different C#/ASP.NET projects I always end up using NHibernate. I often start with a completely different stack - ADO.NET, ActiveRecord, hand rolled wierdness. There are numerous reasons why NHibernate can work in a wide range of situations, but the absolutely stand out for me is the saving in time, especially when linked to code generation. You can change the datamodel, and the entities get rebuilt, but most/all the other code doesn't need to be changed.
MS does have a nasty habit of pushing technologies in this area that parallel existing open source, and then dropping them when they don't take off. Does anyone remember ObjectSpaces?
Added for new technologies:
With Microsoft Sql Server out for Linux in Beta right now, I think it's ok to not be database agnostic. The .Net Core Path and MS-SQL route allows you to run on Linux servers like Ubuntu entirely with no windows dependencies.
As such, imo, a very good flow is to not use a full ORM framework or data controls and leverage the power of SSDT Visual Studio Projects (Sql Server Data Tools) and a Micro ORM.
In Visual Studio you can create a Sql Server Project as a legit Visual Studio Project. Doing so allows you to create the entire database via table designers or raw query editing right inside visual studio.
Secondly, you get SSDT's Schema Compare tool which you can use to compare your database project to a live database in Microsoft Sql Server and update it. You can sync your Visual Studio Project with the server causing updates in your project to go out to the server. Or you can sync the server with your project causing your source code to update. Via this route you can easily pick up changes the DBA made in maintenance last night and push out your new development changes for a new feature easily with a simple tool.
Using that same tool you can compute the migration script without actually running it, if you need to pass that off to an operations department and submit a change order, it works for that flow to.
Now for writing code against you MS-SQL Database, I recommend PetaPoco.
Because PetaPoco works Perfectly inline with the above SSDT solution. PetaPoco comes with T4 text templates you can use to generate all your data entity classes, and it generates the bulk data layer classes for you.
The catch is, you have to write queries yourself, which isn't a bad thing.
So you end up with something like this:
var people = dbContext.Fetch<Person>("SELECT * FROM People where Username Like '%#0%'", "bob");
PetaPoco automatically handles parameterizing #0 for you, it also has the handy Sql class for building queries.
Furthermore, PetaPoco is an order of magnitude faster than EF6 and 8+ times faster than EF7.
So in total, this solution involves using SSDT for SCHEMA management, and PetaPoco for code integration at the gain of high maintainability, customization, and very good performance.
The only downfall to this approach, is that you're hard tieing yourself to Microsoft Sql Server. However, imo, Microsoft Sql Server is one of the best RDBM's out there.
It's got DBMail, Jobs, CLR object capabilities, and on and on. Plus the integration between Visual Studio and MS-SQL server is phenomenal and you don't get any of that if you choose a different RDBMS.
I must say that I never used NHibernate for the immense time that needed to start using... time wasted on the XML setup.
I recently did a web application in MVC2, where I did choose ADO Entities Framework and I use Linq all the time.
I must say, I was impressed with the speed! and our site was having around 35 000 unique visitors per day, in around 60Gb bandwidth per day (I reduced radically this 60Gb number by hosting all static files in Amazon S3 - Great .NET wrapper they have, I must say).
I will always go this way. It's easy to start (just add new data item, choose tables and that's it! for every change in the database we just need to refresh the model - made automatically in just 2 clicks) and it's fun to use - Linq rules!