We Keep Coding

Best practise for SQL in .NET project

I'm a very beginner in .NET and now I'm developing a little project (web API) using NancyFX framework. In my project, I need to use SQL database for some very basic tasks like storing registered users' details or getting some user information. I'd like to know what is the most popular, convenient and modern way of using SQL in .NET for beginners? I mean, should I use LINQ or just pure SQLClient functionality or are there any good libraries for working with SQL on .NET? I've tried to implement LINQ to SQL pattern but ended up with huge chunks of unused auto generated code and even bigger mess in my head...

For a framework to communicate with you're database I would recommend using Entity framework, its very convenient and easy and has the Code first approach which you should read about.
More over i suggest you follow the repository pattern,
https://msdn.microsoft.com/en-us/library/ff649690.aspx
This basically means - each object you save in the db, will have a repository which will contain all the object of its kind and that will be you're entry point to reading/inserting/updatibg/and deleting rows from the db, while abstracting away all details of implementation - in our case I recommend entity framework as I mentioned before.
Good luck

How can I improve my business layer objects mapping into a database? Is it time for a O/R mapper?

As I began writing web applications with ASP.NET I started with small projects that used a Linq-To-SQL mapper for database access to a MSSQL Server.
After gaining some experience, I switched into a classic three-tiered approach with a graphic Layer, business Layer, and a data Layer. The only function of the data layer was to provide insert/update/delete-methods without any logic and logic the form of selection methods.
Over the time I realized that it would be better not to provide the database classes up to the GUI (took some time, unfortunately). I switched to using business classes in the BL that are used for all operations performed by the BL and displayed by the GUI in the form of getting List from the business layer.
A great advantage is that I can provide additional properties that are not represented by the database itself. However, I did that mapping inside the business layer myself with methods that mapped the corresponding business layer class to the database class.
I guess that's where O/R mapper come in handy? Until now, I haven't realized their purpose, but I think I just found it. I've recently tried out using the new Entity Framework with .NET Framework 4, but I'm only using it like the Linq-To-SQL DataContext.
Is there a way to achieve the mapping automatically? If yes, is that something the new Entity Framework provides or do I need to look for a O/R Mapper like NHibernate?

I use NHibernate exclusively in my projects. I like the control and flexibility it gives me. There is a 'shortcut' called Active Record that uses NHibernate under the covers but provides a really nice an simple interface to NHibernate.
NHibernate has a steep learning curve, but when you get past that - it is really smooth sailing. When (and if) you venture the way of NHibernate, check out Ayende for cool tips.

(Entity Framework is an O/R Mapper.)
If you're serious about getting your hands dirty with ORM (but relatively new to that area), I highly recommend something like TekPub's videos on these topics. You'll be able to see these tools in use starting from scratch. It is a graceful introduction to some simple, but real-world issues like the ones you mention.

LinqToSql is an ORM, so you are already using one. Taking LinqToSql out and replacing it with EntityFramework or NHibernate won't solve the problems you appear to be having right now.
Here are some things you should learn more about to help give you additional context:
AutoMapper
Data Transfer Objects (DTOs)
Plain Old CLR Object (POCO)

I've had a great time using Entity Framework 4.0 (+ the CTP). I think you'd have a much easier time dealing with an ORM like that. EF4 provides everything you need to interoperate with MSSQL from C#/.NET. You won't have to write a single line of SQL, and it has full support for LINQ (through ObjectQuery).

Basic clarifications about NHibernate

Given that I'm very good with SQL and c#,
Should I learn another layer on top like NHibernate ?
Is NHibernate just a library (that stores in a Database)? Or it's a service?
Should I use NHibernate or ADO.NET Entity Framework?
If you think I should learn/use an ORM, give me your top reason.

You should use an ORM as long as you need to convert database data to and from business objects, since it will save you a lot of work and will allow you to focus on your application logic.
NHibernate is a .NET library that does just that, mapping .NET objects to database tables according to how you configure it. In this sense it is the same as the Entity Framework, only that EF is already embedded in the .NET framework and NHibernate is a separate assembly that you must reference in your project.
Last but not least, if you use SQL Server you should add LINQ to SQL to the list of possible ORM candidates, it is simpler that EF and for many scenarios it is more than appropriate.

It depends on your applications.
NHibernate is a library. So it's a DLL.
Depends on what you want. NHibernate is based on Hibernate which is battle tested.
It doesn't matter how good anyone is with SQL or C#. There is a fundamental gap with the tools when dealing with SQL and C#. Aside from all the other productivity boosts that I've had when I learned to Stop Worrying and Just Use an ORM, I found only having to deal with C# most of the time has helped greatly. I have far fewer impedance mismatches in my work now and I do believe that contributes to fewer bugs.
Less code you have to write is less code you have to maintain. ORMs allow you to worry less about certain details so you are free to concentrate on higher level tasks.

No, I tried Fluent NH and Castle Active Record and Spring Framework NH Extensions but they all obscure basic operations and make things less visible. Start using native NH, then add a layer after a year.
Yes, NH is a library, not a service. But the way you use it in your code makes it feel almost like a service (e.g. a data repository service)
I tried EF and found it nauseating so I would go with NH
For OLTP-like systems, ORM is the way of the future. Not using ORM for me is like not using unit-tests or programming in non-OOP language.

Probably, but it depends on what kind of applications you normally write.
NHibernate is primarily a DLL, but there is more to it than that.
NHibernate (Read this for more details: NHibernate, Entity Framework, active records or linq2sql)
My top reason would be so you can use Linq. Right now, you pretty much need an ORM to use Linq.

Unless it's a very small application, then the answer is 'yes'.
Library.
I hear people swear by the EF, but I'm very leery of it. I also don't like tying myself to all Microsoft technologies. NHibernate would be my suggestion.
First, you don't want to go through the time and headache of writing all the SQL and classes and such; it's just not worth it. Second, it allows for greater ability to switch from one RDBMS to another without having to change much code. Third, it'll give you more control in the future in terms of database abstraction and such.

Should I Use Entity Framework, DataSet or Custom classes?

I am really having a hard time here. I need to design a "Desktop app" that will use WCF as the communications channel. Its a multi-tiered application (DB and application server are the same, the client goes through the internet cloud).
The application is a little complex (in terms of SQL and code logics) then the usual LOB applications, but the concept is the same: Read from DB, update to DB, handle concurrency etc. My problem is that now with Entity Framework out in the open, I cant decide which way to proceed: Should I use Entity Framework, Dataset or Custom Classes.
As I understand by Entity Framework, it will create the object mapping of my DB tables ALONG WITH the CRUD scripts as well. Thats all well and good for simple CRUD, but most of the times the "Select" is complex and it requires a custom SQL. I understand I can use Stored Procedures in EF (I dont like SP btw, i dont know why, I like to code my SQL in the DAL by hand, I feel more secure and comfortable that way).
With DataSet, I will use my custom SQLs and populate on the data set. With Custom classes (objects for DB tables) I will populate my custom SQLs on those custom classes (collections and lists etc). I want to use EF, but i dont feel confident in deploying an application whose SQL I have not written and cant see in the code. Am I missing something here.
Any help in this regard would be greatly appreciated.
Xeshu

I would agree with Marc G. 100% - DataSets suck, especially in a WCF scenario (they add a lot of overhead for handling in-memory data manipulation) - don't use those. They're okay for beginners and two-tier desktop apps on a small scale maybe - but I wouldn't use them in a serious, professional app.
Basically, your question boils down to how do you transform your rows from the database into something you can remote across WCF. This means some form of mapping - either you do it yourself, using DataReaders and then shoving all the data into WCF [DataContract] classes - you can certainly do that, gives you the ultimate control, but it's also tedious, cumbersome, and error-prone.
Or you let some ready-made ORM handle this grunt work for you - take your pick amongst Linq-to-SQL (great, easy-to-use, flexible, but SQL Server only), EF v4 (out by March 2010 - looks very promising, very flexible) or any other ORM, really - whatever suits your needs best.
Other serious competitors in the ORM space might include Subsonic 3.0 and NHibernate (amongst many many others).
So to sum up:
forget about Datasets
either you have 100% control and to the mapping between SQL and your objects yourself
you let some capable ORM handle that (Linq-to-SQL, EF v4, Subsonic, NHibernate et al) - which one really doesn't matter all that much, i.e. it's also a matter of personal preference and coding style

I can't advocate datasets, especially in an SOA environment like WCF - it'll work, but for mostly the wrong reasons. They simply aren't portable, and IMO don't really "work" over service boundaries. Of course, IMO they don't work in most other scenarios too ;-p
So then it comes down to how much plumbing you want to do. Most ORMs will create WCF-serializable types for you; personally I'd use LINQ-to-SQL at the moment; it is both simpler and more complete than EF, although EF 4.0 is meant to be much better than EF in 3.5sp1. You can use custom TSQL (via ExecuteQuery, which still does the mapping back to objects), but I tend to use either SPROC (for complex queries) or LINQ-generated queries (for simple requests).
Writing the types yourself is fine too, and will work with NHibernate etc. So many options.

While EF works with WCF and sounds very promising, you should consider the effort to get on speed with it. Especially when doing some non trivial stuff, the designer in VS2008 can't open the model anymore and you have to code your model in xml.
Also keep in mind that EF works on a very high abstraction level. Because of the law of leaky abstractions its not all that shiny as it supposed to be :)
The other way round that means, you have to deal with very crazy and hard to read sql statements sent to your database when it comes to troubleshooting / performance issues.

Best design practices for .NET architecture with LINQ to SQL (DAL necessary? Can we truly use POCOs? Design pattern to adopt?)

I was avoiding writing what may seem like another thread on .net arch/n-tier architecture, but bear with me.
I, hopefully like others still am not 100% satisfied or clear on the best approach to take given today's trends and new emerging technologies when it comes to selecting an architecture to use for enterprise applications.
I suppose I am seeking mass community opinion on the direction and architectural implementation you would chose when building an enterprise application utilising most facets of today's .NET technology and what direction you would take. I better make this about me and my question, in fear of this being too vague otherwise; I would like to improve my architecture to improve and would really like to hear what you guys think given the list of technologies I am about to write.
Any and all best practices and architectural patterns you would suggest are welcome and if you have created a solution before for a similar type setup, any pitfalls or caveats you may have hit or overcome.
Here is a list of technologies adopted in my latest project, yep pretty much everything except WPF :)
Smart Client (WinForms)
WCF
Used by Smart Client
ASP.NET MVC
Admin tool
Client tool
LINQ to SQL
Used by WCF
Used ASP.NET MVC
Microsoft SQL Server 2008
Utilities and additional components to consider:
Dependency Injection - StructureMap
Exception Management - Custom
Unit Testing - MBUnit
I currently have this running in an n-Tier arch. adopting a Service-based design pattern utilising Request/Response (Not sure of it's formal name) and the Repository pattern, most of my structure was adopted from Rob Conery's Storefront.
I suppose I am more or less happy with most of my tiers (It's really just the DAL which I am a bit uneasy on).
Before I finish, these are the real questions I have faced with my current architecture:
I have a big question mark on if I should have a custom data access layer given the use of LINQ to SQL. Should I perform LINQ to SQL directly in my service/business layer or in a DAL in a repository method? Should you create a new instance of your DB context in each repository method call (using using())? or one in the class constructor/through DI?
Do you believe we can truly use POCO (plain old CLR objects) when using LINQ to SQL? I would love to see some examples as I encountered problems and it would have been particularly handy with the WCF work as I can't obviously be carrying L2S objects across the wire.
Creating an ASP.NET MVC project by itself quite clearly displays the design pattern you should adopt, by keeping view logic in the view, controller calling service/business methods and of course your data access in the model, but would you drop the 'model' facet for larger projects, particularly where the data access is shared, what approach would you take to get your data?
Thanks for hearing me out and would love to see sample code-bases on architectures and how it is split. As said I have seen Storefront, I am yet to really go through Oxite but just thought it would benefit myself and everyone.
Added additional question in DAL bullet point. / 15:42 GMT+10

To answer your questions:
Should I perform LINQ to SQL directly in my service/business layer or in a DAL in a repository method? LINQ to SQL specifically only makes sense if your database maps 1-to-1 with your business objects. In most enterprise situations that's not the case and Entities is more appropriate.
That having been said, LINQ in general is highly appropriate to use directly in your business layer, because the LINQ provider (whether that is LINQ to SQL or something else) is your DAL. The benefit of LINQ is that it allows you to be much more flexible and expressive in your business layer than DAL.GetBusinessEntityById(id), but the close-to-the-metal code which makes both LINQ and the traditional DAL code work are encapsulated away from you, having the same positive effect.
Do you believe we can truly use POCO (plain old CLR objects) when using LINQ to SQL? Without more specific info on your POCO problems regarding LINQ to SQL, it's difficult to say.
would you drop the 'model' facet for larger projects The MVC pattern in general is far more broad than a superficial look at ASP.NET MVC might imply. By definition, whatever you choose to use to connect to your data backing in your application becomes your model. If that is utilizing WCF or MQ to connect to an enterprise data cloud, so be it.

When I looked at Rob Connery's Storefront, it looked like he is using POCOs and Linq to SQL; However, he is doing it by translating from the Linq to SQL created entities to POCO (and back), which seems a little silly to me - Essentially we have a DAL for a DAL.
However, this appears to be the only way to use POCOs with Linq to SQL.
I would say you should use a Repository pattern, let it hide your Linq to SQL layer(or whatever you end up using for data access). That doesn't mean you can't use Linq in the other tiers, just make sure your repository returns IQueryable<T>.

Whether or not you use LINQ-to-SQL, it is often cmomon to use a separate DTO object for things like WCF. I have cited a few thoughts on this subject here: Pragmatic LINQ - but for me, the biggest is: don't expose IQueryable<T> / Expression<...> on the repository interface. If you do, your repository is no longer a black box, and cannot be tested in isolation, since it is at the whim of the caller. Likewise, you can't profile/optimise the DAL in isolation.
A bigger problem is the fact that IQueryable<T> leaks (LOLA). For example, Entity Framework doesn't like Single(), or Take() without an explicit OrderBy() - but L2S is fine with that. L2S should be an implementation detail of the DAL - it shouldn't define the repository.
For similar reasons, I mark the L2S association properties as internal - I can use them in the DAL to create interesting queries, but...