We Keep Coding

LINQ to Entities v/s ADO.Net

Can LINQ to Entities entirely replace ADO.Net technology?
What I really want to know is, Is it possible to achieve every features and functionality that ADO.Net provides by using LINQ ? Who is the Boss?

Since you said in your comment that you mean LINQ to Entities, I will answer thus:
LINQ to Entities internally use ADO.Net, so it would be impossible for LINQ to Entities to replace ADO.Net, because it uses it. However I am assuming that you mean to ask if you can use LINQ to Entities in every situation that you could use ADO.Net. The answer is no, you cannot.
There are cases that you would not want to (or could not) use LINQ to Entities. For example if you are in a certain situation where you need better performance for a query, it would be better to use a stored procedure. Generally LINQ to Entities is a better choice when developer productivity has a higher priority than the execution speed of queries. Other cases where LINQ to Entities is not a good choice is when a DB has been poorly designed and does not have a primary key for example.
But the main point I think is this: What has higher priority in your situation? Developer productivity or performance? It is also quite acceptable I think to use both technologies. Complete the project with LINQ to Entities, and if there are performance issues, use stored procedures or ADO.Net.
Also, I would not use LINQ to Entities for large queries, or queries that have many joins, LINQ syntax for joins is not very terse, and may hurt the readability of your code.

#Reddy, it entirely depends on the scenario where you are deciding which technology to use. ADO.NET gives you tuple structure using DataSet at its focus and is used even day compared to ORM like tech i.e. LINQ-Entity. I have refrained to ADO.NET when the prime aspect of the application revolves around record processing, used in the SSIS pipeline but have stuck to LINQ-Entity when I primarily follow Test Driven Development or Domain Driven Development approaches.

1)ADO.NET is the database connectivity technology, so we can use ado.net to do operations with only Databases , but if we get a requirement to get the data from non database datasources like xml,collections etc.. its not possible.
But using LINQ it is possible to get data from many datasources like collections, XML files, Entities, even with Databases also.
2) But if you come to the performance wise ADO.NET is very faster than any other database connective technologies.
3) If you go with LINQ Run time errors can be rectified so bugs will be less ,so reliability increases
so ADO.NET vs LINQ technologies both having their own positives and negative features ,Microsoft given huge amount of featured technologies, so according to the application requirement we need to go with that technology.

Do SQL Queries Need to be that Complicated?

An open-ended question which may not have a "right" answer, but expert input on this would be appreciated.
Do SQL Queries Need to be that Complicated?
From a Web Dev point of view, as C#/.Net progresses, it seems that there are plenty of easy ways (LINQ, Generics) to do a lot of the things that some people tend to do in their SQL queries (sorting, ordering, merging, etc). That being said, since SQL tends to be the processing "bottleneck" for a lot of apps, a lot of the logic for SQL queries is being moved to the business layer.
As this trend continues, I'm seeing less of a need for large SQL queries.
What do you all think? Are you still writing large SQL queries? If so, is it because you need to or because you are more comfortable doing so than working in the business layer?

What's a "large" query?
The "bottleneck" encountered IME is typically because the tables were modeled poorly, compounded by someone constructing SQL queries that has little to no experience with SQL (the most common issue being thinking SQL is procedural when it's actually SET based). Lack of indexing is the next most common issue.
ORM has evolved to support native queries -- clear recognition that ORM simplifies database interaction, but can't perform as well as proper SQL query development.
Keeping the persistence handling in the business layer is justified by desiring database independence (at the risk of performance). Otherwise, it's a waste of money and resources to ignore what the database can handle in far larger loads, in a central location (that can be clustered).

It depends entirely on the processing. If you're trying to do lots of crazy stuff in your SQL which does things like pivoting or text processing, or whatever, and it turns out to be faster to avoid doing it in SQL and process it outside the database server instead, then yes, you were probably using SQL wrong, and the code belongs in the business layer or on the client.
In contrast, SQL excels at set operations, and that's what it should primarily be used for. I've seen an awful lot of applications slowed down because business logic or display code was grabbing a million rows of resultset from the database, bringing them back one at a time, and then throwing 990,000 of them away by doing what's effectively a set operation (JOIN, whatever) outside the database, instead of selecting the 10,000 interesting results using a query on the server and then processing the results of that.
So. It depends on what you mean by "large SQL queries". I feel from the way you're asking the question that what you mean is "overly-complex, non-set-based translations of business/presentation logic into SQL queries that should never have been written in the first place."

in many data-in/data-out cases, no.
in some cases, yes.
If all you need to work with is a simple navigation hierarchy (mainly focusing on parent, sibling, child, etc), then LINQ and it's friends are excellent choices - they reduce the pain (and effort and risk) from the majority of queries. But there are a number of scenarios where it doesn't work so well:
large-scale set-based operations: I can do a wide-ranging query in TSQL without the need to drag that data over the network in one large query, and then (even worse) update each record individually (since in many cases the ORM tools will choose individual UPDATE/INSERT/DELETE operations etc). Not only is this slow, it increases the chances of data drift. So to counter that you might add a transaction - but a long-lived transaction (while you suck a glut of data over the network) is bad
simply: there are a lot of queries where hand-tuning it achieves things that the ORMs simply can't; I had a scenario recently where a relatively basic LINQ query was performing badly. I hand tuned it (using some ROW_NUMBER() etc) and the IO stats went down to only 5% of what they were with the generated query.
there are some queries that are exceptionally difficult to express in some query syntax options, and even if you do - would lead to bad queries. Yet which can be expressed very elegantly in TSQL: example: Linq to Sql: select query with a custom order by

This is a subjective question.
IMO, SQL (or whatever query language you use to access the db) should be as complicated as necessary to solve performance problems.
There are two competing interests:
Performance: This means, load the least amount of data you need in the smallest number of queries.
Maintainability: Load as much as possible (lets say, as it makes sense) with the simplest, most reusable kind of query and do everything else in memory.
So you always need to find your way between performance and maintainability. This is actually nothing special - that's what you do when programming all the time.
Newer ways of doing db queries don't change a lot in this situation. Even if you use NHibernate's HQL, you consider performance and maintainability. You already went a step to maintainability, but you may fall back to SQL to tune some queries.

For me, the deciding factor between writing a giant sql query or a bunch of simple queries and then do everything in the code is usually performance. The latter is preferred but if it goes way too slow, I'll do the former (Sql is optimized for data processing after all).
The reason because I prefer the latter is, that in general my team is more comfortable with code then sql queries. I like sql a lot but if a giant sql query means that I'm only one who can debug/understand it in a reasonable amount of time, that's not a good thing. Another reason is also that with a giant query, you will usually program some business logic in it. If I have a business layer, I prefer too have as much of my business logic there as possible.
Off course, you could decide to stuff all your business logic in stored procedures. Your program is then nothing more then a GUI interface to the API of your database. It depends on the requirements of your project and if your team can handle this.
That said, you give Linq as an alternative technology. I have noticed in my team that thanks to my experience with SQL, I'm very comfortable with Linq while my colleagues are not. The problem on a deeper level is procedural vs set based thinking. Linq is comparable to sql. If you are not comfortable with SQL, chances are you won't be with Linq.

How about performance of linq to sql, Entity Framework and NHibernate?

I was learning this ORM because think this is good technology for most projects. But most employers required acquirement of ADO.NET and SQL.
This ORM not will use in high-loaded system (like popular web-sites)? In which types of projects this ORM will be useful? Are highly loaded projects using ORM?

If you want the best possible performance, don't use an ORM. That said, not all parts of an application need the best possible performance and good ORMs (custom built or off the shelf) significantly increase development speed.
I'm not a big fan of the ORMBattle website, but searching for questions including that term on StackOverflow will give you additional information to read about .NET ORM performance:
http://www.google.com/search?q=site:stackoverflow.com+ormbattle
For instance:
Testing custom ORM solution performance overhead - how to?
ORM (esp. NHibernate) performance for complex queries
Good ORMs result in very little overhead (on top of ADO.NET) and the performance will be just fine in the large majority of cases.
A good ORM will allow you to easily "drop to the metal" (i.e. get closer to raw SQL performance) when you need extra performance.

ORMs can certainly be performance killers. I've measured performance of Entity Framework (v1) and LINQ to SQL against ADO.NET (both datasets and datareaders). EF performance was simply unacceptable, especially in web apps where data contexts would be created and discarded very frequently. LINQ to SQL wasn't too bad, but wouldn't qualify for a high-performance application. The difference between the two is that EF is a two-layer model, without generated code to optimze the mapping between layers. LINQ to SQL is a single layer, and doesn't offer nearly as much customization; the trade-off is that LINQ queries map more closely to the relational model, so there is much less overhead.
The concept of an ORM is certainly valuable; it results in much cleaner code at the application level. But there's no such thing as a free lunch. I'm currently writing a custom ORM that maps a single data model onto both SQL Server and Oracle, with the ability to switch between servers with a simple app config setting. However, there's no LINQ IQueryable provider, and all queries are written or generated as dynamic SQL and run as ADO.NET queries. All database interaction is interfaced to the application as method calls for specific operations, and IEnumerables of entity classes returned as results. Performance is within 10% of straight ADO.NET coding, but that level of performance was a requirement from the start of the project.
While the core components of this ORM could be used in any project, the only way to get ORM and high performance is to avoid any on-the-fly mapping, either between layers (as EF does) or translating LINQ into SQL. (It's very painful to write that, because I dearly love LINQ, but the mapping cost is too much. I did make sure that LINQ to Objects works both within the ORM and in application usage with the results.)

All depends on you requirements and your architecture.
ORMs are evils when you have an reporting system and a very good for simple logic. If will implement a Repository Pattern may be will achieved a good performance.
But, as I said all depends on your requirements and architecture.
Have a look at CQRS (Command-Query Responsibility Segregation) here, this is an interesting approach of system design.

Have a look at Foundations of Programming, this is where i started.
I like to use an ORM where I have a Relational Db with a domain model (objects to be persisted). I find it to save time on development and provide cleaner code.
regarding your post about jobs, I noticed this too. However I can only speculate the answer is they still have so many .Net developers who are still learning this (ORM) there for the frameworks are not in their production systems.
I have noticed a number of Consultancy companies which seem to be using ORMS (if you are one of these companies and you do not use an orm, please correct this, I based it on your technical blogs)
IMeta (Offers Commercial support for NHibernate) UK
Engine Room Apps (they offer lessons and write apps using Nhibernate) UK
EMC2 (they did the whocanhelpme on codeplex)
HTH
bones

Should I Use Entity Framework, DataSet or Custom classes?

I am really having a hard time here. I need to design a "Desktop app" that will use WCF as the communications channel. Its a multi-tiered application (DB and application server are the same, the client goes through the internet cloud).
The application is a little complex (in terms of SQL and code logics) then the usual LOB applications, but the concept is the same: Read from DB, update to DB, handle concurrency etc. My problem is that now with Entity Framework out in the open, I cant decide which way to proceed: Should I use Entity Framework, Dataset or Custom Classes.
As I understand by Entity Framework, it will create the object mapping of my DB tables ALONG WITH the CRUD scripts as well. Thats all well and good for simple CRUD, but most of the times the "Select" is complex and it requires a custom SQL. I understand I can use Stored Procedures in EF (I dont like SP btw, i dont know why, I like to code my SQL in the DAL by hand, I feel more secure and comfortable that way).
With DataSet, I will use my custom SQLs and populate on the data set. With Custom classes (objects for DB tables) I will populate my custom SQLs on those custom classes (collections and lists etc). I want to use EF, but i dont feel confident in deploying an application whose SQL I have not written and cant see in the code. Am I missing something here.
Any help in this regard would be greatly appreciated.
Xeshu

I would agree with Marc G. 100% - DataSets suck, especially in a WCF scenario (they add a lot of overhead for handling in-memory data manipulation) - don't use those. They're okay for beginners and two-tier desktop apps on a small scale maybe - but I wouldn't use them in a serious, professional app.
Basically, your question boils down to how do you transform your rows from the database into something you can remote across WCF. This means some form of mapping - either you do it yourself, using DataReaders and then shoving all the data into WCF [DataContract] classes - you can certainly do that, gives you the ultimate control, but it's also tedious, cumbersome, and error-prone.
Or you let some ready-made ORM handle this grunt work for you - take your pick amongst Linq-to-SQL (great, easy-to-use, flexible, but SQL Server only), EF v4 (out by March 2010 - looks very promising, very flexible) or any other ORM, really - whatever suits your needs best.
Other serious competitors in the ORM space might include Subsonic 3.0 and NHibernate (amongst many many others).
So to sum up:
forget about Datasets
either you have 100% control and to the mapping between SQL and your objects yourself
you let some capable ORM handle that (Linq-to-SQL, EF v4, Subsonic, NHibernate et al) - which one really doesn't matter all that much, i.e. it's also a matter of personal preference and coding style

I can't advocate datasets, especially in an SOA environment like WCF - it'll work, but for mostly the wrong reasons. They simply aren't portable, and IMO don't really "work" over service boundaries. Of course, IMO they don't work in most other scenarios too ;-p
So then it comes down to how much plumbing you want to do. Most ORMs will create WCF-serializable types for you; personally I'd use LINQ-to-SQL at the moment; it is both simpler and more complete than EF, although EF 4.0 is meant to be much better than EF in 3.5sp1. You can use custom TSQL (via ExecuteQuery, which still does the mapping back to objects), but I tend to use either SPROC (for complex queries) or LINQ-generated queries (for simple requests).
Writing the types yourself is fine too, and will work with NHibernate etc. So many options.

While EF works with WCF and sounds very promising, you should consider the effort to get on speed with it. Especially when doing some non trivial stuff, the designer in VS2008 can't open the model anymore and you have to code your model in xml.
Also keep in mind that EF works on a very high abstraction level. Because of the law of leaky abstractions its not all that shiny as it supposed to be :)
The other way round that means, you have to deal with very crazy and hard to read sql statements sent to your database when it comes to troubleshooting / performance issues.

Should I start using LINQ To SQL?

Currently I am using NetTiers to generate my data access layer and service layer. I have been using NetTiers for over 2 years and have found it to be very useful. At some point I need to look at LINQ so my questions are...
Has anyone else gone from NetTiers to LINQ To SQL?
Was this switch over a good or bad thing?
Is there anything that I should be aware of?
Would you recommend this switch?
Basically I would welcome any thoughts
.

No
See #1
You should beware of standard abstraction overhead. Also it's very SQL Server based in it's current state.
Are you using SQL Server, then maybe. If you are using LINQ for other things right now like over XML data (great), Object data, Datasets, then yes you should could switch to have a uniform data syntax for all of them. Like lagerdalek mentioned if it ain't broke don't fix it.
From the quick look at .netTiers Application Framework, I'd say if you already have an investment with that solution it seems to give you much more than a simple Data Access Layer and you should stick with it.
From my experience LINQ to SQL is a good solution for small-medium sized projects. It is an ORM which is a great way to enhance productivity. It also should give you another layer of abstraction that will allow you to change out the layer underneath for something else. The designer in Visual Studio (and I belive VS Express also) is very easy and simple to use. It gives you the common drag-drop and property-based editing of the object mappings.
# Jason Jackson - The Designer does let you add properties by hand, however you need to specify the attributes for that property, but you do this once, it might take 3 minutes longer than the initial dragging of the table into the designer, however it is only necessary once per change in the database itself. This is not too different from other ORMs, however you are correct that they could make this much easier, and find only those properties that have changed, or even implement some kind of refactoring tool for such needs.
Resources:
Why use LINQ to SQL?
Scott Guthrie on LINQ to SQL
10 Tips to Improve your LINQ to SQL Application Performance
LINQ To SQL and Visual Studio 2008 Performance Update
Performance Comparisons LINQ to SQL / ADO / C#
LINQ to SQL 5 Minute Overview
Note that Parallel LINQ is being developed to allow for much greater performance on multi-core machines.

I tried to use Linq to SQL on a small project, thinking that I wanted something I could generate quickly. I ran into a lot of problems in the designer. For example, anytime you need to add a column to a table you basically have to remove and re-add the table definition in the designer. If you have set any properties on the table then you have to re-set those properties. For me this really slowed down the development process.
LINQ to SQL itself is nice. I really like the extensibility. If they can improve the designer I might try it again. I think that the framework would benefit from a little more functionality aimed at a disconnected model like web development.
Check out Scott Guthrie's LINQ to SQL series of blog posts for some great examples of how to use it.

NetTiers is very good for generating a heavy and robust DAL, and we use it internally for core libraries and frameworks.
As I see it, LINQ (in all its incarnations, but specifically as I think you're asking to SQL) is fantastic for quick data access, and we generally use it for more agile cases.
Both technologies are quite inflexible to change without regeneration of the code or dbml layer.
That being said, used properly LINQ 2 SQL is quite a robust solution, and you might even start using it for future development due to it's ease of use, but I wouldn't throw away your current DAL for it - if it aint broke ...

My experience tells me that using by using linq you can get things done faster, however the actual actions to the database are slower.
So... if you have a small database, i'll say go for it. If not, i would wait for some improvements before changing

I'm using LINQ to SQL on fairly large project right now (about 150 tables) and it is working out very well for me. The last ORM I used was IBatis and it worked well but took alot of legwork to get your mappings done. LINQ to SQL performs very well for me and so far has proved to be very easy to use out of the box. There are definately some differences you have to overcome in transition, but I would recommend it's use.
Side note, I have never used or read about NetTiers so I won't discount it's effectiveness, but LINQ to SQL in general has proven to be an extremely viable ORM.

Our team used to use NetTiers and found it to be useful. BUT... the more we used it, the more we found headaches and pain points with it. For example, anytime you make a change to the database, you need to re-generate the DAL with CodeSmith which involved:
re-generating thousands of lines of code in 3 separate projects
re-generating hundreds of stored procedures
Maybe there are other ways of doing it, but this is what we had to do. The re-gen of the source code was ok, scary, but ok. The real issue came with the stored procedures. It didn't clean any unused stored procedures so if you removed a table from your schema and re-gened your DAL, the stored procedures for that table did not get removed. Also, this became quite a headache for database change scripts where we had to compare the old database structure to the new one and create a change script to update client installations. This script could run into the tens of thousands of lines of sql code and if there was an issue executing it, which there invariably was, it was quite a pain to resolve it.
Then the light came on, NHibernate as an ORM. It certainly has a ramp-up time to it but it is well worth it. There is a ton of support for it so if there's something you need done, more than likely it's been done before. It is extremely flexible and allows you to control every aspect of it and then some. It is also becoming easier and easier to use. Fluent Nhibernate is up and coming as a great way to get rid of the xml mapping files that are needed and NHibernate Profiler provides an excellent interface to see what's going on behind the scenes to increase efficiency and remove redundancy.
Moving from NetTiers to NHibernate has been painful, but in a good way. It has forced us to move into a better architecture and re-evaluate functional needs. NetTiers provided tons of data access code, get this entity by its id, get this other entity by its foreign key, get a tlist and vlist of this and that, but most of it was unnecessary and unused. NHibernate with a generic repository and custom repositories only where needed reduced tons of unused code and really increased readability and reliability.