We Keep Coding

SQL vs. Entity Framework for databases at run-time

I'm currently building a query builder in C#, which first starts by prompting the user to specify which database they would like to query on their local machine.
Thus, the program allows databases to be "plugged-in" to it, rather than have a number of databases with tables that are always used.
Thus, for my query builder to generate SQL statements, would it make more sense to output and execute SQL statements in the form of strings, or could I use Entity Framework? I have no experience with Entity Framework, however from what I can understand, it makes more sense to utilise EF if you've got a static database whose tables and schema you are aware of - versus potentially any database being specified by the user at run-time.
I've currently been working with SQL statements - i.e. the users' interaction with the query builder, literally executes string-based SQL statements which are generated by the application. Would it be possible or worthwhile to switch to Entity Framework?

From my experience using EF, if you are generating queries dynamically, you're better stick to SQL strings.
The only way you could use EF to query a unknown schema is by generating the Entities through reflection, which would be a hell of lot of work. And I'm not sure it would work. And also, you'd lose all benefits from using EF.
So, if this is the case, no.

Entity Framework does not provide any advantages in this scenario. In fact, it limits severely.
It is possible to write a generic SortHelper, PagerHelper, FilterHelper, etc. that takes an expression tree as IQuerable and applies the sort you desire. This sort of generic programming is great, as it avoids SQL Injection.
However, if you use Entity Framework for your query generator, you would have to use reflection to generate your Entity Data Model. Moreover, you would have to decide how to do open-ended select statements. Further, you would be tied to how Entity Framework represents and evaluates queries, which is still not as robust as it should be for an ORM at version 5.0! For example, there is no good way to represent right joins, and you have to always represent them as left joins if you want decent SQL generated. Another limitation is that if you want to write a projection, you would need to generate an anonymous class. .NET does not have a good way to unload types from memory, and every type you generate in an AppDomain is held in memory until the AppDomain gets unloaded. That is why F# 3.0 uses type erasure for its F# Type Providers API, to avoid generating a billion types for databases like RDF, where there are billions of "types".
Also, Entity Framework does not do any kind of serious analysis to decide if an expression can be transformed, like SAT solving.
I am basing my answer on real life experience, having built the exact application you are describing, and then some. The application allowed business analysts to write queries visually and compose queries together.
That said, I do recommend studying Entity Framework's design vocabulary. I have shifted over to using very similar vocabulary, even though I don't use Entity Framework. For example, Navigational Properties. I don't call them Properties, since that is an object-oriented abstraction for those who use object query languages and doesn't make sense in a visual query language. I call them Paths. But I like the Any() operator to imply left join, as well as Include(). Those little modeling ideas were valuable to me.

Difference between Native SQL and Entity SQL

I have gone through an article of using EntityConnection, EntityCommands for executing Entity sql queries. But I was unable to understand that Why are we using Entity sql? Why not directly using the Classes and objects for processing CRUD operations on database?
Or If we want to execute sql queries then why we are using Entity Sql , why not directly Ado.net ?
Is there any performance difference or something else?
I have already gone through the page http://msdn.microsoft.com/en-us/library/bb738573.aspx. But I want answer in a more simpler way. Can you please answer me?
Thanks

Why not directly using the Classes and objects for processing CRUD operations on database?
That is (should be) the way for almost all operations. But sometimes there is a need for accessing the db more directly and precisely.
If we want to execute sql queries then why we are using Entity Sql , why not directly Ado.net ?
E-SQL will still let you work with entities (instead of Rows). This is much easier and more powerful, consider inheritance for example.
E-SQL is also supposed to be independent of the actual database, ie the same for Oracle etc. I have no experience with this yet.
Is there any performance difference or something else?
It can be used to improve performance, yes. But not automatically.

The main difference
SQL is database dependent query language working on storage (relational) objects - tables / rows
ESQL is database independent query language working on conceptual (EDMX) objects - entities
ESQL was created prior to LINQ. In some scenarios ESQL offers more functionality than LINQ.

LINQ to SQL - Sql tuning

I am using LINQ to SQL for much of the data access layer and am trying to view and tune the underlying SQL used. Does anyone know a good mechanism to intercept the SQL used by C# LINQ and/or replace it with a more tuned query?

I highly recommend using LINQPad to analyze and optimize your LINQ. You can display generated SQL and check also how long certain query took.
You can also use it with existing project that uses Entity Framework. See more details here.

The DataContext itself has a Log property which is a TextWriter, using this you can write out to whatever you want be it the framework TraceClasses or something Like Log4Net or NLog

Why do I need LINQ if I use NHibernate-like ORMs?

I was reading this SO question but still I am not clear about one specific thing.
If I use NHibernate, why do I need LINQ?
The question in my mind becomes more aggravated when I knew that NHibernate also included LINQ support.
LINQ to NHibernate?
WTF!

LINQ is a query language. It allows you to express queries in a way that is not tied in to your persistence layer.
You may be thinking about the LINQ 2 SQL ORM.
Using LINQ in naming the two is causes unfortunate confusions like yours.
nHibernate, EF, LINQ2XML are all LINQ providers - they all allow you to query a data source using the LINQ syntax.

Well, you don't need Linq, you can always do without it, but you might want it.
Linq provides a way to express operations that behave on sets of data that can be queried and where we can then perform other operations based on the state of that data. It's deliberately written so as to be as agnostic as possible whether that data is in-memory collections, XML, database, etc. Ultimately it's always operating on some sort of in-memory object, with some means of converting between in-memory and the ultimate source, though some bindings go further than others in pushing some of the operations down to a different layer. E.g. calling .Count() can end up looking at a Count property, spinning through a collection and keeping a tally, sending a Count(*) query to a database or maybe something else.
ORMs provide a way to have in-memory objects and database rows reflect each other, with changes to one being reflected by changes to the other.
That fits nicely into the "some means of converting" bit above. Hence Linq2SQL, EF and Linq2NHibernate all fulfil both the ORM role and the Linq provider role.
Considering that Linq can work on collections you'd have to be pretty perverse to create an ORM that couldn't support Linq at all (you'd have to design your collections to not implement IEnumerable<T> and hence not work with foreach). More directly supporting it though means you can offer better support. At the very least it should make for more efficient queries. For example if an ORM gave us a means to get a Users object that reflected all rows in a users table, then we would always be able to do:
int uID = (from u in Users where u.Username == "Alice" select u.ID).FirstOrDefault();
Without direct support for Linq by making Users implement IQueryable<User>, then this would become:
SELECT * FROM Users
Followed by:
while(dataReader.Read())
yield return ConstructUser(dataReader);
Followed by:
foreach(var user in Users)
if(user.Username == "Alice")
return user.ID;
return 0;
Actually, it'd be just slightly worse than that. With direct support the SQL query produced would be:
SELECT TOP 1 id FROM Users WHERE username = 'Alice'
Then the C# becomes equivalent to
return dataReader.Read() ? dataReader.GetInt32(0) : 0;
It should be pretty clear how the greater built-in Linq support of a Linq provider should lead to better operation.
Linq is an in-language feature of C# and VB.NET and can also be used by any .NET language though not always with that same in-language syntax. As such, ever .NET developer should know it, and every C# and VB.NET developer should particularly know it (or they don't know C# or VB.NET) and that's the group NHibernate is designed to be used by, so they can depend on not needing to explain a whole bunch of operations by just implementing them the Linq way. Not supporting it in a .NET library that represents queryable data should be considered a lack of completeness at best; the whole point of an ORM is to make manipulating a database as close to non-DB related operations in the programming language in use, as possible. In .NET that means Linq supprt.

First of all LINQ alone is not an ORM. It is a DSL to query the objects irrespective of the source it came from.
So it makes perfect sense that you can use LINQ with Nhibernate too
I believe you misunderstood the LINQ to SQL with plain LINQ.

Common sense?
There is a difference between an ORM like NHibernate and compiler integrated way to express queries which is use full in many more scenarios.
Or: Usage of LINQ (not LINQ to SQL etc. - the langauge, which is what you talk about though I am not sure you meant what you said) means you dont have to deal with Nhibernate special query syntax.
Or: Anyone NOT using LINQ - regardless of NHibernate or not - de qualifies without good explanation.

You don't need it, but you might find it useful. Bear in mind that Linq, as others have said, is not the same thing as Linq to SQL. Where I work, we write our own SQL queries to retrieve data, but it's quite common for us to use Linq to manipulate that data in order to serve a specific need. For instance, you might have a data access method that allows you to retrieve all dogs owned by Dave:
new DogOwnerDal().GetForOwner(id);
If you're only interested in Dave's daschunds for one specific need, and performance isn't that much of an issue, you can use Linq to filter the response for all of Dave's dogs down to the specific data that you need:
new DogOwnerDal().GetForOwner(id).Where(d => d.Breed == DogBreeds.Daschund);
If performance was crucial, you might want to write a specific data access method to retrieve dogs by owner and breed, but in many cases the effort expended to create the new data access method doesn't increase efficiency enough to be worth doing.
In your example, you might want to use NHibernate to retrieve a lump of data, and then use Linq to break that data into lots of individual subsets for some form of processing. It might well be cheaper to get the data once and use Linq to split it up, instead of repeatedly interrogating the database for different mixtures of the same data.

ADO.NET or Linq to SQL?

I am building a forum, and it has got 4 tables: Users, Threads, Comments, Topics.
I established the connection and the pages.. I started using the ADO.net way to insert data and select data..but then I found that to make more complex manipulations i need to know SQL. So I was looking for another way, and I found that I can open Visual Studio 2010, add Linq to SQL file that produced object relational designer. I read about how to write code, and I saw that I simply need to use a using statement with DataContext object with a simple code to update, add, delete rows in the tables.
I wanted to know, what are the advantages of using one way of querying over another?

ADO.NET gives you low level control over your queries. If query speed is going to be of importance, this is where you want to be. If you speed is not very important, but rapid development and an Object Relational Model is, LINQ to SQL is a safe bet.
I would recommend Linq to SQL over ADO.NET though.
Development is rapid and thinking in an ORM way is natural.
If your queries are too slow, using the .ExecuteQuery method will allow you to pass in a sql statement that you have optimized as if you were doing it in the ADO.NET way. I have had much success with Linq to Sql.
Also I would look at Entity Framework. It gives you more control over your objects and how they are implemented, used and handled than Linq.

LINQ to SQL is part of the ADO.NET family of technologies. It is based on services provided by the ADO.NET provider model. You can therefore mix LINQ to SQL code with existing ADO.NET applications and migrate current ADO.NET solutions to LINQ to SQL. The following illustration provides a high-level view of the relationship.
Refer to the following:
ADO.NET and LINQ to SQL
Advantages & Disadvantages of LINQ
Performance of LINQ to SQL over Normal Stored procedure
LINQ-to-SQL and Stored Procedures

LINQ to SQL is great in that it generates alot of the plumbing code for you. But it is basically the same as using straight up ADO.NET/SQL. To do more complex data manipulation in LINQ to SQL you have to know how write complex joins in LINQ just as you would in SQL.
Look into Entity Framework - it might give you a higher level of abstraction that you are looking for.

The two are on different abstraction levels. ADO.NET is the lowest level of data access in .NET. Anything else will build upon it.
Every abstraction should give you power to express higher-level concepts at the cost of lower level concepts.
If I sound like a philosopher it's because it's Friday.

In addition to Entity Framework, you can take a look at NHibernate (another .net Object Relational Mapper). It's been around longer than EF so it's a bit more mature, but it isn't developed by Microsoft if that matters to you.