I'm currently building a query builder in C#, which first starts by prompting the user to specify which database they would like to query on their local machine.
Thus, the program allows databases to be "plugged-in" to it, rather than have a number of databases with tables that are always used.
Thus, for my query builder to generate SQL statements, would it make more sense to output and execute SQL statements in the form of strings, or could I use Entity Framework? I have no experience with Entity Framework, however from what I can understand, it makes more sense to utilise EF if you've got a static database whose tables and schema you are aware of - versus potentially any database being specified by the user at run-time.
I've currently been working with SQL statements - i.e. the users' interaction with the query builder, literally executes string-based SQL statements which are generated by the application. Would it be possible or worthwhile to switch to Entity Framework?
From my experience using EF, if you are generating queries dynamically, you're better stick to SQL strings.
The only way you could use EF to query a unknown schema is by generating the Entities through reflection, which would be a hell of lot of work. And I'm not sure it would work. And also, you'd lose all benefits from using EF.
So, if this is the case, no.
Entity Framework does not provide any advantages in this scenario. In fact, it limits severely.
It is possible to write a generic SortHelper, PagerHelper, FilterHelper, etc. that takes an expression tree as IQuerable and applies the sort you desire. This sort of generic programming is great, as it avoids SQL Injection.
However, if you use Entity Framework for your query generator, you would have to use reflection to generate your Entity Data Model. Moreover, you would have to decide how to do open-ended select statements. Further, you would be tied to how Entity Framework represents and evaluates queries, which is still not as robust as it should be for an ORM at version 5.0! For example, there is no good way to represent right joins, and you have to always represent them as left joins if you want decent SQL generated. Another limitation is that if you want to write a projection, you would need to generate an anonymous class. .NET does not have a good way to unload types from memory, and every type you generate in an AppDomain is held in memory until the AppDomain gets unloaded. That is why F# 3.0 uses type erasure for its F# Type Providers API, to avoid generating a billion types for databases like RDF, where there are billions of "types".
Also, Entity Framework does not do any kind of serious analysis to decide if an expression can be transformed, like SAT solving.
I am basing my answer on real life experience, having built the exact application you are describing, and then some. The application allowed business analysts to write queries visually and compose queries together.
That said, I do recommend studying Entity Framework's design vocabulary. I have shifted over to using very similar vocabulary, even though I don't use Entity Framework. For example, Navigational Properties. I don't call them Properties, since that is an object-oriented abstraction for those who use object query languages and doesn't make sense in a visual query language. I call them Paths. But I like the Any() operator to imply left join, as well as Include(). Those little modeling ideas were valuable to me.
Related
I have been using EF for my latest project and have enjoyed the ease of programming with it. Although it is slightly harder to setup and uses its own rules on connections, but once in place, treating your database the same as you classes and their methods adds a great layer of simplicity to it.
However, in my case I encountered occasions with which I am still not able to use EF. My program has a main database to work with. That is fine! But it also communicates with other databases on the same server. E.g. it reads data from an outside table that the user specifies inside the program. Those tables are of course not recognized by EF and they have different structures every time. To communicate with those tables I have to resort to normal Sql codes as before. So I feel a little bit between the two worlds.
I am just wondering if it is possible for foreign data to dynamically become part of the Entity Model or whether this foreign data can be accommodated in the framework?
As I know, it's impossible. Entity Framework is ORM, so it maps your Model (classe) to database. So if you don't know your Model you can't tell EF how to map it. You should use another approach to communicate with dynamic data.
Give LINQ-to-SQL (L2S) a try; I think you'll find, a lot of what you like about EF is the expressiveness of LINQ-to-Entities (L2E), and the LINQ syntax is the same.
I won't lie to you, though: there does always come a time using L2S where I wished my relations were first class properties on a POCO, but you can't have everything in this situation.
I'm having trouble choosing an appropriate data access framework, partly because I'm very picky with my preferences and mostly because I don't have much experience with most of them :-)
I need a framework that will allow me to easily map between the DB tables (SQL Server) and my entities, and that will handle the CRUD operations for me (for the most part).
I want my entities to reside in a separate assembly from my DAL.
I prefer using attributes for the mappings over external file like XML.
It doesn't have to be an ORM, and I want to code my entities myself.
I don't mind writing stored procedures.
The project's database won't be very big. Less than 50 tables.
I'd like some of my entities to correspond to an inner join of two tables - one for static data entered manually during development and the other with data filled during runtime - without using two entities that reference one another (the result of this join will be a single entity).
Entity Framework sounded perfect until I realized it doesn't support Enums (yet - and I can't wait for EF 5.0).
I want these entities to include Enums, and plan on using lookup tables for the enums + code generation for the enum to keep it synchronized with the database.
Linq-to-SQL seems like a good candidate, but I don't know if it copes well with my previous demands.
Using Enterprise Library 5.0 DAAB with it's RowMapper, and extending it's abilities to perform updates and inserts is also an option (but will require more coding on my part).
I plan on implementing the Repository Pattern.
How about NHibernate? Would it do? No experience there either.
I would be happy to hear all suggestions.. the more the merrier! Thanks in advance!
I think nHibernate is the way to go, although some of its main strengths (ORM, stored procedure generation, etc) are things you listed as non-requirements. Anyway, nHibernate will do everything you want it to do. Technically it does use xml mappings, but these can easily be auto-generated using fluent attribute mapping. I like this, as it IS done for you, but you get the customization too just in case you need it. Good luck!
I was reading this SO question but still I am not clear about one specific thing.
If I use NHibernate, why do I need LINQ?
The question in my mind becomes more aggravated when I knew that NHibernate also included LINQ support.
LINQ to NHibernate?
WTF!
LINQ is a query language. It allows you to express queries in a way that is not tied in to your persistence layer.
You may be thinking about the LINQ 2 SQL ORM.
Using LINQ in naming the two is causes unfortunate confusions like yours.
nHibernate, EF, LINQ2XML are all LINQ providers - they all allow you to query a data source using the LINQ syntax.
Well, you don't need Linq, you can always do without it, but you might want it.
Linq provides a way to express operations that behave on sets of data that can be queried and where we can then perform other operations based on the state of that data. It's deliberately written so as to be as agnostic as possible whether that data is in-memory collections, XML, database, etc. Ultimately it's always operating on some sort of in-memory object, with some means of converting between in-memory and the ultimate source, though some bindings go further than others in pushing some of the operations down to a different layer. E.g. calling .Count() can end up looking at a Count property, spinning through a collection and keeping a tally, sending a Count(*) query to a database or maybe something else.
ORMs provide a way to have in-memory objects and database rows reflect each other, with changes to one being reflected by changes to the other.
That fits nicely into the "some means of converting" bit above. Hence Linq2SQL, EF and Linq2NHibernate all fulfil both the ORM role and the Linq provider role.
Considering that Linq can work on collections you'd have to be pretty perverse to create an ORM that couldn't support Linq at all (you'd have to design your collections to not implement IEnumerable<T> and hence not work with foreach). More directly supporting it though means you can offer better support. At the very least it should make for more efficient queries. For example if an ORM gave us a means to get a Users object that reflected all rows in a users table, then we would always be able to do:
int uID = (from u in Users where u.Username == "Alice" select u.ID).FirstOrDefault();
Without direct support for Linq by making Users implement IQueryable<User>, then this would become:
SELECT * FROM Users
Followed by:
while(dataReader.Read())
yield return ConstructUser(dataReader);
Followed by:
foreach(var user in Users)
if(user.Username == "Alice")
return user.ID;
return 0;
Actually, it'd be just slightly worse than that. With direct support the SQL query produced would be:
SELECT TOP 1 id FROM Users WHERE username = 'Alice'
Then the C# becomes equivalent to
return dataReader.Read() ? dataReader.GetInt32(0) : 0;
It should be pretty clear how the greater built-in Linq support of a Linq provider should lead to better operation.
Linq is an in-language feature of C# and VB.NET and can also be used by any .NET language though not always with that same in-language syntax. As such, ever .NET developer should know it, and every C# and VB.NET developer should particularly know it (or they don't know C# or VB.NET) and that's the group NHibernate is designed to be used by, so they can depend on not needing to explain a whole bunch of operations by just implementing them the Linq way. Not supporting it in a .NET library that represents queryable data should be considered a lack of completeness at best; the whole point of an ORM is to make manipulating a database as close to non-DB related operations in the programming language in use, as possible. In .NET that means Linq supprt.
First of all LINQ alone is not an ORM. It is a DSL to query the objects irrespective of the source it came from.
So it makes perfect sense that you can use LINQ with Nhibernate too
I believe you misunderstood the LINQ to SQL with plain LINQ.
Common sense?
There is a difference between an ORM like NHibernate and compiler integrated way to express queries which is use full in many more scenarios.
Or: Usage of LINQ (not LINQ to SQL etc. - the langauge, which is what you talk about though I am not sure you meant what you said) means you dont have to deal with Nhibernate special query syntax.
Or: Anyone NOT using LINQ - regardless of NHibernate or not - de qualifies without good explanation.
You don't need it, but you might find it useful. Bear in mind that Linq, as others have said, is not the same thing as Linq to SQL. Where I work, we write our own SQL queries to retrieve data, but it's quite common for us to use Linq to manipulate that data in order to serve a specific need. For instance, you might have a data access method that allows you to retrieve all dogs owned by Dave:
new DogOwnerDal().GetForOwner(id);
If you're only interested in Dave's daschunds for one specific need, and performance isn't that much of an issue, you can use Linq to filter the response for all of Dave's dogs down to the specific data that you need:
new DogOwnerDal().GetForOwner(id).Where(d => d.Breed == DogBreeds.Daschund);
If performance was crucial, you might want to write a specific data access method to retrieve dogs by owner and breed, but in many cases the effort expended to create the new data access method doesn't increase efficiency enough to be worth doing.
In your example, you might want to use NHibernate to retrieve a lump of data, and then use Linq to break that data into lots of individual subsets for some form of processing. It might well be cheaper to get the data once and use Linq to split it up, instead of repeatedly interrogating the database for different mixtures of the same data.
Few questions on ORM mappers like nhibernate (for .net/c# environment).
When queries are run against a sqlserver database, does it use parameter sizes internally?
paramaters.Add("#column1", SqlDataType.Int, 4)
Do they all use reflection at runtime? i.e. hand coding is always a tad faster?
does it support temp tables, table variables?
ORM World is powerfull and full featured one, I think today obstacles are peoples theirself, this to say that using ORM's needs to take changes in mind, in the way of thinkng applications, architectures and patterns.
To answer you questions:
Query execution depends on more factors, it's possible to fully customize the engine/environment to take advantages of various features like, deffered execution, future queries (queries executed in a future moment), multiple queries, and last but not least, session management involves in this area:
Deferred execution is based on concepts like lazy-loading and lazy execution, this means that queries are executed against the database just qhen you perform some actions, like accessing to methods of the NHibernate Session like ToList(), another example of deferred execution is with using of LinqToNhibernate, where just when you access to certain objects, queries are executed
Future queries are as I said beforre queries executed in a future moment, Ayende speaks well about that
Multiple queries are queries that can be "packed" together and executed in one time avoiding multiple roundtrips to the DB, and this could be a very cool feature
Session Management, this is another chapter to mention... but take in mind that if you manage well your session, or better, let NHibernate engine to manage well the session, sometime it's not needed to go to the DN to obtain data
in all the cases, tools like NHibernate generates queries for you, and parametrized queries are managed well with parameters even depending on the underlying DB engine and conseguently DB Dialect you choose!
It's clear that frameworks like NHibernate, most of the time, use reflection at runtime, but it's needed to mention the multiple Reflection optimization are used, see for example Dynamic Proxies... It's clear that somethime, or maybe all the time direct code could be faster, but just in the unit, in the big picture this could involve in more mistakes and bottlenecks
Talking about NHibernate, or better saying, It's usefull to understand what you mean when talk about Temp Tables and temp data.. In terms as are, NHibernate, as I know, doesn't support natively Temp Tables, in the sense of Runtime tables, but this could be done because NHibernate permit to create object mapping at runtime, so a mechanism of Temp data could be implemented using that api
I hope I provided an usefull answer!
...and oops, sorry for my bad english!
nhiberate uses an optimised form of reflection that creates proxy objects on startup that perform better than plain reflection, as it only incurs a one time cost. You have the option of disabling this feature as well which has nhibernate behave in a more typical manner, with the permanent use of reflection.
This feature is set with the following key:
<add key="hibernate.use_reflection_optimizer" value="true" />
Nhibernate can be used with variable table names. See this SO thread for a good solution.
NHibernate and SubSonic, LinqToSql, EF and I think most of the other ones use parameterized sql.
Most ORM's use some sort of reflection, there are some that generate all the code and SQL Query for you at design time so they don't have to use reflection code it might work a bit faster but it makes you domain a real mess and you have to use their application to regenerate all your code.
I am almost sure that they all don't support that but most have a way that you can use SP's and Views to do this.
You can check this out NHibernate Screencast Series http://www.summerofnhibernate.com/
I am trying to leverage ORM given the following requirements:
1) Using .NET Framework (latest Framework is okay)
2) Must be able to use Sybase, Oracle, MSSQL interchangeably
3) The schema is mostly static, BUT there are dynamic parts.
I am somewhat familiar with SubSonic and NHibernate, but not deeply.
I get the nagging feeling that the ORM can do what I want, but I don't know how to leverage it at the moment.
SubSonic probably isn't optimal, since it doesn't currently support Sybase, and writing my own provider for it is beyond my resources and ability right now.
For #3 (above), there are a couple of metadata tables, which describe tables which the vendors can "staple on" to the existing database.
Let's call these MetaTables, and MetaFields.
There is a base static schema, which the ORM (NHibernate ATM) handles nicely.
However, a vendor can add a table to the database (physically) as long as they also add the data to the metadata tables to describe their structure.
What I'd really like is for me to be able to somehow "feed" the ORM with that metadata (in a way that it understands) and have it at that point allow me to manipulate the data.
My primary goal is to reduce the amount of generic SQL statement building I have to do on these dynamic tables.
I'd also like to avoid having to worry about the differences in SQL being sent to Sybase,Oracle, or MSSQL.
My primary problem is that I don't have a way to let ORM know about the dynamic tables until runtime, when I'll have access to the metadata
Edit: An example of the usage might be like the one outlined here:
IDataReader rdr=new Query("DynamicTable1").WHERE("ArbitraryId",2).ExecuteReader();
(However, it doesn't look like SubSonic will work, as there is no Sybase provider (see above)
Acording to this blog you can in fact use NHibernate with dynamic mapping. It takes a bit of tweaking though...
We did some of the using NHibernate, however we stopped the project since it didn't provide us with the ROI we wanted. We ended up writing our own ORM/SQL layer which worked very well (worked since I no longer work there, I'm guessing it still works).
Our system used a open source project to generate the SQL (don't remember the name any more) and we built all our queries in our own Xml based language (Query Markup Language - QML). We could then build an xmlDocument with selects, wheres, groups etc. and then send that to the SqlEngine that would turn it into a Sql statement and execute it. We discusse, but never implemented, a cache in all of this. That would've allowed us to cache the Qmls for frequently used queries.
I am a little confused as to how the orm would be used then at runtime? If the ORM would dynamically build something at runtime, how does the runtime code know what the orm did dynamically?
"have it at that point allow me to manipulate the data" - What is manipulating the data?
I may be missing something here and i aplogize if thats the case. (I only have really used bottom up approach with ORM)
IDataReader doesn't map anything to an object you know. So your example should be written using classic query builder.
Have you looked into using the ADO.NET Entity Framework?
MSDN: LINQ to Entities
It allows you to map database tables to an object model in such a manner that you can code without thinking about which database vendor is being used, and without worrying about minor variations made by a DBA to the actual tables. The mapping is kept in configuration files that can be modified when the db tables are modified without requiring a recompile.
Also, using LINQ to Entities, you can build queries in an OO manner, so you aren't writing actual SQL query strings.