I want to know which is the best way to build dynamic queries in LINQ. Queries will be complex and nested. While searching I found a few ways:
Linq dynamic (System.Linq.Dynamic)
Albahari's Predicate builder class
Linq.Expression
There may be more options than these. Which is the best way?
This depends on your circumstances: how fast do you need it, what is your starting point, and so on. In an unconstrained world, I think the best thing is to roll your own library for building dynamic queries. You can use Scott's or Joseph's work as an inspiration, but in the end it all "bottoms out" in the Linq.Expression library.
One advantage to the "do it yourself" approach is that you would not need to bridge from your code to someone's framework. Rather, you would code directly to .NET APIs. This may be useful when you already have a representation of your dynamic queries, for example, in a model that you present to users through a UI, in an XML file, etc. All you need is to walk that representation recursively, and produce System.Linq.Expression as the return.
FWIW, my company took this approach when .NET 3.5 came out, and we are very happy with the outcome.
Linq queries can be written in two ways and let you use any kind of nesting.
Query Syntax
IEnumerable<int> numQuery1 =
from num in numbers
where num % 2 == 0
orderby num
select num;
Method Syntax
IEnumerable<int> numQuery2 = numbers.Where(num => num % 2 == 0).OrderBy(n => n);
For more information about Linq, you can visit Microsoft's LINQ (Language-Integrated Query). It contains everything from getting started to sample tutorials
Related
I am new to LINQ queries. I have read/researched about all advantages of LINQ queries over SQL but i have one basic question why do we need to use these queries as i feel their syntax is more complicated than traditional sql queries?
For example look at below example for simple Left Outer Join
var q=(from pd in dataContext.tblProducts
join od in dataContext.tblOrders on pd.ProductID equals od.ProductID into t
from rt in t.DefaultIfEmpty()
orderby pd.ProductID
select new
{
//To handle null values do type casting as int?(NULL int)
//since OrderID is defined NOT NULL in tblOrders
OrderID=(int?)rt.OrderID,
pd.ProductID,
pd.Name,
pd.UnitPrice,
//no need to check for null since it is defined NULL in database
rt.Quantity,
rt.Price,
})
.ToList();
So, the point of LINQ (Language Integrated Queries) is to provide easy ways of working with enumerable collections in executing memory. Contrast to SQL, which is a language for determining what the user gets from a set of data in a database.
Because of the SQL-like syntax, it's easy to confuse LINQ code with SQL, and think that they're 'alike' - they're really not. SQL gets a subset of data from a superset; LINQ is 'syntactic sugar' that hides common operations involving foreach loops.
For instance, this is a common programming pattern:
foreach(Thing thing in things)
{
if(thing.SomeProperty() == "Some Value")
return true;
}
...this is done rather easily in LINQ:
return things.Any(t => t.SomeProperty() == "Some Value");
The two code are functionally the same, and I'm pretty sure even compile to roughly the same IL code. The difference is how it looks to you.
You don't have to use LINQ; you can choose to use a standard foreach, and there are times, such as complex loops, where it is useful to do so. Ultimately it is a question of readability - my counter-question to you is, is the LINQ version of your foreach loop more, or less, readable than the original foreach loop?
If the answer is 'less', then I suggest converting it back to a foreach.
I'm by no means an sql or a linq expert, I use them both.
There is a trend to either make linq into something bad or a silver bullet depending on what side are you.
You need to seriously consider your project requirements in order to choose. The choice is not mutually exclusive. Take what is good from them both .
Advantages
Quick turn around for development
Queries can be dynamically
Tables are automatically created into class
Columns are automatically created into properties
Relationship are automatically appeaded to classes
Lambda expressions are awesome
Data is easy to setup and use
Disadvantages
No clear outline for Tiers
No good way of view permissions
Small data sets will take longer to build the query than execute
There is an overhead for creating queries
When queries are moved from sql to application side, joins are very slow
DBML concurrency issues
Hard to understand advance queries using Expressions
I found that programmers used to Sql will have a hard time figuring out the tricks with LINQ. But programmers with Sql knowledge, but haven't done a ton of work with it, will pick up linq quicker.
The main issue when people start using LINQ is that they keep thinking in the SQL way, they design the SQL query first and then translate it to LINQ. You need to learn how to think in the LINQ way and your LINQ query will become neater and simpler. For instance, in your LINQ you don't need joins. You should use Associations/Navigation Properties instead. Check this post for more details.
I was able to create a LINQ statement that I thought was strange and wanted to see if anyone else had experience with it.
I've simplified it to this:
var x = db.Test
.Where(a => a.Field1 == Utils.CreateHash(Preferences.getValue(a.Field2)))
.FirstOrDefault();
Now how does this translate to database code? Wouldn't LINQ need to do a double query for every single row, i.e. for row a:
1) Query a.Field2
2) Return value to run Utils.CreateHash(Preferences.getValue(a.Field2))
3) Take that value from step 2 and compare it against a.Field1
4) Repeat 1-3 until I've gone through all the rows or returned a matching row
Wouldn't this be extremely inefficient? Or is LINQ smart enough to run this in a better way? Note, I haven't actually run this code so another possibility is a runtime error. Why wouldn't LINQ be smart enough to detect a conflict then and not let me compile it?
The query as is will not work since have a call to Utils.CreateHash in your lambda that you are trying to execute on the DB - in that context you cannot execute that method since there simply is no equivalent on the DB side hence the query will fail.
In general the ability of 3rd party Linq IQuerable providers (e.g. Linq to SQL, Linq to Entities) to access in memory constructs such as methods or classes is very limited, as a rule of thumb at most accessing primitive values or collections of primitives will work.
Just to add fast...
A good example to know how this works would be to write (extreme case I agree, but best :) or go through the source code for a custom (open source) LINQ provider (e.g. http://relinq.codeplex.com/ has one etc.).
Basically (I'm simplifying things here a bit), a LINQ provider can only 'map' to Db (supported SQL, functions) what he 'knows' about.
i.e. it has a standard set it can work with, other than that, and with your custom methods (that do not translate to constants etc.) in the frame, there is no way to resolve that on the 'Db/SQL side'.
E.g. with your 'custom' linq provider (not the case here) you could add a specific extension call e.g. .MyCalc() - which would be properly resolved and translated into SQL equivalent - and then you'd be able to use it.
Other than that, I think if I recall correct, provider will leave that as an expression, to resolve when it returns from the Db 'fetch', query operation. Or complain about it in certain cases.
Linq is based on IQueryable - and you can take a look at extension methods provided there for SQL equivalents supported.
hope this helps
EDIT: whether things 'work' or not doesn't matter - it still doesn't mean it'd execute on the Db context - i.e. it'd be unacceptable performance wise in most cases. IQueryable works with expressions (and if you look at the interface) - and linq is executed when you invoke or enumerate usually. At that point some of the expressions may evaluate to a const value that can be worked into a SQL, but not in your case.
Best way to test is to test back the SQL generated by query (possibly this one I think Translate LINQ to sql statement).
No.
The LINQ provider will run a single SELECT query that selects both fields, then execute your lambda expression with the two values for each returned row.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
i know how linq works but i am not getting idea where exactly i should use linq?
The question is rather like asking "I know how plus and minus work, but when do I use addition? When do I use subtraction?"
You use addition when you want to know the sum of two things and subtraction when you want to know their difference.
Essentially all LINQ does is add operators to the C# and VB languages for sorting, filtering, projecting, joining and grouping. When do you use LINQ? When you want to sort, filter, project, join or group something.
I say use LINQ.. Absolutely anywhere that it will help accomplish your goal in a concise and maintainable way.
There is no broad use or don't use restrictions, you really have to make a judgement for the piece of code at hand to whether or not it will benefit from LINQ, in general some of the old overly verbose pieces of code (nested looping etc) can be more clearly expressed in LINQ and might be beneficial.
And it should be noted that LINQ is NOT synonymous with LINQ to SQL, LINQ stands for Language Integrated Query and is extraordinarily useful outside a database context.
LINQ is used generally when you have to work with IEnumerable (or IQueryable in case of operations against a database). It makes your code cleaner and more functional.
You could use it for example to replace a for loop:
from i in Enumerable.Range(0, n)
where i % 2 == 0
select i;
instead of:
for (int i = 0; i < n; i++)
{
if (i % 2 == 0)
{
something.Add(i);
}
}
You should use LINQ where you find it appropriate -because it can be both more expressive/concise than conventional common procedural programming idioms.
You can utilize it for a number of scenarios, e.g:
Retrieving data from Entity Framework or using Linq to SQL
Perform operations on collections (sorting, selecting, etc)
Traversing XML documents
These operations all require significantly less code using Linq.
LINQ is a standard. Custom layers you make are not, not unless your team pre-built and agreed upon a custom layer. If your team won't make custom layers, then using LINQ might alleviate the problem of each people having their own way of making requests to the database.
Anytime you want to filter data within your application without having to run a query. Here's a good place to look at examples of how you may want to use it.
http://msdn.microsoft.com/en-us/vcsharp/aa336746.aspx
For me it makes my code more concise and easier to maintain. use it anywhere you have to work with accessing or manipulting Collection of data like Databases, XML, Lists, etc.
But just keep in mind some of it quirks.
At places where it makes code more readable and maintainable.
In places where you work with collections of data and you want to transform, filter, connect or sort it.
You do not have to use LINQ, it is not the cure for all problems. It may actually perform worse than the procedual programmed way in some cases.
Start using it and with the time and experience you get the feeling where it makes sense and is most useful.
thanks for answer.
the answer might take pages why linq? but in short after reading ceratin books,
if i say The main goal of LINQ and LINQ to SQL is to
get rid of, or at least reduce An impedance mismatch between Relational database structure and object oriented . it’s easier to adapt the C# language than to change SQL or XML.
With LINQ, the aim is toward deeply integrating the capabilities of data query and
manipulation languages into programming languages.
LINQ removes many of the barriers among objects, databases, and XML. It
enables us to work with each of these paradigms using the same language-integrated
facilities.
i got answer from book *Linq In Action. *
in this first chapter clears why linq.
this is really good book.
Let's take an example. You want to write a simple query that retrieves customers as follows:
SELECT UPPER(Name)
FROM Customer
WHERE Name LIKE 'A%'
ORDER BY Name
That doesn't look too bad, right? But now suppose these results are feeding a web page, and we want to retrieve just rows 21-30. Suddenly, you need a subquery:
SELECT UPPER(Name) FROM
(
SELECT *, RN = row_number()
OVER (ORDER BY Name)
FROM Customer
WHERE Name LIKE 'A%'
) A
WHERE RN BETWEEN 21 AND 30
ORDER BY Name
Here's same query in LINQ. The gain in simplicity is clear:
var query =
from c in db.Customers
where c.Name.StartsWith ("A")
orderby c.Name
select c.Name.ToUpper();
var thirdPage = query.Skip(20).Take(10);
Only when we enumerate thirdPage will the query actually execute. In the case of LINQ to SQL or Entity Framework, the translation engine will convert the query (that we composed in two steps) into a single SQL statement optimized for the database server to which it's connected.
I have a
class A {
public int X;
public double Y;
public string Z;
// and more fields/properties ...
};
and a List<A> data and can build a linq query like e.g.
var q = from a in data where a.X > 20 select new {a.Y, a.Z};
Then dataGridView1.DataSource = q.ToList(); displays the selection in my DataGridView.
Now the question, is it possible to build the query from a text the user has entered at runtime? Like
var q = QueryFromText("from a in data where a.X > 20 select new {a.Y, a.Z}");
The point being, that the user (having programming skills) can dynamically and freely select the displayed data.
Dynamic Linq baby!
r.e. comment.
Yes, the example as written may not be possible using Dynamic Linq, but if you factor out the constants, e.g. 'from a in data' you are left with a 'where' and a 'select' which can be expressed with dynamic linq.
so two text boxes, maybe three if you include an orderby, could possibly satisfy your requirements.
Just a thought.
Jon has an interesting approach but i would be leery of compiling and executing unrestrained code.
Well, you can use CSharpCodeProvider to compile code at execution time. Have a look at Snippy for an example of this. In this case you'd need to compile the user code in a method which accepts a List<A> called data. My experience is that it works, but it can be slightly fiddly to get right - particularly in terms of adding the appropriate references etc.
Answering it pretty late; though, it will help someone who visits this page.
I had similar requirement and I solved it by dynamically compiling string as LINQ query, executing it over in-memory collection and collecting the result. Only catch is user input needs to be valid C# compile-able code else it returns an exception message instead of result.
Code is pretty long so here is the github link
Sample application on github shows multiple examples including projection.
Although there may be some ways to do this, LINQ simply isn't designed for this scenario. Using CodeDOM (as Jon suggested) is probably the only way to get that easily done. If you trust the user and he/she has programming skills, you could perhaps just use old fashioned methods and let the user enter the query using SQL?
If you, on the other hand, choose to create some visual tool for constructing queries, you don't need to build them by composing strings and you can compose expression trees instead. For example using Linq Kit and AsExpandable.
check out this library
http://msdn.microsoft.com/en-us/vcsharp/bb894665.aspx
I am having a lot of fun with Linq2Sql. Expression Trees have been great, and just the standard Linq2Sql syntax has been a lot of fun.
I am now down to part of my application where I have to somehow store queries in a database, that are custom for different customers that use the same database and same tables (well, view, but you know what I mean). Basically, I cant hard-code anything, and I have to leave the query language clear text so someone can write a new where-clause type query.
So, if that description was harsh, let me clarify:
In a previous version of our application, we used to do direct SQL calls to the db using raw SQL. Yea. it was fun, dirty, and it worked. We would have a database table fulled of different criteria like
(EventType = 6 and Total > 0)
or a subquery style
(EventType = 7
AND Exists (
select *
from events as e1
where events.EventType = e1.EventType
and e1.objectNumber = 89)
)
(sql injection anyone?)
In Linq2Sql, this is a little more challenging. I can make all these queries no problem in the CLR, but being able to pass dynamic where criterias to Linq is a little more challenging, especially if I want to perform a sub query (like the above example).
Some ideas I had:
Get the raw expression, and store it --- but I have no idea how to take the raw text expression and reverse it back to executable to object expression.
Write a SQl like language, and have it parse the code and generate Linq Expression -- wow, that could be a lot of fun
I am quite sure there is no SomeIqueryable.Where("EventType = 6 and Total > 54"). I was reading that it was available in beta1, but I don't see how you can do that now.
var exp2 = context.POSDataEventView.Where("EmployeeNumber == #0", 8310);
This would be the easiest way for me to deploy.. I think.
Store serialized Expressions -- wow.. that would be confusing to a user trying to write a query --- hell, I'm not sure I could even type it all out.
So, I am looking for some ideas on how I can store a query in some kind of clear text, and then execute it against my Linq2Sql objects in some fashion without calling the ExecuteSQL. I want to use the LinqObjects.
P.S. I am using pLinqo for this application if that helps. Its still linq2sql though.
Thanks in advance!
Perhaps the Dynamic LINQ Library (in the MSDN samples) would help?
In particular, usage like:
This should work with any IQueryable<T> source - including LINQ-to-Objects simply by calling .AsQueryable() on the sequence (typically IEnumerable<T>).