Apologized to post a problem for which i could not construct anything solid. it is very shameful for me to post like this kind of question even after having so high reputation for this web site.
Most of the time i write sql in store procedure in sql server and hardly use LINQ. so facing problem to construct a nested LINQ query for a specific scenario. just wonder if anyone could help me or give me hints to construct such query with linq.
here i am providing a sample sql query which i want to construct the same one with LINQ.
SELECT EmployeeName,
(Select count(*) from table1 where condition) as data1,
(Select count(*) from table1 where condition) as data2,
(Select count(*) from table1 where condition) as data3
though i have seen couple of nested linq sample from these urls but still could not figure out how to construct mine.
http://odetocode.com/blogs/scott/archive/2007/09/24/nested-selects-in-linq-to-sql.aspx
https://social.msdn.microsoft.com/Forums/en-US/00340c95-221a-4b16-9c47-d1acbf2415dc/linq-nested-select-issue?forum=linqtosql
http://www.mercurynewmedia.com/blog/blog-detail/mercury-new-media-blog/2014/05/19/linq-join-queries-vs-nested-sub-queries
http://www.codethinked.com/the-linq-selectmany-operator
Nested Select linq query
http://www.java2s.com/Code/CSharp/LINQ/Nestedquerylist.htm
so just wonder if anyone could help.
This is a good example of something to do in Linq. You will find that you can do this easily and even more powerful queries if you understand two things:
Don't try to overly literally translate from SQL into Linq.
Lambdas.
The first is easy to understand but hard to carry out. Some of the Linq keywords are intentionally like SQL keywords. Linq tries to mirror some aspects of SQL without working in exactly the same way. One key difference is that with Linq, each function yields (no pun intended) a Linq object which can be passed to another Linq function. Thus filtering, transforming, aggregating, etc. can be done repeatedly in the order the programmer chooses. In SQL a single query is built up of several elements such as WHERE, FROM, some of which are optional, and the SQL engine decides in which order to evaluate them.
Get LinqPad and use it for playing with queries, and for doing database queries instead of writing SQL code in SSMS. When you have seen and written enough Linq constructs, you will no longer be writing transliterated SQL, and you will be able to switch back and forth between the two.
Secondly, make sure you understand lambdas. What does
x => x + 23
mean? What is its type? What is the type of its argument and return value? What about
(x, y) => x + y * 23
and
() => "Fish"
?
Select in Linq takes as argument a function. What Select does is the simplest possible thing to understand - it takes an iterator of input objects and a function, and returns an iterator of output objects, which are the input objects having had the function applied to them. This is actually easier to understand with a lambda than with a named function.
Remember that everything in C# has a type. A lambda has a type, even if the type is not explicit. Error messages usually tell you exactly what is wrong, except that sometimes they can be misinterpreted and seem to be pointing to exactly the wrong piece of code, and this can be very time-consuming to figure out. Break down the code you are writing into as small pieces as possible, and debug them in Linqpad.
You can get a long way with this. Try looking at a suggestion for a similar query to yours below and see if you can build on it.
/* A very simple dataset */
string s = "THE QUICK BROWN FOX JUMPED OVER THE LAZY DOG";
var x = s.Split();
/* The letters list plays the role of your EmployeeName */
var letters = new List<String>{"A", "E", "I", "O", "U"};
var result = letters
.Select(letter => new {letter = letter,
startswith = x.Count(w => w.StartsWith(letter)),
contains = x.Count(w => w.Contains(letter)),
endswith = x.Count(w => w.EndsWith(letter))});
If you get this far, you might want to learn more about how the Linq functions operate on iterators. Check out Jon Skeet's Edulinq series of blog posts: http://codeblog.jonskeet.uk/2011/02/23/reimplementing-linq-to-objects-part-45-conclusion-and-list-of-posts/ He reimplements Linq to show how it might be designed and how it should work.
Linqpad and Edulinq and two of the best programming resources I know of, in any language, and I recommend them in almost every Linq question I answer. Check them out.
Related
I am new to LINQ queries. I have read/researched about all advantages of LINQ queries over SQL but i have one basic question why do we need to use these queries as i feel their syntax is more complicated than traditional sql queries?
For example look at below example for simple Left Outer Join
var q=(from pd in dataContext.tblProducts
join od in dataContext.tblOrders on pd.ProductID equals od.ProductID into t
from rt in t.DefaultIfEmpty()
orderby pd.ProductID
select new
{
//To handle null values do type casting as int?(NULL int)
//since OrderID is defined NOT NULL in tblOrders
OrderID=(int?)rt.OrderID,
pd.ProductID,
pd.Name,
pd.UnitPrice,
//no need to check for null since it is defined NULL in database
rt.Quantity,
rt.Price,
})
.ToList();
So, the point of LINQ (Language Integrated Queries) is to provide easy ways of working with enumerable collections in executing memory. Contrast to SQL, which is a language for determining what the user gets from a set of data in a database.
Because of the SQL-like syntax, it's easy to confuse LINQ code with SQL, and think that they're 'alike' - they're really not. SQL gets a subset of data from a superset; LINQ is 'syntactic sugar' that hides common operations involving foreach loops.
For instance, this is a common programming pattern:
foreach(Thing thing in things)
{
if(thing.SomeProperty() == "Some Value")
return true;
}
...this is done rather easily in LINQ:
return things.Any(t => t.SomeProperty() == "Some Value");
The two code are functionally the same, and I'm pretty sure even compile to roughly the same IL code. The difference is how it looks to you.
You don't have to use LINQ; you can choose to use a standard foreach, and there are times, such as complex loops, where it is useful to do so. Ultimately it is a question of readability - my counter-question to you is, is the LINQ version of your foreach loop more, or less, readable than the original foreach loop?
If the answer is 'less', then I suggest converting it back to a foreach.
I'm by no means an sql or a linq expert, I use them both.
There is a trend to either make linq into something bad or a silver bullet depending on what side are you.
You need to seriously consider your project requirements in order to choose. The choice is not mutually exclusive. Take what is good from them both .
Advantages
Quick turn around for development
Queries can be dynamically
Tables are automatically created into class
Columns are automatically created into properties
Relationship are automatically appeaded to classes
Lambda expressions are awesome
Data is easy to setup and use
Disadvantages
No clear outline for Tiers
No good way of view permissions
Small data sets will take longer to build the query than execute
There is an overhead for creating queries
When queries are moved from sql to application side, joins are very slow
DBML concurrency issues
Hard to understand advance queries using Expressions
I found that programmers used to Sql will have a hard time figuring out the tricks with LINQ. But programmers with Sql knowledge, but haven't done a ton of work with it, will pick up linq quicker.
The main issue when people start using LINQ is that they keep thinking in the SQL way, they design the SQL query first and then translate it to LINQ. You need to learn how to think in the LINQ way and your LINQ query will become neater and simpler. For instance, in your LINQ you don't need joins. You should use Associations/Navigation Properties instead. Check this post for more details.
I need to do a query on my database that might be something like this where there could realistically be 100 or more search terms.
public IQueryable<Address> GetAddressesWithTown(string[] towns)
{
IQueryable<Address> addressQuery = DbContext.Addresses;
addressQuery.Where( x => towns.Any( y=> x.Town == y ) );
return addressQuery;
}
However when it contains more than about 15 terms it throws and exception on execution because the SQL generated is too long.
Can this kind of query be done through Entity Framework?
What other options are there available to complete a query like this?
Sorry, are we talking about THIS EXACT SQL?
In that case it is a very simple "open your eyes thing".
There is a way (contains) to map that string into an IN Clause, that results in ONE sql condition (town in ('','',''))
Let me see whether I get this right:
addressQuery.Where( x => towns.Any( y=> x.Town == y ) );
should be
addressQuery.Where ( x => towns.Contains (x.Town)
The resulting SQL will be a LOT smaller. 100 items is still taxing it - I would dare saying you may have a db or app design issue here and that requires a business side analysis, I have not me this requirement in 20 years I work with databases.
This looks like a scenario where you'd want to use the PredicateBuilder as this will help you create an Or based predicate and construct your dynamic lambda expression.
This is part of a library called LinqKit by Joseph Albahari who created LinqPad.
public IQueryable<Address> GetAddressesWithTown(string[] towns)
{
var predicate = PredicateBuilder.False<Address>();
foreach (string town in towns)
{
string temp = town;
predicate = predicate.Or (p => p.Town.Equals(temp));
}
return DbContext.Addresses.Where (predicate);
}
You've broadly got two options:
You can replace .Any with a .Contains alternative.
You can use plain SQL with table-valued-parameters.
Using .Contains is easier to implement and will help performance because it translated to an inline sql IN clause; so 100 towns shouldn't be a problem. However, it also means that the exact sql depends on the exact number of towns: you're forcing sql-server to recompile the query for each number of towns. These recompilations can be expensive when the query is complex; and they can evict other query plans from the cache as well.
Using table-valued-parameters is the more general solution, but it's more work to implement, particularly because it means you'll need to write the SQL query yourself and cannot rely on the entity framework. (Using ObjectContext.Translate you can still unpack the query results into strongly-typed objects, despite writing sql). Unfortunately, you cannot use the entity framework yet to pass a lot of data to sql server efficiently. The entity framework doesn't support table-valued-parameters, nor temporary tables (it's a commonly requested feature, however).
A bit of TVP sql would look like this select ... from ... join #townTableArg townArg on townArg.town = address.town or select ... from ... where address.town in (select town from #townTableArg).
You probably can work around the EF restriction, but it's not going to be fast and will probably be tricky. A workaround would be to insert your values into some intermediate table, then join with that - that's still 100 inserts, but those are separate statements. If a future version of EF supports batch CUD statements, this might actually work reasonably.
Almost equivalent to table-valued paramters would be to bulk-insert into a temporary table and join with that in your query. Mostly that just means you're table name will start with '#' rather than '#' :-). The temp table has a little more overhead, but you can put indexes on it and in some cases that means the subsequent query will be much faster (for really huge data-quantities).
Unfortunately, using either temporary tables or bulk insert from C# is a hassle. The simplest solution here is to make a DataTable; this can be passed to either. However, datatables are relatively slow; the over might be relevant once you start adding millions of rows. The fastest (general) solution is to implement a custom IDataReader, almost as fast is an IEnumerable<SqlDataRecord>.
By the way, to use a table-valued-parameter, the shape ("type") of the table parameter needs to be declared on the server; if you use a temporary table you'll need to create it too.
Some pointers to get you started:
http://lennilobel.wordpress.com/2009/07/29/sql-server-2008-table-valued-parameters-and-c-custom-iterators-a-match-made-in-heaven/
SqlBulkCopy from a List<>
I'm from old school where DB had all data access encapsulated into views, procedures, etc. Now I'm forcing myself into using LINQ for most of the obvious queries.
What I'm wondering though, is when to stop and what practical? Today I needed to run query like this:
SELECT D.DeviceKey, D.DeviceId, DR.DriverId, TR.TruckId, LP.Description
FROM dbo.MBLDevice D
LEFT OUTER JOIN dbo.DSPDriver DR ON D.DeviceKey = DR.DeviceKey
LEFT OUTER JOIN dbo.DSPTruck TR ON D.DeviceKey = TR.DeviceKey
LEFT OUTER JOIN
(
SELECT LastPositions.DeviceKey, P.Description, P.Latitude, P.Longitude, P.Speed, P.DeviceTime
FROM dbo.MBLPosition P
INNER JOIN
(
SELECT D.DeviceKey, MAX(P.PositionKey) LastPositionKey
FROM dbo.MBLPosition P
INNER JOIN dbo.MBLDevice D ON P.DeviceKey = D.DeviceKey
GROUP BY D.DeviceKey
) LastPositions ON P.PositionKey = LastPositions.LastPositionKey
) LP ON D.DeviceKey = LP.DeviceKey
WHERE D.IsActive = 1
Personally, I'm not able to write corresponing LINQ. So, I found tool online and got back 2 page long LINQ. It works properly-I can see it in profiler but it's not maintainable IMO. Another problem is that I'm doing projection and getting Anonymous object back. Or, I can manually create class and project into that custom class.
At this point I wonder if it is better to create View on SQL Server and add it to my model? It will break my "all SQL on cliens side" mantra but will be easier to read and maintain. No?
I wonder where you stop with T-SQL vs LINQ ?
EDIT
Model description.
I have DSPTrucks, DSPDrivers and MBLDevices.
Device can be attached to Truck or to Driver or to both.
I also have MBLPositions which is basically pings from device (timestamp and GPS position)
What this query does - in one shot it returns all device-truck-driver information so I know what this device attached to and it also get's me last GPS position for those devices. Response may look like so:
There is some redundant stuff but it's OK. I need to get it in one query.
In general, I would also default to LINQ for most simple queries.
However, when you get at a point where the corresponding LINQ query becomes harder to write and maintain, then what's the point really? So I would simply leave that query in place. It works, after all. To make it easier to use it's pretty straight-forward to map a view or cough stored procedure in your EF model. Nothing wrong with that, really (IMO).
You can firstly store Linq queries in variables which may help to make it not only more readable, but also reusable.
An example maybe like the following:
var redCars = from c in cars
where c.Colour == "red"
select c;
var redSportsCars = from c in redCars
where c.Type == "Sports"
select c;
Queries are lazily executed and not composed until you compile them or iterate over them so you'll notice in profiler that this does produce an effecient query
You will also benifit from defining relationships in the model and using navigation properties, rather than using the linq join syntax. This (again) will make these relationships reusable between queries, and more readable (because you don't specify the relationships in the query like the SQL above)
Generally speaking your LINQ query will be shorter than the equivalent SQL, but I'd suggest trying to work it out by hand rather than using a conversion tool.
With the exception of CTEs (which I'm fairly sure you can't do in LINQ) I would write all queries in LINQ these days
I find when using LINQ its best to ignore whatever sql it generates as long as its retrieving the right thing and is performant, only when one of those doesn't work do I actually look at what its generating.
In terms of the sql it generates being maintainable, you shouldn't really worry about the SQL being maintainable but more the LINQ query that is generating the SQL.
In the end if the sql is not quite right I believe there are various things you can do to make LINQ generate SQL more along the lines you want..to some extent.
AFAIK there isn't any inherent problem with getting anonymous objects back, however if you are doing it it multiple places you may want to create a class to keep things neater.
I'm fairly new to Linq and struggling using dynamic where over a many to many relationship.
Database tables are like so:
Products <-> Products_SubCategories <-> SubCategories
with Products_SubCategories being a link table.
My full linq statement is
db.Products.Where("it.SubCategories.SubCategoryID = 2")
.Include("SubCategories")
.OrderBy(searchOrderBy)
.Skip(currentPage * pageSize)
.Take(pageSize)
.ToList()
.ForEach(p => AddResultItem(items, p));
So ignoring everything bar the Where() I'm just trying to pull out all products which are linked to sub category ID 2, this fails with
To extract properties out of collections, you must use a sub-query to iterate over the collection., near multipart identifier, line 8, column 1.
I think using the SQL-esque syntax I can do a subquery as per this link. However I'm not sure how to do that in the lambda / chaining syntax.
This is the start of a search function and I would like to build up the where string dynamically, as I have with the searchOrderBy string to avoid a large SELECT CASE. Products is linked to another table via a link table that I will need to include once I understand how to do this example.
Any help would be much appreciated!
Thanks
This is wrong:
db.Products.Where("it.SubCategories.SubCategoryID = 2")
SubCategories is a list. It does not have a property called SubCategoryID. Rather, it contains a group of entities which each have a property called SubCategoryID. That's a critical distinction.
When you run into a situation where you don't know how to proceed in there are multiple problems, it is good to break the problem down into several, smaller problems.
Let's start by removing the dynamic query. It will be easier to solve the problem with a non-dynamic query. Once you've done that, you can go back and make it dynamic again.
So start by using the non-dynamic syntax. Type something like this in Visual Studio, and see what IntelliSense does for you:
db.Products.Where(p => p.SubCategories.
You will quickly see that there is no SubCategoryID property. Instead, you will see a bunch of LINQ API methods for working with lists. If you know LINQ well, you will recognize that the Any() method is what you want here:
db.Products.Where(p => p.SubCategories.Any(sc => sc.SubCategoryID == 2))
Go ahead and run that query. Does it work? If so, you can move ahead to making it dynamic. I'm no ESQL expert, but I'd start with something along the lines of:
db.Products.Where("EXISTS(SELECT SC FROM it.SubCategories AS SC WHERE SC.SubCategoryID = 2");
As an aside, I use MS Dynamic Query ("Dynamic LINQ") for this sort of thing rather than Query Builder, as it's more testable.
It worked for me.
db.Products.Where("SubCategories.Any(SubCategoryID = 2)")
I am having a lot of fun with Linq2Sql. Expression Trees have been great, and just the standard Linq2Sql syntax has been a lot of fun.
I am now down to part of my application where I have to somehow store queries in a database, that are custom for different customers that use the same database and same tables (well, view, but you know what I mean). Basically, I cant hard-code anything, and I have to leave the query language clear text so someone can write a new where-clause type query.
So, if that description was harsh, let me clarify:
In a previous version of our application, we used to do direct SQL calls to the db using raw SQL. Yea. it was fun, dirty, and it worked. We would have a database table fulled of different criteria like
(EventType = 6 and Total > 0)
or a subquery style
(EventType = 7
AND Exists (
select *
from events as e1
where events.EventType = e1.EventType
and e1.objectNumber = 89)
)
(sql injection anyone?)
In Linq2Sql, this is a little more challenging. I can make all these queries no problem in the CLR, but being able to pass dynamic where criterias to Linq is a little more challenging, especially if I want to perform a sub query (like the above example).
Some ideas I had:
Get the raw expression, and store it --- but I have no idea how to take the raw text expression and reverse it back to executable to object expression.
Write a SQl like language, and have it parse the code and generate Linq Expression -- wow, that could be a lot of fun
I am quite sure there is no SomeIqueryable.Where("EventType = 6 and Total > 54"). I was reading that it was available in beta1, but I don't see how you can do that now.
var exp2 = context.POSDataEventView.Where("EmployeeNumber == #0", 8310);
This would be the easiest way for me to deploy.. I think.
Store serialized Expressions -- wow.. that would be confusing to a user trying to write a query --- hell, I'm not sure I could even type it all out.
So, I am looking for some ideas on how I can store a query in some kind of clear text, and then execute it against my Linq2Sql objects in some fashion without calling the ExecuteSQL. I want to use the LinqObjects.
P.S. I am using pLinqo for this application if that helps. Its still linq2sql though.
Thanks in advance!
Perhaps the Dynamic LINQ Library (in the MSDN samples) would help?
In particular, usage like:
This should work with any IQueryable<T> source - including LINQ-to-Objects simply by calling .AsQueryable() on the sequence (typically IEnumerable<T>).