We Keep Coding

Using lambdas LINQ query in C#

I am still in the midst of converting our web app from vb to C# and this complex query, that I just finished converting and have tested and it works properly, needs refactoring.
This section could probably perform a little better if lambdas were used (I've read in numerous places that lambda expressions perform faster)
foreach (var datapersymbol in (from symId in symblist
select (
from q in ctx.HistoricalPriceData.Where(q => q.SymbolId == symId)
join x in ctx.Symbols on q.SymbolId equals x.SymbolId
orderby q.Date descending
select new {q.Date, q.LastPrice, x.Symbol1})).ToList())
As you can see I've added lambdas where I thought appropriate but Im certain they can be in other parts of this expressions as well. I'm still getting up to speed in C# so Im not an expert here.

Assuming you have a navigation property on HistoricalPriceData to Symbols called "Symbol":
foreach(var datapersymbol in ctx
.HistoricalPriceData
.Where(h=>symblist.Contains(h.SymbolId))
.OrderByDescending(h=>h.Date)
.Select(h=>new {h.Date,h.LastPrice,h.Symbol.Symbol1}))
As Rahul pointed out, there is no performance difference between lamda/method syntax and query syntax, however, this will perform slightly better because it drops the .ToList(), which will allow the loop to begin processing as soon as it gets the first record from the database rather than waiting for the entire result (and it doesn't need to create a List). I've also simplified it somewhat by using .Contains instead of a subquery as well, and switched to using navigation properties rather than a explicit join which makes it easier to read/more maintainable.

For the clarity of the code I will rather refactor it like below;
var datapersymbols = from symId in symblist
select (
from q in ctx.HistoricalPriceData.Where(q => q.SymbolId == symId)
join x in ctx.Symbols on q.SymbolId equals x.SymbolId
orderby q.Date descending
select new {q.Date, q.LastPrice, x.Symbol1});
foreach (var datapersymbol in datapersymbols)
{
}
As far as I can see, in this case, there is not much that you can gain from the C# side. You have to check the query plan for the SQL query and then add necessary indexes. The SymbolId field should be an indexed field.

LinqToSQL joining local list to table - confusion

I am developing an application using LinqToSQL. As part of this I create a list of integers, which represent keys I want to filter. Every time in the past that I've done this and tried to join my list and the data table I get the following error:
Local sequence cannot be used in LINQ to SQL implementation of query operators except the Contains() operator
Now this is fine because, as I understand it, it is a limitaiton/feature of LinqToSQl. I've been using the Contains operator for my queries as shown:
List<CargoProduct> cargoProducts = context.CargoProducts
.Where(cp => cargos.Contains(cp.CargoID))
.ToList();
Recently I've come across the 2100 item limitation in Contains, so was looking for other ways to do it, eventually coming up with the following:
List<CargoProduct> cargoProducts = context.CargoProducts.AsEnumerable()
.Join(cargos, cp => cp.CargoID, c => c, (cp, c) => cp)
.ToList();
Now, that works fine so I was putting together a knowledge sharing email for the other developers in case they came across this limitation. I was trying to get the error message so put together another query than I'd expect to fail:
List<CargoProduct> results = (from c in cargos
join cp in context.CargoProducts on c equals cp.CargoID
select cp).ToList();
Much to my surprise, not only did this not throw an error but it returned exactly the same results as the previous query. So, what am I missing here? I'm sure it's something obvious!
For reference context is my LinqToSQl connection and cargos is instantiated as:
List<int> cargos = context.Cargos.Select(c => c.CargoID).ToList();
Update
As mentioned in the reply it would indeed appear to be the order in which I am joining stuff, as if I use the following then I get the expected error message:
List<CargoProduct> test3 = (from cp in context.CargoProducts
join c in cargos on cp.CargoID equals c
select cp).ToList();
It's interesting functionality and I think I understand why it is doing what it does. Could be a good workaround instead of using Contains for smaller datasets.

In this query
List<CargoProduct> results = (from c in cargos
join cp in context.CargoProducts on c equals cp.CargoID
select cp).ToList();
the left operand in the join statement is of type IEnumerable, then the Enumerable.Join extension method is being chosen on method overload resolution. This means that the whole CargoProducts table is being loaded in memory and and filtered via Linq To Objects. It is similar to do context.CargoProducts.AsEnumerable().

Sort Linq list with one column

I guess it should be really simple, but i cannot find how to do it.
I have a linq query, that selects one column, of type int, and i need it sorted.
var values = (from p in context.Products
where p.LockedSince == null
select Convert.ToInt32(p.SearchColumn3)).Distinct();
values = values.OrderBy(x => x);
SearchColumn3 is op type string, but i only contains integers. So i thought, converting to Int32 and ordering would definitely give me a nice 1,2,3 sorted list of values. But instead, the list stays ordered like it were strings.
199 20 201
Update:
I've done some tests with C# code and LinqPad.
LinqPad generates the following SQL:
SELECT [t2].[value]
FROM (
SELECT DISTINCT [t1].[value]
FROM (
SELECT CONVERT(Int,[t0].[SearchColumn3]) AS [value], [t0].[LockedSince], [t0].[SearchColumn3]
FROM [Product] AS [t0]
) AS [t1]
WHERE ([t1].[LockedSince] IS NULL)
) AS [t2]
ORDER BY [t2].[value]
And my SQL profiler says that my C# code generates this piece of SQL:
SELECT DISTINCT a.[SearchColumn3] AS COL1
FROM [Product] a
WHERE a.[LockedSince] IS NULL
ORDER BY a.[SearchColumn3]
So it look like C# Linq code just omits the Convert.ToInt32.
Can anyone say something useful about this?

[Disclaimer - I work at Telerik]
You can solve this problem with Telerik OpenAccess ORM too. Here is what i would suggest in this case.
var values = (from p in context.Products
where p.LockedSince == null
orderby "cast({0} as integer)".SQL<int>(p.SearchColumn3)
select "cast({0} as integer)".SQL<int>(p.SearchColumn3)).ToList().Distinct();
OpenAccess provides the SQL extension method, which gives you the ability to add some specific sql code to the generated sql statement.
We have started working on improving this behavior.
Thank you for pointing this out.
Regards
Ralph

Same answer as one my other questions, it turns out that the Linq provider i'm using, the one that comes with Telerik OpenAccess ORM does things different than the standard Linq to SQL provider! See the SQL i've posted in my opening post! I totally wasn't expecting something like this, but i seem that the Telerik OpenAccess thing still needs a lot of improvement. So be careful before you start using it. It looks nice, but it has some serious shortcomings.

I can't replicate this problem. But just make sure you're enumerating the collection when you inspect it. How are you checking the result?
values = values.OrderBy(x => x);
foreach (var v in values)
{
Console.WriteLine(v.ToString());
}
Remember, this won't change the order of the records in the database or anywhere else - only the order that you can retrieve them from the values enumeration.

Because your values variable is a result of a Linq expression, so that it doest not really have values until you calling a method such as ToList, ToArray, etc.
Get back to your example, the variable x in OrderBy method, will be treated as p.SearchColumn3 and therefore, it's a string.
To avoid that, you need to let p.SearchColumn3 become integer before OrderBy method.
You should add a let statement in to your code as below:
var values = (from p in context.Products
where p.LockedSince == null
let val = Convert.ToInt32(p.SearchColumn3)
select val).Distinct();
values = values.OrderBy(x => x);
In addition, you can combine order by statement with the first, it will be fine.

Working with Cross Context Joins in LINQ-to-SQL

Initially I had written this query using LINQ-to-SQL
var result = from w in PatternDataContext.Windows
join cf in PatternDataContext.ControlFocus on w.WindowId equals cf.WindowId
join p in PatternDataContext.Patterns on cf.CFId equals p.CFId
join r in ResultDataContext.Results on p.PatternId equals r.PatternId
join fi in ResultDataContext.IclFileInfos on r.IclFileId equals fi.IclFileId
join sp in sessionProfileDataContext.ServerProfiles on fi.ServerProfileId equals sp.ProfileId
join u in infrastructure.Users on sp.UserId equals u.Id
where w.Process.Equals(processName)
select u.DistributedAppId;
And when I executed it, and saw result in the QuickWatch.., it showed this message:
the query contains references to items defined on a different data context
On googling, I found this topic at Stackoverflow itself, where I learned simulating cross context joins and as suggested there, I changed my query a bit to this:
var result = from w in PatternDataContext.Windows
join cf in PatternDataContext.ControlFocus on w.WindowId equals cf.WindowId
join p in PatternDataContext.Patterns on cf.CFId equals p.CFId
join r in SimulateJoinResults() on p.PatternId equals r.PatternId
join fi in SimulateJoinIclFileInfos() on r.IclFileId equals fi.IclFileId
join sp in SimulateJoinServerProfiles() on fi.ServerProfileId equals sp.ProfileId
join u in SimulateJoinUsers() on sp.UserId equals u.Id
where w.Process.Equals(processName)
select u.DistributedAppId;
This query is using these SimulateXyz methods:
private static IQueryable<Result> SimulateJoinResults()
{
return from r in SessionDataProvider.Instance.ResultDataContext.Results select r;
}
private static IQueryable<IclFileInfo> SimulateJoinIclFileInfos()
{
return from f in SessionDataProvider.Instance.ResultDataContext.IclFileInfos select f;
}
private static IQueryable<ServerProfile> SimulateJoinServerProfiles()
{
return from sp in sessionProfileDataContext.ServerProfiles select sp;
}
private static IQueryable<User> SimulateJoinUsers()
{
return from u in infrastructureDataContext.Users select u;
}
But even this approach didn't solve the problem. I'm still getting this message in QuickWatch...:
the query contains references to items defined on a different data context
Any solution for this problem? Along with the solution, I would also want to know why the problem still exists, and how exactly the new solution removes it, so that from next time I could solve such problems myself. I'm new to LINQ, by the way.

I've had to do this before, and there are two ways to do it.
The first is to move all the servers into a single context. You do this by pointing LINQ-to-SQL to a single server, then, in that server, create linked servers to all the other servers. Then you just create views for any tables you're interested from the other servers, and add those views to your context.
The second is to manually do the joins yourself, by pulling in data from one context, and using just the properties you need to join into another context. For example,
int[] patternIds = SessionDataProvider.Instance.ResultDataContext.Results.Select(o => o.patternId).ToArray();
var results = from p in PatternDataContext.Patterns
where patternIds.Contains(p.PatternId)
select p;
Though the first is easier to work with, it does have its share of problems. The problem is that you're relying on SQL Server to be performant with linked servers, something it is notoriously bad at. For example, consider this query:
var results = from p in DataContext.Patterns
join r in DataContext.LinkedServerResults on p.PatternId equals r.PatternId
where r.userId = 10;
When you enumerate this query, the following will occur (let's call the normal and linked servers MyServer and MyLinkedServer, respectively)
MyServer asks MyLinkedServer for the Results
MyLinkedServer sends the Results back to MyServer
MyServer takes those Results, joins them on the Patterns table, and returns only the ones with Results.userId = 10
So now the question is: When is the filtering done - on MyServer or MyLinkedServer? In my experience, for such a simple query, it will usually be done on MyLinkedServer. However, once the query gets more complicated, you'll suddenly find that MyServer is requesting the entire Results table from MyLinkedServer and doing the filtering after the join! This wastes bandwidth, and, if the Results tables is large enough, could turn a 50ms query into a 50 second query!
You could fix unperformant cross-server joins using stored procedures, but if you do a lot of complex cross-server joins, you may end up writing stored procedures for most of your queries, which is a lot of work and defeats part of the purpose of using L2SQL in the first place (not having to write a lot of SQL).
In comparison, the following code would always perform the filtering on the server containing the Results table:
int[] patternIds = (from r in SessionDataProvider.Instance.ResultDataContext.Results
where r.userId = 10
select r.PatternId).ToArray();
var results = from p in PatternDataContext.Patterns
where patternIds.Contains(p.PatternId)
select p;
Which is best for your situation is up to your best judgement.
Note that there is a third potential solution which I did not mention, as it is not really a programmer-solution: you could ask your server admins to set up a replication task to copy the necessary data from MyLinkedServer to MyServer once a day/week/month. This is only an option if:
Your program can work with slightly stale data from MyLinkedServer
You only need to read, never write, to MyLinkedServer
The tables you need from MyLinkedServers are not exorbitantly huge
You have the space/bandwidth available
Your database admins are not stingy/lazy

Your SimulateJoins can't work because they return IQueryable. Your current solution is exactly the same as your former one and that is the reason why you get the same exception. If you check the linked question again you will see that their helper methods return IEnumerable which is the only way to make cross context operations. As you probably already know it means that join will be performed in memory on the application server instead of the database server = it will pull all data from your partial queries and execute join as linq-to-objects.
Cross context join on database level is IMO not possible. You can have different connections, different connection strings with different servers, etc. Linq-to-sql does not handle this.

You could work around it by "escaping from" Linq to SQL on the second context, i.e., calling for instance .ToList() on ResultDataContext.Results and ResultDataContext.IclFileInfos so that your query ended up looking like:
var result = from w in PatternDataContext.Windows
join cf in PatternDataContext.ControlFocus on w.WindowId equals cf.WindowId
join p in PatternDataContext.Patterns on cf.CFId equals p.CFId
join r in ResultDataContext.Results.ToList()
on p.PatternId equals r.PatternId
join fi in ResultDataContext.IclFileInfos.ToList()
on r.IclFileId equals fi.IclFileId
join sp in sessionProfileDataContext.ServerProfiles on
fi.ServerProfileId equals sp.ProfileId
join u in infrastructure.Users on sp.UserId equals u.Id
where w.Process.Equals(processName)
select u.DistributedAppId;
Or AsEnumerable() as long as you "get out" of Linq to SQL and into Linq to Objects for the "offending" context.

Old question, but as I happened to have the same problem, my solution was to pass the manually crafted T-SQL cross-server query (with linked servers) directly to the provider through the ExecuteQuery method of the first context:
db.ExecuteQuery(Of cTechSupportCall)(strSql).ToList
This just saves you from having to create a view server side, and Linq to SQL still maps the results to the proper type. This is useful when there is that one query that is just impossible to formulate in Linq.

Joins and subqueries in LINQ

I am trying to do a join with a sub query and can't seem to get it. Here is what is looks like working in sql. How do I get to to work in linq?
SELECT po.*, p.PermissionID
FROM PermissibleObjects po
INNER JOIN PermissibleObjects_Permissions po_p ON (po.PermissibleObjectID = po_p.PermissibleObjectID)
INNER JOIN Permissions p ON (po_p.PermissionID = p.PermissionID)
LEFT OUTER JOIN
(
SELECT u_po.PermissionID, u_po.PermissibleObjectID
FROM Users_PermissibleObjects u_po
WHERE u_po.UserID = '2F160457-7355-4B59-861F-9871A45FD166'
) used ON (p.PermissionID = used.PermissionID AND po.PermissibleObjectID = used.PermissibleObjectID)
WHERE used.PermissionID is null

Without seeing your database and data model, it's pretty impossible to offer any real help. But, probably the best way to go is:
download linqpad - http://www.linqpad.net/
create a connection to your database
start with the innermost piece - the subquery with the "where" clause
get each small query working, then join them up. Linqpad will show you the generated SQL, as well as the results, so build your small queries up until they are right
So, basically, split your problem up into smaller pieces. Linqpad is fantastic as it lets you test these things out, and check your results as you go
hope this helps, good luck
Toby

The LINQ translation for your query is suprisingly simple:
from pop in PermissibleObjectPermissions
where !pop.UserPermissibleObjects.Any (
upo => upo.UserID == new Guid ("2F160457-7355-4B59-861F-9871A45FD166"))
select new { pop.PermissibleObject, pop.PermissionID }
In words: "From all object permissions, retrieve those with at least one user-permission whose UserID is 2F160457-7355-4B59-861F-9871A45FD16".
You'll notice that this query uses association properties for navigating relationships - this avoids the need for "joining" and simplfies the query. As a result, the LINQ query is much closer to its description in English than the original SQL query.
The trick, when writing LINQ queries, is to get out of the habit of "transliterating" SQL into LINQ.