We Keep Coding

How to write a LEFT OUTER JOIN where the right side is null in LINQ to SQL?

How do you write a LEFT OUTER JOIN in LINQ to SQL, where the right side is null?
A graphical representation of the result I want would be this:
Image credits Jeff Atwood.
Take this SQL for example:
select Document.*
from Document left outer join Invoice
on Document.DocumentId = Invoice.DocumentId
where Invoice.DocumentId is null
Basically I want all documents that are not invoices but some other kind of document, doesn't matter what.
I would greatly appreciate examples in both LINQ Query Syntax and LINQ Method (Fluent) Syntax.
Thank you!

Firstly, even in SQL that query should really be a not exists, it is generally more efficient than the left join / is null construct.
The compiler also understands exists better, as it understands that the not exists cannot add more rows to the resultset, so it can keep any uniqueness guarantee that may be there. The compiler does not see that left join with an is null check cannot add rows (perhaps it should, but there is no logic currently built into it to do so).
select Document.*
from Document
where not exists (select 1
from Invoice
where Document.DocumentId = Invoice.DocumentId);
Now it's obvious how to do it in Linq:
var docs =
from doc in Documents
where !Invoices.Any(inv => doc.DocumentId == inv.DocumentId);
var docs =
Documents
.Where(doc => !Invoices.Any(inv => doc.DocumentId == inv.DocumentId));

Linq query with Contains only works with is IQueryable is in external variable

I am using Entity Framework with Linq to Entities, trying to select some data from my database. When I create a Linq query that uses the method IQueryable<int>.Contains, it can only filter the data if I use an external variable! Let me show some example.
This block of code, works perfectly:
var volumes = (from v in work.VolumeAdditiveRepository.All
where v.AdditivesID == AdditivesID
select v.MetricID);
var metrics =
from m in work.MetricRepository.All
where !volumes.Contains(m.ID)
select m;
If you take a good look, you can see I use the variable volumes inside this snippet, in the where condition. If I copy the contents of this variable and paste it inside the metrics variable, leading to the code below, it raises the error: "Unable to create a constant value of type 'CalculadoraRFS.Models.Domain.VolumeAditivo'. Only primitive types ('such as Int32, String, and Guid') are supported in this context.".
var metrics =
from m in work.MetricRepository.All
where !(from v in work.VolumeAdditiveRepository.All
where v.AdditivesID == AdditivesID
select v.MetricID).Contains(m.ID)
select m;
How can I variable substitution cause such error?! Am I doing (certainly) something wrong?
Thank you!
UPDATE:
Actually, I find out that the Repository Pattern or DbContext seems to be the problem, as #jhamm pointed out. The snippet below doesn't work either:
var query = from m in work._context.Metric
where !(from v in work._context.VolumeAdditive
where v.AdditivesID == AdditivesID
select v.MetricID).Contains(m.ID)
select m;
But the snippet below works. I just took the context out of the UnitOfWork class, though it is very simply defined there: public CalculadoraRFSContext _context = new CalculadoraRFSContext();.
var _context = new CalculadoraRFSContext();
var query = from m in _context.Metric
where !(from v in _context.VolumeAdditive
where v.AdditivesID == AdditivesID
select v.MetricID).Contains(m.ID)
select m;
Now I'm really confused about this stuff! Wasn't it supposed to work as expected?!

I used LINQPad to use my EF Database First model on a similar type of query. Both the combined and separate queries gave the same correct results and generated the same SQL. Here is a link on how to use LINQPad with Entity Framework. One difference could be the use of the Repository Pattern, I am not using one. I would recommend testing with the first query to see what SQL is generated. After you run a query, LINQPad has a SQL tab that may help troubleshoot what is going on by looking at the generated SQL.
If you are still having trouble with the combined LINQ statement, a good next step would be to try the Entity Framework object without the Repository objects. If this query works, there may be something wrong with your Repository objects.
// Guessing that Metric and VolumeAdditive are the EF Entities
// LINQPad database dropdown sets the context so they were not set it in these samples
var metrics =
from m in Metric
where !(from v in VolumeAdditive
where v.AdditivesID == AdditivesID
select v.MetricID).Contains(m.ID)
select m;
To find out where the issue lies, I would next use the MetricRepository with the VolumeAdditive EF object.
var metrics =
from m in work.MetricRepository.All
where !(from v in VolumeAdditive
where v.AdditivesID == AdditivesID
select v.MetricID).Contains(m.ID)
select m;
Then I would switch them to use the Metric EF object with the VolumeAdditiveRepository.
var metrics =
from m in Metric
where !(from v in work.VolumeAdditiveRepository.All
where v.AdditivesID == AdditivesID
select v.MetricID).Contains(m.ID)
select m;
Based on the generated SQL and which queries work, I think this should help point you in the right direction. This is based on removing parts of the problem until it works. Then adding them back in until they break to indicate where the issue is. These steps should be done using small incremental changes to minimize the problem space.
Update:
Based on the new information, lets try to divide the new information into new questions that we need to answer.
Maybe the LINQ expression is not able to figure out what to do with the work._context.VolumeAdditive in the where clause. So lets test this theory by using the following. This sets the context to a single variable instead of using work._context.
var _context = work._context;
var query = from m in _context.Metric
where !(from v in _context.VolumeAdditive
where v.AdditivesID == AdditivesID
select v.MetricID).Contains(m.ID)
select m;
Maybe using a let statement to define the MetricID's could resolve this issue.
var metrics =
from m in work.MetricRepository.All
let volumes = from v in work.VolumeAdditiveRepository.All
where v.AdditivesID == AdditivesID
select v.MetricID
where !volumes.Contains(m.ID)
select m;
Based on the results of these tests and mixing and matching the previous 3 tests/questions, we should be getting closer to the answer. When I come up against issues like this, I try to ask my self questions with verifiable answers. Basically, I try to use the Scientific Method to narrow down the issue to find a resolution.

Sort Linq list with one column

I guess it should be really simple, but i cannot find how to do it.
I have a linq query, that selects one column, of type int, and i need it sorted.
var values = (from p in context.Products
where p.LockedSince == null
select Convert.ToInt32(p.SearchColumn3)).Distinct();
values = values.OrderBy(x => x);
SearchColumn3 is op type string, but i only contains integers. So i thought, converting to Int32 and ordering would definitely give me a nice 1,2,3 sorted list of values. But instead, the list stays ordered like it were strings.
199 20 201
Update:
I've done some tests with C# code and LinqPad.
LinqPad generates the following SQL:
SELECT [t2].[value]
FROM (
SELECT DISTINCT [t1].[value]
FROM (
SELECT CONVERT(Int,[t0].[SearchColumn3]) AS [value], [t0].[LockedSince], [t0].[SearchColumn3]
FROM [Product] AS [t0]
) AS [t1]
WHERE ([t1].[LockedSince] IS NULL)
) AS [t2]
ORDER BY [t2].[value]
And my SQL profiler says that my C# code generates this piece of SQL:
SELECT DISTINCT a.[SearchColumn3] AS COL1
FROM [Product] a
WHERE a.[LockedSince] IS NULL
ORDER BY a.[SearchColumn3]
So it look like C# Linq code just omits the Convert.ToInt32.
Can anyone say something useful about this?

[Disclaimer - I work at Telerik]
You can solve this problem with Telerik OpenAccess ORM too. Here is what i would suggest in this case.
var values = (from p in context.Products
where p.LockedSince == null
orderby "cast({0} as integer)".SQL<int>(p.SearchColumn3)
select "cast({0} as integer)".SQL<int>(p.SearchColumn3)).ToList().Distinct();
OpenAccess provides the SQL extension method, which gives you the ability to add some specific sql code to the generated sql statement.
We have started working on improving this behavior.
Thank you for pointing this out.
Regards
Ralph

Same answer as one my other questions, it turns out that the Linq provider i'm using, the one that comes with Telerik OpenAccess ORM does things different than the standard Linq to SQL provider! See the SQL i've posted in my opening post! I totally wasn't expecting something like this, but i seem that the Telerik OpenAccess thing still needs a lot of improvement. So be careful before you start using it. It looks nice, but it has some serious shortcomings.

I can't replicate this problem. But just make sure you're enumerating the collection when you inspect it. How are you checking the result?
values = values.OrderBy(x => x);
foreach (var v in values)
{
Console.WriteLine(v.ToString());
}
Remember, this won't change the order of the records in the database or anywhere else - only the order that you can retrieve them from the values enumeration.

Because your values variable is a result of a Linq expression, so that it doest not really have values until you calling a method such as ToList, ToArray, etc.
Get back to your example, the variable x in OrderBy method, will be treated as p.SearchColumn3 and therefore, it's a string.
To avoid that, you need to let p.SearchColumn3 become integer before OrderBy method.
You should add a let statement in to your code as below:
var values = (from p in context.Products
where p.LockedSince == null
let val = Convert.ToInt32(p.SearchColumn3)
select val).Distinct();
values = values.OrderBy(x => x);
In addition, you can combine order by statement with the first, it will be fine.

Joins and subqueries in LINQ

I am trying to do a join with a sub query and can't seem to get it. Here is what is looks like working in sql. How do I get to to work in linq?
SELECT po.*, p.PermissionID
FROM PermissibleObjects po
INNER JOIN PermissibleObjects_Permissions po_p ON (po.PermissibleObjectID = po_p.PermissibleObjectID)
INNER JOIN Permissions p ON (po_p.PermissionID = p.PermissionID)
LEFT OUTER JOIN
(
SELECT u_po.PermissionID, u_po.PermissibleObjectID
FROM Users_PermissibleObjects u_po
WHERE u_po.UserID = '2F160457-7355-4B59-861F-9871A45FD166'
) used ON (p.PermissionID = used.PermissionID AND po.PermissibleObjectID = used.PermissibleObjectID)
WHERE used.PermissionID is null

Without seeing your database and data model, it's pretty impossible to offer any real help. But, probably the best way to go is:
download linqpad - http://www.linqpad.net/
create a connection to your database
start with the innermost piece - the subquery with the "where" clause
get each small query working, then join them up. Linqpad will show you the generated SQL, as well as the results, so build your small queries up until they are right
So, basically, split your problem up into smaller pieces. Linqpad is fantastic as it lets you test these things out, and check your results as you go
hope this helps, good luck
Toby

The LINQ translation for your query is suprisingly simple:
from pop in PermissibleObjectPermissions
where !pop.UserPermissibleObjects.Any (
upo => upo.UserID == new Guid ("2F160457-7355-4B59-861F-9871A45FD166"))
select new { pop.PermissibleObject, pop.PermissionID }
In words: "From all object permissions, retrieve those with at least one user-permission whose UserID is 2F160457-7355-4B59-861F-9871A45FD16".
You'll notice that this query uses association properties for navigating relationships - this avoids the need for "joining" and simplfies the query. As a result, the LINQ query is much closer to its description in English than the original SQL query.
The trick, when writing LINQ queries, is to get out of the habit of "transliterating" SQL into LINQ.

linq sql - aggregate count

I have some linq that returns the correct data.
var numEmails = (from row in EmailBatchProposal
where row.EmailBatchId == emailBatchId
select row.EmailBatchProposalId).Count();
However, if I understand linq correctly, this does not perform optimally. It grabs all the data and then walks through the list and counts the rows. What I'd really like is for linq (in the background) to use like:
Select count(*) from ...
I trust the performance reasons are obvious.
Does anyone know the proper way to do this?

Actually, if the linq query is used with a collection that implements IQueryable and supports translation into underlying SQL variant, it is quite a basic functionality to translate the Count function from your example correctly.

People generally learn best by practicing. I would suggest you get a copy of LinqPad (free), enter in your Linq query, and see what SQL it generates. Then you can modify the Linq query until you get exactly what you want.

Actually, the LINQ-to-SQL is smart enough to know that it should do a count... For example, if I have the following query:
var v = (from u in TblUsers
select u).Count();
the SQL that actually executes when I run this is:
SELECT COUNT(*) AS [value]
FROM [tblUsers] AS [t0]
Amazing, huh? Another poster made the really good suggestion of getting LinqPad - it is a WONDERFUL tool. It will show you the exact SQL that gets executed. You can always confirm with SQL profiler too.

Check the Log of the SQL used in the query.
using (var dbc = new siteDataContext())
{
dbc.Log = Console.Out;
var bd = (from b in dbc.birthdays
select b).Count();
Console.WriteLine("{0}", bd);
}
gives the query:
SELECT COUNT(*) AS [value]
FROM [dbo].[birthdays] AS [t0]
-- Context: SqlProvider(Sql2008) Model: AttributedMetaModel Build: 3.5.30729.1

You can just use the argument of .Count().
int numEmails = EmailBatchProposal.Count(x => x.EmailBatchId == emailBatchId)
As noted below, this doesn't actually resolve to any different SQL, but I think it's at least a cleaner looking alternative.