I'm trying to replicate the below SQL query in LINQ Lambda by using String.Join. Could someone please point out how I can correct Lambda query.
I have formatted my SQL and placed the error message here.
Here is the error I receive:
System.InvalidOperationException: The LINQ expression 'DbSet<TblObligor>()
.Join(
inner: DbSet<TblObligorGuaranties>(),
outerKeySelector: t => t.ObligorId,
innerKeySelector: t0 => (decimal)t0.ObligorID,
resultSelector: (t, t0) => new TransparentIdentifier<TblObligor, TblObligorGuaranties>(
Outer = t,
Inner = t0
))
.Join(
inner: DbSet<TblObligorGuarantyTypes>(),
outerKeySelector: ti => ObligorService.MapObligorGuaranties(
o: ti.Outer,
og: ti.Inner).GuarantyTypeID,
innerKeySelector: t1 => (Nullable<int>)t1.GuarantyTypeID,
resultSelector: (ti, t1) => new TransparentIdentifier<TransparentIdentifier<TblObligor, TblObligorGuaranties>, TblObligorGuarantyTypes>(
Outer = ti,
Inner = t1
))' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'.
I want a string concatenated colmn of GuarantyTypeDescription.
select
o2.ObligorID,
STUFF(
(
select
',' + cast(
ogt.GuarantyTypeDescription as nvarchar
)
from
tblObligor o
left join tblObligorGuaranties og on o.ObligorId = og.ObligorID
left join tblObligorGuarantyTypes ogt on og.GuarantyTypeID = ogt.GuarantyTypeID
where
1 = 1
and o.ObligorID = o2.ObligorID
and o.assetid = 1996323923 for xml path('')
),
1,
1,
''
) as xmlstring
from
tblObligor o2
where
1 = 1
and o2.assetid = 1996323923
Here is my code:
public async Task<IEnumerable<ObligorGuarantyDTO>> GetObligorsListAsync(int? assetId)
{
var obligorGuarantiesList = _context.TblObligor
.Join(_context.TblObligorGuaranties, o => o.ObligorId, og => og.ObligorID, (o, og) => new { o, og })
.Select(join => MapObligorGuaranties(join.o, join.og))
.Join(_context.TblObligorGuarantyTypes, og => og.GuarantyTypeID, ogt => ogt.GuarantyTypeID, (og, ogt) => new { og, ogt })
.Select(join => MapObligorGuarantyTypes(join.og, join.ogt))
.AsEnumerable();
return obligorGuarantiesList;
}
Here are my maps:
private static ObligorGuarantyDTO MapObligorGuaranties(TblObligor o, TblObligorGuaranties og)
=> new ObligorGuarantyDTO()
{
ObligorID = o.ObligorId,
GuarantyID = og.GuarantyID,
GuarantyTypeID = og.GuarantyTypeID,
Release = og.Release,
ReleaseDate= og.ReleaseDate,
Note= og.Note,
EditBy = og.EditBy,
EditTime= og.EditTime
};
private static ObligorGuarantyDTO MapObligorGuarantyTypes(ObligorGuarantyDTO og, TblObligorGuarantyTypes ogt)
=> new ObligorGuarantyDTO()
{
GuarantyTypeID = ogt.GuarantyTypeID,
GuarantyTypeDescription = String.Join(", ", ogt.GuarantyTypeDescription)
};
I'm a bit out of my element with multi-level joins in LINQ, but I think the following should put you on the right track.
As far as I know, there is nothing you can write in LINQ that will translate to the FOR-XML string concatenation SQL. Although recent versions of SQL Server now have a STRING_AGG() function, I'm not sure is there is a LINQ mapping for that either. (Someone may correct me if I am wrong.)
So the apparent best plan is to write a query to return a collection of ObligorID/GuarantyTypeDescription pairs, group by ObligorID, and then String.Join() the GuarantyTypeDescription values.
The first step is writing LINQ that will translate into the equivalent of the following SQL:
select o.ObligorID, ogt.GuarantyTypeDescription
from
tblObligor o
left join tblObligorGuaranties og on o.ObligorId = og.ObligorID
left join tblObligorGuarantyTypes ogt on og.GuarantyTypeID = ogt.GuarantyTypeID
where 1=1
and o.assetid = 1996323923
Assuming that you are using a provider like Entity Framework that adds collection properties to your classes that represent foreign key relationships, I think the following may do the job:
_context.TblObligor
.Where(o => o.assetid = 1996323923)
.SelectMany(o => o.TblObligorGuaranties, (o, og) => new { o, og })
.SelectMany(o2 => oog.TblObligorGuarantyTypes, (o2, ogt) => new { o2.o, o2.og, ogt })
.Select(o3 = new { o3.o.ObligorID, o3.ogt.GuarantyTypeDescription })
If your classes don't have those collection properties, the following can be used (similar to what you had, but without the mapping classes):
_context.TblObligor
.Where(o => o.assetid = 1996323923)
.Join(_context.TblObligorGuaranties, o => o.ObligorId, og => og.ObligorId,
(o, og) => new { o, og })
.Join(_context.TblObligorGuarantyTypes, o2 => o2.o.ObligorId, ogt => og.ObligorId,
(o2, ogt) => new { o2.o, o2.og, ogt })
.Select(o3 = new { o3.o.ObligorID, o3.ogt.GuarantyTypeDescription })
At this point we need to switch to IEnumerable<> to to the grouping and concatenation in memory:
If you care about order, you may need apply an .OrderBy() before the .Select().
.AsEnumerable()
.GroupBy(x => x.ObligorID)
.Select(g => new { ObligorID = g.Key, Descriptions = String.Join(", ", g })
.ToList();
The above is untested and I can't even guarantee that it is correct at this time, but I think it is close to what you need.
(I invite others who identify errors or have improvements to post a comment or directly update this answer.)
Follow up: After writing this up, I did a search on "LINQ to STRING_AGG" and the following was one of the top results: Converting T-SQL string_agg to LINQ C#. Worth a look.
Related
The situation
We have the following tables in our MySQL database:
Table club (There are more columns but we only care about the Id for now):
Id | ...
---+----
1 | ...
Table genre:
Id | GenreName
---+-----------
1 | Rock
2 | Classic
3 | Techno
And table club2genre:
ClubId | GenreId
-------+--------
1 | 1
1 | 2
1 | 3
The MySQL query
Using the following MySQL query:
SELECT
club.Id,
GROUP_CONCAT(genre.GenreName) as Music
FROM
club
INNER JOIN club2genre
ON club2genre.clubid = club.id
INNER JOIN genre
ON genre.id = club2genre.genreid
GROUP BY
club.Id;
we get (as expected) this result:
Id | Music
---+---------
1 | Rock,Classic,Techno
The (buggy) C# LINQ expression in question
var queryResult = DbContext.Club.Join(
DbContext.Club2Genre,
club => club.Id,
club2genre => club2genre.ClubId,
(club, cg) => new
{
Id = club.Id,
GenreId = cg.GenreId
})
.GroupJoin(
DbContext.Genre,
temp => temp.GenreId,
genre => genre.Id,
(temp, genreList) => new
{
ClubId = temp.Id,
Genres = genreList.Select(genre => genre.Name)
})
.GroupBy(result => result.ClubId);
var result = result.ToList();
In theory this Entity Framework Core LINQ expression should be a 1:1 translation of the MySQL query above where the result column that is aliased as Music (the GROUP_CONCAT one) should be an IEnumerablehowever upon execution the following exception is thrown:
Interestingly enough it only fails upon calling ToList() on the queryResult so I'm not even sure the LINQ expression itself is the problem. And (at least for me) the error message doesn't seem to be especially helpful other than essentially saying "EF Core might be buggy but also maybe not. Who knows".
So my questions are:
1.) Is what I'm trying to do even possible using LINQ and EF Core?
2.) If so, what is wrong my code and why does it crash?
3.) How do I fix it?
Using an extension function, you can generate a LEFT JOIN and create a flattened result that can then be grouped on the client side to emulate the GroupJoin.
I assume that a single query that returns multiple copies of the left hand side is preferable to multiple queries splitting the left and right side, as that is what LINQ to SQL does, but it is also possible to implement a translation that does two queries, one for the left side, and one for the right side where it matches the left side, then join on the client. This eliminates the duplicate left hand side at the expense of two queries. You can also minimize the left duplication and network traffic by putting in Selects that narrows each side down to only the columns needed for the keys and the result.
public static class QueryableExt {
// implement GroupJoin for EF Core 3 by using `LEFT JOIN` and grouping on client side
public static IEnumerable<TRes> GroupJoinEF<TLeft,TRight,TKey,TRes>(
this IQueryable<TLeft> leftq,
IQueryable<TRight> rightq,
Expression<Func<TLeft,TKey>> leftKeyExp,
Expression<Func<TRight,TKey>> rightKeyExp,
Func<TLeft, IEnumerable<TRight>, TRes> resFn) =>
leftq.GroupJoin(rightq, leftKeyExp, rightKeyExp, (left, rightj) => new { left, rightj })
.SelectMany(lrj => lrj.rightj.DefaultIfEmpty(), (lrj, right) => new { lrj.left, right })
.AsEnumerable()
.GroupBy(lr => lr.left, lr => lr.right, (left, rightg) => resFn(left, rightg));
}
With this extension, your query becomes:
var queryResult = DbContext.Club
.GroupJoinEF(DbContext.Club2Genre.Join(DbContext.Genre, c2g => c2g.GenreId, g => g.Id, (c2g, g) => new { c2g.Id, g.Genre }),
c => c.Id,
cg => cg.Id,
(c, cgj) => new {
c.Id,
Music = String.Join(",", cgj.Select(cg => cg?.Genre))
}
);
However, since you aren't using LEFT JOIN in your query, you don't need a full GroupJoin implementation, and could just use Join and again, group on the client side:
var queryResult = DbContext.Club
.Join(DbContext.Club2Genre, c => c.Id, c2g => c2g.Id, (c, c2g) => new { c.Id, c2g.GenreId })
.Join(DbContext.Genre, cgi => cgi.GenreId, g => g.Id, (cgi, g) => new { cgi.Id, g.Genre })
.AsEnumerable()
.GroupBy(cg => cg.Id, (cId, cgg) => new { Id = cId, Music = String.Join(",", cgg.Select(cg => cg.Genre)) });
I have 2 tables:
USERS
UserId
Name
Scores (collection of table Scores)
SCORES
UserId
CategoryId
Points
I need to show all the users and a SUM of their points, but also I need to show the name of the user. It can be filtered by CategoryId or not.
Context.Scores
.Where(p => p.CategoryId == categoryId) * OPTIONAL
.GroupBy(p => p.UserId)
.Select(p => new
{
UserId = p.Key,
Points = p.Sum(s => s.Points),
Name = p.Select(s => s.User.Name).FirstOrDefault()
}).OrderBy(p => p.Points).ToList();
The problem is that when I add the
Name = p.Select(s => s.User.Name).FirstOrDefault()
It takes so long. I don't know how to access the properties that are not inside the GroupBy or are a SUM. This example is very simple becaouse I don't have only the Name, but also other properties from User table.
How can I solve this?
It takes so long because the query is causing client evaluation. See Client evaluation performance issues and how to use Client evaluation logging to identify related issues.
If you are really on EF Core 2.0, there is nothing you can do than upgrading to v2.1 which contains improved LINQ GroupBy translation. Even with it the solution is not straight forward - the query still uses client evaluation. But it could be rewritten by separating the GroupBy part into subquery and joining it to the Users table to get the additional information needed.
Something like this:
var scores = db.Scores.AsQueryable();
// Optional
// scores = scores.Where(p => p.CategoryId == categoryId);
var points = scores
.GroupBy(s => s.UserId)
.Select(g => new
{
UserId = g.Key,
Points = g.Sum(s => s.Points),
});
var result = db.Users
.Join(points, u => u.UserId, p => p.UserId, (u, p) => new
{
u.UserId,
u.Name,
p.Points
})
.OrderBy(p => p.Points)
.ToList();
This still produces a warning
The LINQ expression 'orderby [p].Points asc' could not be translated and will be evaluated locally.
but at least the query is translated and executes as single SQL:
SELECT [t].[UserId], [t].[Points], [u].[UserId] AS [UserId0], [u].[Name]
FROM [Users] AS [u]
INNER JOIN (
SELECT [s].[UserId], SUM([s].[Points]) AS [Points]
FROM [Scores] AS [s]
GROUP BY [s].[UserId]
) AS [t] ON [u].[UserId] = [t].[UserId]
I have a table "Book" with a many-to-many relationship to "Tag" and need a distinct Book-count pr. Tag. In SQL, the query looks like this:
SELECT t.NAME,
count(DISTINCT b.BookId)
FROM _Tag t
JOIN Book.BookTag bt
ON t.Id = bt.TagId
JOIN Books b
ON b.BookId = bt.BookId
GROUP BY t.NAME
ORDER BY count(DISTINCT b.BookId) DESC;
I have fetched the tags and included the Books navigation-property and from this I should be able to get distinct BookId's pr. tagname. I want to get the result in a tuple.
So far I have tried the following:
var tagTuples = from tag in tags
join book in tags.Select(t => t.Books) on tag.Books equals book
group new {tag, book} by tag.Name
into g
select new Tuple<string, string, int>("tags", g.Key, g.Select(x => x.book).Count());
...and...
var tagTuples = tags.GroupBy(t => t.Name)
.Select(t2 => new Tuple<string, string, int>("tags", t2.Key, t2.Sum(t4 => t4.Books
.Select(b => b.BookId).Distinct().Count())))
.Where(t3 => !string.IsNullOrWhiteSpace(t3.Item2)).Take(15);
...and my latest version:
var tagTuples =
tags.Select(t => new {t.Name, BookId = t.Books.Select(b => b.BookId)})
.GroupBy(tb => tb.Name)
.Select(tb2 => new Tuple<string, string, int>("tags", tb2.Key, tb2.Sum(tb3 => tb3.BookId.Distinct().Count())));
Nevermind the small differences in the query's - I'm only interested in a solution to the problem described above.
Frustration! It takes me 2 minutes to write an SQL query that does this and I'm pretty sure there's a simple answer, but I lack EF routine.
Thankyou for your time. :)
using(var ctx = new EntitiesContext())
{
// GROUP By name
var bookCountByTag = ctx.Tags.GroupBy(t => t.Name)
.Select(t2 => new {
// SELECT the key (tag name)
t2.Key,
// "GroupBy" has many result, so use SelectMany
Count = t2.SelectMany(t3 => t3.book)
.Distinct()
.Count()})
.ToList();
}
I have an existing (working!) linq expression:
from ca in db.CustomAnswer
join ss in db.SurveySubmission on ca.SubmissionId equals ss.Id
join cq in db.CustomQuestion on ca.QuestionId equals cq.Id
where (ss.SurveyId == request.SurveyId)
orderby ss.Submitted, cq.SortOrder
select new
{
SubmissionId = ss.Id,
Answer = ca.Answer
}
I want to add the index of the select into the new object, e.g.
from ca in db.CustomAnswer
join ss in db.SurveySubmission on ca.SubmissionId equals ss.Id
join cq in db.CustomQuestion on ca.QuestionId equals cq.Id
where (ss.SurveyId == request.SurveyId)
orderby ss.Submitted, cq.SortOrder
select new
{
SubmissionId = ss.Id,
**Code = selectIndex,**
Answer = ca.Answer
}
To do this, I believe I need to first convert my query to method syntax so I can use the Select((q, index) => ...) form. To my simple mind, I think it should be:
db.SurveySubmission
.Where(ss => ss.SurveyId == request.SurveyId)
.OrderBy(ss => ss.Submitted)
.Join(db.CustomAnswer, ss => ss.Id, ca => ca.SubmissionId, (ss, ca) => new { ss, ca })
.Join(db.CustomQuestion, o => o.ca.QuestionId, cq => cq.Id, (o, cq) => new { o.ss, o.ca, cq })
.OrderBy(q => q.cq.SortOrder)
.Select((q, idx) => new {
SubmissionId = q.ss.Id,
Answer = q.ca.Answer,
Code = idx
});
However, when the expression is evaluated I get an error:
LINQ to Entities does not recognize the method
'System.Linq.IQueryable1[<>f__AnonymousTypef3[System.Guid,System.String,System.Int32]]
Select[<>f_AnonymousTypee3,<>f__AnonymousTypef3]
(System.Linq.IQueryable1[<>f__AnonymousTypee3[My.Data.Namespace.SurveySubmission,
My.Data.Namespace.CustomAnswer,My.Data.Namespace.CustomQuestion]],
System.Linq.Expressions.Expression1[System.Func3[<>f_AnonymousTypee3[My.Data.Namespace.SurveySubmission,
My.Data.Namespace.CustomAnswer,My.Data.Namespace.CustomQuestion],System.Int32,<>f__AnonymousTypef3[System.Guid,
System.String,System.Int32]]])'
method, and this method cannot be translated into a store expression.
I'm hoping this is glaringly obvious to someone? I've stared at it for several hours and the only conclusion I can make is that I'm not clever enough ... can anyone help please??
EF can't translate that into SQL, because in SQL sets are unordered; the idea of an index just doesn't make any sense to it.
Instead do everything but getting the index using an EF query, and then tack on the indexes in a linq to objects query:
var query = //your original query goes here
var finalQuery = query.AsEnumerable()
.Select((answer, index) => new
{
answer.SubmissionId,
answer.Answer,
Code = index,
});
I would like to create a LINQ join statement equivalent of a Left Join
My tables are set up like so:
Recipe
RecipeID
...
Instruction
RecipeID
StepID
SomeFlag
...
Equivalent SQL:
SELECT *
FROM Recipe r
LEFT JOIN Instruction i
ON r.RecipeID = i.RecipeID
AND SomeFlag > 0
This is what I have so far:
var tmp = db.Recipe
.GroupJoin(
db.Instruction,
r => r.RecipeID,
i => i.RecipeID,
(r, i) => new {r, i},
???);
Firstly, is GroupJoin the correct choice for this type of operation? From what I understand, Join is equivalent to the SQL 'Inner Join' and GroupJoin is equivalent to 'Left Join'. Second, what is the correct syntax to obtain my desired result? I have been searching for a while and I can't seem to find a suitable answer using extension methods.
Don't forget to read the help from (GroupJoin: MSDN http://msdn.microsoft.com/en-us/library/bb535047.aspx and Join MSDN http://msdn.microsoft.com/fr-fr/library/bb534675.aspx)
The last argument of the GroupJoin and Join is optional (by overload) and is not usually used.
It is a function that allow you to specify how to compare r.RecipeID with i.RecipeID. As RecipeID must be an integer, using the default comparer is a good choice. So let it with:
var tmp = db.Recipe
.Join(db.Instruction,
r => r.RecipeID,
i => i.RecipeID,
(r, i) => new {r, i});
Now what you want to have is to remove all the instructions that have SomeFlag > 0. Why not do this before joining?
Like this:
var tmp = db.Recipe
.Join(db.Instruction.Where(instruction => instruction.SomeFlag > 0),
r => r.RecipeID,
i => i.RecipeID,
(r, i) => new {r, i});
Update
#usr has perfectly commented saying Join performs an INNER JOIN.
As you may have remarked, LINQ does not have different methods for INNER, OUTER, LEFT, RIGHT joins. To know the equivalent LINQ of a particular SQL join you may find help on MSDN ( http://msdn.microsoft.com/en-us/library/vstudio/bb397676.aspx ).
var tmp = from recipe in Recipes
join instruction in
from instruction in Instructions
where instruction.SomeFlag > 0
select instruction
on recipe.RecipeID equals instruction.RecipeID into gj
from instruction in gj.DefaultIfEmpty()
select new
{
recipe,
instruction
};
using extension methods it is a bit of an ugly solution:
var tmp = Recipes.GroupJoin(Instructions.Where(instruction => instruction.SomeFlag > 0),
recipe => recipe.RecipeID,
instruction => instruction.RecipeID,
(recipe, gj) => new { recipe, gj })
.SelectMany(#t => #t.gj.DefaultIfEmpty(),
(#t, instruction) => new
{
#t.recipe,
instruction
});
Please tell me if I did't understand you, but this extension method returns the same result that you priveded in sql.
public static IEnumerable<ResultType> GetLeftJoinWith(this IEnumerable<Recipe>, IEnumerable<Instructions> ins)
{
var filteredInstructions = ins.Where(x => x.SomeFlag > 0);
var res = from r in rec
join tmpIns in filteredInstructions on r.RecipeID equals t.RecipeID into instructions
from instruction in instructions.DefaultIfEmpty()
select new { r, instruction };
return res;
}
try this
var model = db.Recipe
.GroupJoin(db.Instructions.Where(instruction => instruction.SomeFlag > 0),r => r.RecipeID,i => i.RecipeID, (r, i) => new { Recipe = r, Instructions = i })
.SelectMany(t => t.Instructions.DefaultIfEmpty(),(t, Instructions) => new
{
Recipe = t.Recipe,
Instructions = Instructions
});