We Keep Coding

LINQ join query with relational entities

I am using ef-core 2.1, I have the following simplified entities where one Account maps to zero or more Attribute objects:
public class Account
{
public int Id { get; set; }
public int LongId { get; set; }
public List<Attribute> Attributes { get; set; }
}
public class Attribute
{
public int Id { get; set; }
public string Name { get; set; }
public string Value { get; set; }
public Account Account { get; set; }
}
I have an initial collection of strings that resemble an Attribute.Value for a given fixed Attribute.Name, I want to find a second associated Attribute object from the same parent Account and obtain its Attribute.Value.
I want to left join the ef entities against the initial string collection so I can easily infer:
Whether no corresponding Account exists or an Account exists without the related Attribute objects (both equate to the same use case).
If an Account exists and it contains all the required Attribute objects, I want to obtain the value of the secondary Attribute.
Without LINQ and ef, I run the following SQL query which ignores the parent Account and produces the result set I want:
CREATE TABLE #Temp
(
id nvarchar(20) not null
);
INSERT INTO #Temp (id) VALUES ('cejawq'), ('issokq'), ('cqlpjq'), ('mbgzvi'), ('wqwlff'), ('iedifh');
SELECT t.[Id], attr2.[Value]
FROM #Temp t
LEFT OUTER JOIN [dbo].[Attributes] attr1
ON t.[Id]=attr1.[Value]
AND attr1.[Name]='uid'
LEFT OUTER JOIN [dbo].[Attributes] attr2
ON attr1.[AccountId]=attr2.[AccountId]
AND attr2.[Name]='objType';
I get the following result set:
id|objType
-----------
cejawq|ext
issokq|ext
cqlpjq|int
mbgzvi|int
wqwlff|ext
iedifh|null
I am struggling with mapping this to efficient LINQ such that the SQL generated produces the result set remotely and ships back data that I can project to an equivalent anonymous type. Do I need to care about the parent objects in the LINQ case? I don't have an index on the Attribute.Value column.
The Attributes table contains the following data:
Id|Name |Value |AccountId
1 |uid |cejawq|1
2 |objType|ext |1
3 |uid |issokq|2
4 |objType|ext |2
5 |uid |cqlpjq|3
6 |objType|int |3
7 |uid |mbgzvi|4
8 |objType|int |4
9 |uid |wqwlff|5
10|objType|ext |5

Since the EF Core does not support joins with in memory sequences (yet), you can split the query in two parts - one which takes the data server side ([Attributes to [Attributes join) using in memory collection as filter (SQL IN through LINQ Contains method), and second which performs left join in memory with the result of the db query:
DbContext db = ...;
var uids = new [] { "cejawq", "issokq", "cqlpjq", "mbgzvi", "wqwlff", "iedifh" };
var dbQuery =
from attr1 in db.Set<Attribute>()
where attr1.Name == "uid" && uids.Contains(attr1.Value)
join attr2 in db.Set<Attribute>()
on new { AccountId = attr1.Account.Id, Name = "objType" }
equals new { AccountId = attr2.Account.Id, attr2.Name }
into attr2Group from attr2 in attr2Group.DefaultIfEmpty() // left outer join
select new { uid = attr1.Value, objType = attr2.Value };
var query =
from uid in uids
join dbResult in dbQuery on uid equals dbResult.uid
into dbResultGroup from dbResult in dbResultGroup.DefaultIfEmpty() // left outer join
select new { uid, dbResult?.objType };
var result = query.ToList();
It translates to a single db query like this:
SELECT [attr1].[Value] AS [uid], [attr2].[Value] AS [objType]
FROM [Attributes] AS [attr1]
LEFT JOIN [Attributes] AS [attr2] ON ([attr1].[AccountId] = [attr2].[AccountId]) AND (N'objType' = [attr2].[Name])
WHERE ([attr1].[Name] = N'uid') AND [attr1].[Value] IN (N'cejawq', N'issokq', N'cqlpjq', N'mbgzvi', N'wqwlff', N'iedifh')

From SQL server, extract a list of Users containing the list of all their bought application (mapping table)

Let Users be a database containing typical users data (name, email...) with an ID as primary key.
Let Applications be a database storing a list of applications (name, developer...) with an ID as primary key.
Let UsersApps be the mapping table between the two, using the primary key. UsersApps thus stores rows of ..., {102, user1_Id, appA_Id}, {103, userN_ID, appB_Id}, {104, user1_Id, appC_Id}, ...
And so on.
I want to retrieve a list of users data {name, email, List<Application> boughtApps}
I am struggling to find a LINQ request to do that and I am trying to do it in two steps, get a big table and then build each user's list of applications.
var q1 = from user in _dbContext.Users
join appUti in _dbContext.AppUsers on user.Id equals appUti.UsersId
join app in _dbContext.Applications on appUti.ApplicationId equals app.Id
orderby user.Id
select new UserDataWithApp { Id = user.Id, Name = user.Name, firstName= user.FirstName, Email = user.Email, App = app };
Then parse q1.toList() to build my required results list.
var qRes = q1.ToList();
int i = 0;
int j = 0;
while (i<qRes.Count())
{
listUsersWithApps[j] = qRes[i];
while (qRes[i].Id == listUsersWithApps[j].Id) // llist is ordered !!
{
listUsersWithApps[j].Apps.Add(qRes[i].Apps[0]);
i++;
}
j++;
}
Isn't there a better way ?

You can use navigation properties to allow the following:
var userApps = context.Users.Select(u => new UserWithApp(u.Name, u.Email, u.Applications))
Just add to following to User:
public virtual ICollection<Application> Applications { get; set; }
and to Application:
public virtual ICollection<Users> Users { get; set; }
So you can "navigate" between your entities and write to following query (just adapt your ordering and what user data to be seleted):
var userApps = from user in context.Users
select new UserDataWithApp { ..., BoughtApps = user.Applications }
See here for an example: http://www.entityframeworktutorial.net/code-first/configure-many-to-many-relationship-in-code-first.aspx
or another interesting blog: https://lostechies.com/jimmybogard/2014/03/12/avoid-many-to-many-mappings-in-orms/

Grab a FK's string value from different Table during Select with linq in C#

I have a table Users and a table PastElo. I am trying to make a new variable (s1rank) to store a User value which is referenced in PastElo during a select statement.
i.e. (tables are condensed down)
---User---
id: int (PK)
name: string
elo: int (FK -> id in PastElo)
---PastElo---
id: int (PK)
rank: string
rank is the name of the User's elo, and I am trying to keep track of elo's string name or at least be able to access it, after returning a List of all Users.
var result = db.Users.Select(x => new { x.name, x.email, x.summoner,
x.birthday,x.school, x.bio, x.champ1, x.champ2, x.champ3,
x.elo, x.facebook, x.rank,x.role1, x.role2, x.s1elo, x.s2elo,
x.s3elo, x.s4elo, x.steam, x.stream, x.summonerID,
//my attempt to get rank's value for elo's int
s1rank = x.PastElo.Rank.Select(x.s1elo == x.PastElo.ID) }
).ToList();
After that occurs it is then passed off to an AngularJS service. Is there a simple way to get that User's string/name of their elo value?
So when I say {{users.s1rank}} I want that to hold the string name for a User's elo rank, not the intvalue of elo.

For those looking for an answer, what I did was the following in order to get the FK ints from several other tables which contained their string value/name.
var query = from users in db.Users
join s1 in db.PastEloes on users.s1elo equals s1.ID
join s2 in db.PastEloes on users.s2elo equals s2.ID
join e in db.eloLists on users.elo equals e.ID
join r in db.rankLists on users.rank equals r.ID
select new
{
elo1 = s1.Rank,
elo2 = s2.Rank,
elo = e.rank,
rank = r.rank,
sortElo = users.elo,
sortYear = users.school,
users.name,
users.email,
users.summoner,
users.birthday,
};

Correct use of multimapping in Dapper

I'm trying to use the multimapping feature of Dapper to return a list of ProductItems and associated Customers.
[Table("Product")]
public class ProductItem
{
public decimal ProductID { get; set; }
public string ProductName { get; set; }
public string AccountOpened { get; set; }
public Customer Customer { get; set; }
}
public class Customer
{
public decimal CustomerId { get; set; }
public string CustomerName { get; set; }
}
My Dapper code:
var sql = #"select * from Product p
inner join Customer c on p.CustomerId = c.CustomerId
order by p.ProductName";
var data = con.Query<ProductItem, Customer, ProductItem>(
sql,
(productItem, customer) => {
productItem.Customer = customer;
return productItem;
},
splitOn: "CustomerId,CustomerName"
);
This works fine, but I seem to have to add the complete column list to the "splitOn" parameter to return all the customers' properties. If I don't add "CustomerName", it returns null. Am I misunderstanding the core functionality of the multimapping feature? I don't want to have to add a complete list of column names each time.

I just ran a test that works fine:
var sql = "select cast(1 as decimal) ProductId, 'a' ProductName, 'x' AccountOpened, cast(1 as decimal) CustomerId, 'name' CustomerName";
var item = connection.Query<ProductItem, Customer, ProductItem>(sql,
(p, c) => { p.Customer = c; return p; }, splitOn: "CustomerId").First();
item.Customer.CustomerId.IsEqualTo(1);
The splitOn param needs to be specified as the split point, it defaults to Id. If there are multiple split points, you will need to add them in a comma delimited list.
Say your recordset looks like this:
ProductID | ProductName | AccountOpened | CustomerId | CustomerName
--------------------------------------- -------------------------
Dapper needs to know how to split the columns in this order into 2 objects. A cursory look shows that the Customer starts at the column CustomerId, hence splitOn: CustomerId.
There is a big caveat here, if the column ordering in the underlying table is flipped for some reason:
ProductID | ProductName | AccountOpened | CustomerName | CustomerId
--------------------------------------- -------------------------
splitOn: CustomerId will result in a null customer name.
If you specify CustomerId,CustomerName as split points, dapper assumes you are trying to split up the result set into 3 objects. First starts at the beginning, second starts at CustomerId, third at CustomerName.

Our tables are named similarly to yours, where something like "CustomerID" might be returned twice using a 'select *' operation. Therefore, Dapper is doing its job but just splitting too early (possibly), because the columns would be:
(select * might return):
ProductID,
ProductName,
CustomerID, --first CustomerID
AccountOpened,
CustomerID, --second CustomerID,
CustomerName.
This makes the splitOn: parameter not so useful, especially when you're not sure what order the columns are returned in. Of course you could manually specify columns... but it's 2017 and we just rarely do that anymore for basic object gets.
What we do, and it's worked great for thousands of queries for many many years, is simply use an alias for Id, and never specify splitOn (using Dapper's default 'Id').
select
p.*,
c.CustomerID AS Id,
c.*
...voila! Dapper will only split on Id by default, and that Id occurs before all the Customer columns. Of course it will add an extra column to your return resultset, but that is extremely minimal overhead for the added utility of knowing exactly which columns belong to what object. And you can easily expand this. Need address and country information?
select
p.*,
c.CustomerID AS Id,
c.*,
address.AddressID AS Id,
address.*,
country.CountryID AS Id,
country.*
Best of all, you're clearly showing in a minimal amount of SQL which columns are associated with which object. Dapper does the rest.

Assuming the following structure where '|' is the point of splitting and Ts are the entities to which the mapping should be applied.
TFirst TSecond TThird TFourth
------------------+-------------+-------------------+------------
col_1 col_2 col_3 | col_n col_m | col_A col_B col_C | col_9 col_8
------------------+-------------+-------------------+------------
Following is the Dapper query that you will have to write.
Query<TFirst, TSecond, TThird, TFourth, TResut> (
sql : query,
map: Func<TFirst, TSecond, TThird, TFourth, TResut> func,
parma: optional,
splitOn: "col_3, col_n, col_A, col_9")
So we want for TFirst to map to col_1 col_2 col_3, for TSecond to col_n col_m ...
The splitOn expression translates to:
Start mapping of all columns into TFirst till you find a column named or aliased as 'col_3', and also include 'col_3' into the mapping result.
Then start mapping into TSecond all columns starting from 'col_n' and continue mapping till new separator is found, which in this case is 'col_A', and mark the start of TThird mapping and so on.
The columns of the SQL query and the props of the mapping object are in a 1:1 relation (meaning that they should be named the same). If the column names resulting from the SQL query are different, you can alias them using the 'AS [Some_Alias_Name]' expression.

If you need to map a large entity write each field must be a hard task.
I tried #BlackjacketMack answer, but one of my tables has an Id Column other ones not (I know it's a DB design problem, but ...) then this insert an extra split on dapper, that's why
select
p.*,
c.CustomerID AS Id,
c.*,
address.AddressID AS Id,
address.*,
country.CountryID AS Id,
country.*
Doesn't work for me. Then I ended with a little change to this, just insert an split point with a name that doesn't match with any field on tables, In may case changed as Id by as _SplitPoint_, the final sql script looks like this:
select
p.*,
c.CustomerID AS _SplitPoint_,
c.*,
address.AddressID AS _SplitPoint_,
address.*,
country.CountryID AS _SplitPoint_,
country.*
Then in dapper add just one splitOn as this
cmd =
"SELECT Materials.*, " +
" Product.ItemtId as _SplitPoint_," +
" Product.*, " +
" MeasureUnit.IntIdUM as _SplitPoint_, " +
" MeasureUnit.* " +
"FROM Materials INNER JOIN " +
" Product ON Materials.ItemtId = Product.ItemtId INNER JOIN " +
" MeasureUnit ON Materials.IntIdUM = MeasureUnit.IntIdUM " +
List < Materials> fTecnica3 = (await dpCx.QueryAsync<Materials>(
cmd,
new[] { typeof(Materials), typeof(Product), typeof(MeasureUnit) },
(objects) =>
{
Materials mat = (Materials)objects[0];
mat.Product = (Product)objects[1];
mat.MeasureUnit = (MeasureUnit)objects[2];
return mat;
},
splitOn: "_SplitPoint_"
)).ToList();

There is one more caveat. If CustomerId field is null (typically in queries with left join) Dapper creates ProductItem with Customer = null. In the example above:
var sql = "select cast(1 as decimal) ProductId, 'a' ProductName, 'x' AccountOpened, cast(null as decimal) CustomerId, 'n' CustomerName";
var item = connection.Query<ProductItem, Customer, ProductItem>(sql, (p, c) => { p.Customer = c; return p; }, splitOn: "CustomerId").First();
Debug.Assert(item.Customer == null);
And even one more caveat/trap. If you don't map the field specified in splitOn and that field contains null Dapper creates and fills the related object (Customer in this case). To demonstrate use this class with previous sql:
public class Customer
{
//public decimal CustomerId { get; set; }
public string CustomerName { get; set; }
}
...
Debug.Assert(item.Customer != null);
Debug.Assert(item.Customer.CustomerName == "n");

I do this generically in my repo, works good for my use case. I thought I'd share. Maybe someone will extend this further.
Some drawbacks are:
This assumes your foreign key properties are the name of your child object + "Id", e.g. UnitId.
I have it only mapping 1 child object to the parent.
The code:
public IEnumerable<TParent> GetParentChild<TParent, TChild>()
{
var sql = string.Format(#"select * from {0} p
inner join {1} c on p.{1}Id = c.Id",
typeof(TParent).Name, typeof(TChild).Name);
Debug.WriteLine(sql);
var data = _con.Query<TParent, TChild, TParent>(
sql,
(p, c) =>
{
p.GetType().GetProperty(typeof (TChild).Name).SetValue(p, c);
return p;
},
splitOn: typeof(TChild).Name + "Id");
return data;
}

I would like to note a very important aspect: the property name within the Entity must match the select statement. Another aspect of splitOn is how it looks for Id by default, so you don't have to specify it unless your naming is something like CustomerId, instead of Id. Let's look at these 2 approaches:
Approach 1
Entity Customer : Id Name
Your query should be something like:
SELECT c.Id as nameof{Customer.Id}, c.Foo As nameof{Customer.Name}.
Then your mapping understands the relationship between the Entity and the table.
Approach 2
Entity Customer: CustomerId, FancyName
Select c.Id as nameof{Customer.CustomerId}, c.WeirdAssName As nameof{Customer.FancyName}
and at the end of the mapping, you have to specify that the Id is the CustomerId by using the SplitOn.
I had an issue where I was not getting my values even though the mapping was correct technically because of a mismatch with the SQL statement.

SQL Linq Question

I have the following working TSQL query in ms SQL 2008
SELECT
Date,
COUNT(click) AS clicks,
COUNT(sale) AS sales,
count(lead) as leads
FROM
(
SELECT ClickDate as date ,ID AS click ,CAST(NULL AS int) AS sale , CAST(null as int) as lead
FROM clicks
UNION ALL
SELECT Date,null, ID ,NULL
FROM sales
UNION ALL
SELECT Date,null, NULL ,ID
FROM leads
) t
GROUP BY Date
How would i convert this to LINQ to SQL?
I wrote this LINQ but it doesn't work.
public class mydata
{
public DateTime date { get; set; }
public int? click { get; set; }
public int? sale { get; set; }
public int? lead { get; set; }
}
var clicks = from c in Clicks
select new mydata
{
date = c.ClickDate, click = c.ID, sale = null, lead = null
};
var sales = from s in Sales
select new mydata
{
date = s.Date, click = null, sale = s.ID, lead = null
};
var leads = from l in Leads
select new mydata
{
date = l.Date, click = null, sale = null, lead = l.ID
};
var v = clicks.Concat(sales).Concat(leads);
var res = from x in v
group x by x.date into xg
select new
{
date = xg.Key, clicks = xg.Count(z => z.click != null)
};
}
How do i correct this LINQ query?
Update:
i Modified the LINQ query based on David B recommendation.
i'm still getting the following error:
"All queries combined using a UNION, INTERSECT or EXCEPT operator must have an equal number of expressions in their target lists."

David's correct about the first two issues. For your final problem (3), Count() does not work like it does in SQL. It expects a predicate that returns a bool. Your using it with ints (i.e. z.click, z.sales, etc.)

The problem is that the anonymous types in the projections are not identical... ID is int in one and Nullable<int> in another.
Instead of using an anonymous type in your projections, use this:
public class ConcatTarget
{
public DateTime TheDate {get;set;}
public int? ID {get;set;}
public string sale {get;set;}
public string lead {get;set;}
}
Even though no instances are actually constructed, LinqToSql uses the class's shape to translate the query.
As for Count, perhaps you meant .Count(x => x.Prop != null) ?
Ok, apparently you've hit upon a buggy translation behavior as described here.
What's happening is the sql translator sees the null assignments, and throws them away. This causes an incorrect number of sql columns to be selected between the sets.
Here's a possible workaround:
int? myNull = null;
var clicks =
from c in Clicks
select new mydata
{
date = c.ClickDate,
click = c.ID,
sale = c.ID + myNull,
lead = myNull + c.ID //note - expressions must be unique
};
The basic idea is to create unique expressions the query translator can't throw away. This is harder than it sounds (above is my ninth attempt).
Here's the other two tables:
var sales = from s in Sales
select new mydata
{
date = s.Date,
click = s.ID + myNull,
sale = s.ID,
lead = myNull + s.ID
};
var leads = from l in Leads
select new mydata
{
date = l.Date,
click = l.ID + myNull,
sale = myNull + l.ID,
lead = l.ID
};
If you had more than 2 columns-to-be-nulled, you could resort to subtraction, division, multiplication, etc.