We Keep Coding

Ef core not ignoring columns in generated SQL

I am using EF for .NET Core
I have a table which I dont want to bring back all the columns for in a query
(from ae in Table1.Select(x => new {x.CreatedDate, x.EntryId, x.Amount})
join si in Table2.Select(x => new {x.SessionId, x.IdToFind})
.Where(y => y.SessionId == Guid.Parse("52F0C862-15D0-4C7A-975D-285C618342B0")) on ae.EntryId equals si.IdToFind
select new
{
create output
}).ToList();
As you can see, I am specifically only including 2 columns from Table2
SELECT [a].[EntryId], [a].[Amount]
FROM [Table1] AS [a]
INNER JOIN (
SELECT [s].[Id], [s].[CreatedBy], [s].[CreatedDate], [s].[IdToFind], [s].[SessionId], [s].[UpdatedDate], [s].[UpdateddBy]
FROM [Table2] AS [s]
WHERE [s].[SessionId] = '52f0c862-15d0-4c7a-975d-285c618342b0'
) AS [t] ON [a].[EntryId] = [t].[IdToFind]
When I use LinqPad to evaluate the generated SQL, I can see that all the columns from Table 2 have been include
How can I prevent this? I dont want to use the Ignore option on the model loading because in other situations I may want the other columns

You have to use this syntax:
ResultClass result =(from ae in Table1
join si in Table2
on ae.EntryId equals si.IdToFind
where(si.SessionId == Guid.Parse("52F0C862-15D0-4C7A-975D-285C618342B0"))
select new ResultClass
{
ae.CreatedDate,
ae.EntryId,
ae.Amount,
si.SessionId,
si.IdToFind
}).ToList();
But in this case you will need to create a new class that contains properties of both classes or you get an anonymos object that you have to use immediately.
When I like to avoid creating a new class I usually add some [NotMapped] properties to one of Table1 or Table2 classes in a partial class code and use this class instead of the ResultClass

Your approach should work, however mixing the Linq query syntax and Fluent methods isn't making it very readable at all and I suspect that may be responsible for the unexpected query generation. Particularly this:
(from ae in Table1.Select(x => new {x.CreatedDate, x.EntryId, x.Amount})
join si in Table2.Select(x => new {x.SessionId, x.IdToFind}).Where(y => y.SessionId == Guid.Parse("52F0C862-15D0-4C7A-975D-285C618342B0"))
Which has the Where condition on the Table2 Select.
From what I can deduce from your approach, you are being too eager with your optimization:
With EF queries, Select projections do not need to be done at a per-entity level, they are done at the end of the expression and EF will work out exactly what columns from what table need to be included. That said, you can pre-select columns and EF should work it all out, but it will do so from your final projection.
Provided you have an actual FK between Table1.EntryId and Table2.IdToFind, ideally you would want that relationship mapped out in the entities rather than relying on an explicit join. Navigation properties make it much easier to organize queries, but there can be more generic cases where Table2 can serve multiple other tables etc.
Assuming that you want to just build an output from the CreatedDate, EntryId, Amount, and SessionId from these two loosely related tables:
var results = context.Table1
.Join(context.Table2.Where(t2 => t2.SessionId == sessionId),
t1 => t1.EntryId,
t2 => t2.IdToFind,
(t1, t2) => new { Table1 = t1, Table2 = t2 })
.Select(x => new ResultViewModel
{
EntryId = x.Table1.EntryId,
CreatedDate = x.Table1.CreatedDate,
Amount = x.Table1.Amount,
// ...
}).ToList();
Assuming you don't even need anything from Table2 then:
// ...
.Join(context.Table2.Where(t2 => t2.SessionId == sessionId),
t1 => t1.EntryId,
t2 => t2.IdToFind,
(t1, t2) => t1)
.Select(x => new ResultViewModel
{
EntryId = x.EntryId,
CreatedDate = x.CreatedDate,
Amount = x.Amount
}).ToList();
When you read the Linq it may seem like EF would be selecting all columns from the related tables so you'd want to add Select projections there, but they are not needed. The final query should select columns strictly by the final Select projection. I would avoid mixing Linq query language (from Table1 join Table2 select ....) with the Fluent builder methods. IMO the Fluent methods are easier to read from a C# code perspective, but ultimately I'd use one or the other, not both.

How to loop through tables from SQL in c# without hardcoding

I'm creating a desktop application. I have 3 table (part time employees table, full time employees table and department table). The ID on department table is the foreign key.
I have created a connection string in my class. I'm trying to loop through the tables to get all departments that are assigned to employees, if a department is not assigned (delete it). I have the code in sql, but I want it in c#. Below is the sql code.
SELECT DepartmentName, DepartmentAddress
FROM Department
WHERE ID IN
(SELECT DISTINCT(ID)
FROM
PartTimeEmployees
);

If you simply want to DELETE departments that don't have a related employee, you can use an EXISTS:
DELETE D
FROM dbo.Department D
WHERE NOT EXISTS(SELECT 1
FROM dbo.Employee E
WHERE E.DepartmentID = D.ID);
There's no need to go any kind of "looping", as SQL is far better at doing these operations in a set-based way.

If deleting department not in part time employees table or full time employees table then you can get the department list as -
var departmentToDelete = departments.Where(d => !PartTimeEmployees.Any(emp => emp.DepartmentId == d.Id) && !FullTimeEmployees.Any(emp => emp.DepartmentId == d.Id)).ToList();

Select latest records and add results as new columns

I have the following database
my StudentRecords for specific sutdent is
How can i get the latest 3 studentRecords based on the latest Records.RecordDate
so for each sutdent, row will have ReportItems.ReportItemName ReportItems.TotalPSR,, Students.FullName, + add the following columns:
The latest studentRecords based on RecordDate and display the psr + reordsNames + recordDate so that one row look like this
Is this even possible?
And how can i do this using EF6 also what will be the SQL select statment

Providing your EF entities are set up with their mappings correctly to associate the Students, StudentRecords, and Records then you should be able to accomplish this in EF using something like the following:
var results = testContext.Students
.SelectMany(s => s.StudentRecords.OrderByDescending(sr => sr.Record.RecordDate).Take(3))
.GroupBy(x => x.Student)
.ToList();
What this will give you is a result per-student in a grouped structure where the grouping is on the student, with the grouped values are StudentResults. From there you can extract the student details from the "Key", while each result is a collection of (up to) 3 latest StudentRecords in descending order. You will need to handle the scenario where the # of results is < 3.
As a simple example extraction...
foreach (var studentGrouping in results)
{
studentName = studentGrouping.Key.FullName;
psr1 = studentGrouping[0].PSR;
recordName1 = studentGrouping[0].Record.RecordName;
recordDate1 = studentGrouping[0].Record.RecordDate;
// ... etc.
}
Now a caveat to the above solution is that it will invoke lazy-loading on the Records, so up to 3 hits per student. Curiously I tried to mitigate this with .Include(s=>s.StudentRecords.Select(r=>r.Record)) or trying the .Include(r=> r.Record) after the SelectMany() but while the initial query looked to include the columns for the Records, it still triggered the lazy loads.
If anyone can offer a correction to address the lazy loads, I'd love to hear it. I was a bit surprised to see them after adding the Include statements.
If your object model is more complex, such as lots of fields in Student/Record, etc. then I would recommend altering the above to return an anonymous type structure to retrieve just the values you're going to need to populate your results. The above is Ok for simple structures.
For example if Student had 30 columns, but we only care about the ID and Name:
var results = testContext.Students
.SelectMany(s => s.StudentRecords.OrderByDescending(sr => sr.Record.RecordDate).Take(3))
.Select( sr=> new {sr.Student.StudentId, sr.Student.FullName, sr.PSR, sr.Record.RecordId, RecordName = sr.Record.Name})
.GroupBy(x => new{StudentId, FullName})
.ToList();
This would return a structure where the Key was an anonymous type containing just the StudentId and Name, with the grouped values being the type containing the record details for that student. This has the benefit of only returning the data you will need in a single query to the database.

You can use Row_Number and Pivot as below:
;With Cte as (
Select RI.ReportItemName, RI.TotalPSR, S.FullName, SR.PSR, R.[Name], R.[Date],
RowNPSR = Row_Number() over(Partition by SR.StudentRecord order by R.RecordDate DESC),
RowName = Row_Number() over(Partition by SR.StudentRecord order by R.RecordDate DESC) + 500,
RowDate = Row_Number() over(Partition by SR.StudentRecord order by R.RecordDate DESC) + 1000
from
StudentRecords SR
Join Students s
on SR.StudentId = S.StudentId
Join Records R
On SR.RecordId = R.recordId
Left Join ReportItems RI
On S.ReportItemId = RI.ReportItemId
)
Select a.*, p1.[1] as PSR1, p1.[2] as PSR2, p1.[3] as PSR3
, p2.[501] as Name1, p2.[502] as Name2, p2.[503] as Name3 from (
Select * from CTE RowNPSR <= 3 ) a
Pivot (max(PSR) for RowNPSR in ([1],[2],[3]) ) p1
Pivot (max([Name]) for RowName in ([501],[502],[503]) ) p2
Pivot (max([Date]) for RowDate in ([1001],[1002],[1003]) ) p3

Linq to Sql - Manual association with manual parameter

I have the following tables in my database:
SageAccount
ID (bigint)
LegacyID (nvarchar)
Customer (bit)
Consignments
ID (bigint)
Customer (nvarchar)
What I want to do is have a navigation property/association in my Linq to Sql dbml from Consignment to SageAccount. The difficulty with this is that not only do we need to match SageAccount.LegacyID => Consignments.Customer but we also need to only join to sage accounts where SageAccount.Customer is TRUE. So on the Consignments end, it isn't joining onto a field but instead a static value.
Is this possible in Linq to Sql? Note this database doesn't (and unfortunately can't) have any foreign keys setup in the database.

Yes it is possible. linq have join method. You can use it ike this in your situation:
var res = from sageAccount in _context.SageAccount
join consignments in _context.Consignments
on
new
{
LegacyID = sageAccount.LegacyID,
Customer = sageAccount.Customer
}
equals
new
{
LegacyID = consignments.ID,
Customer = true
}
select new { SageAccountID = sageAccount.ID };
Note that Property name, Type and order in the anonymous objects that you're joining on must match.
You can't use OR and AND in joins - use just equals one object to other.
This will have a this kind of result in your SQL:
SELECT [t0].[ID] AS [SageAccountID]
FROM [dbo].[SageAccount] AS [t0]
INNER JOIN [dbo].[Consignments] AS [t1] ON (([t0].[LegacyID]) = [t1].[ID])
AND ([t0].[Customer] = 1)

Linq to Sql - Populate JOIN result into a List

I am not sure if this can be done, but here's the scenario.
I want to turn this sql into linq:
SELECT * FROM Department d
INNER JOIN Employee e ON e.DepartmentID = d.DepartmentID
Department - Employee is 1 to many relationship.
I have created a custom object that I would like to populate the result into.
public class DepartmentSummary
{
public Department Department { get; set; }
public List<Employee> Employees {get; set;}
}
The Linq I came up with is
var result = from d in dba.Department
join e in dba.Employee d.DepartmentID equals e.DepartmentID into j1
select new DepartmentSummary
{
Department = d,
Employees = j1.ToList()
};
I tried it out and it's not working. Can anyone shed some light for me please? I would like to perform an inner join between Department and Employee. For each Department in the resultset, I would like to create one DepartmentSummary object which holds that department and a list of employees belonging to that department.
Does Linq provides an ad hoc solution for this or must I iterates through the result set and create a list of DepartmentSummary manually?
Thanks,
EDIT:
Looks like this works for me
var result = from d in dba.Department
join e in dba.Employee d.DepartmentID equals e.DepartmentID into j1
where j1.Count() > 0
select new DepartmentSummary
{
Department = d,
Employees = j1.ToList()
};

The thing is that you're not really taking one SQL and trying to create a Linq-query out of it.
If you were, you'd notice that your SQL query does not really produce one row per department, but it will repeat the department information for each employee in that department.
Now, an initial naive look would suggest you use a group-by clause, since that would allow you to split the data into individual groupings for each department, but groupings in SQL does not really give you a key+all-matching-rows type of result, rather it allows you to do aggregate calculations, like "for each department, how many employees do I have".
So, in order to do what you want, you need to basically do a normal join, which will give you each employee, coupled with the appropriate department information (ie. each employee will be linked to his/her department), and then you need to construct the rest of the data structure yourself.
Now, having said that, if you have the proper relationships set in your data context related classes, each department should already have some kind of property that contains all employees in that department, so perhaps the simple query is just "give me all departments", and then you can, for each department, retrieve the employees?
Of course, doing that would likely execute one SQL for each department, but in this case, you're back to "give me all employees with their department information" and you have to build code to handle the rest.

LINQ to SQL doesn't understand your ToList() call, but you might be able to select the sequence of joined elements and then use LINQ to Objects (via AsEnumerable()) to map to your DepartmentSummary object:
var qResult = from d in dba.Department
join e in dba.Employee d.DepartmentID equals e.DepartmentID into j1
select new
{
Department = d,
Employees = j1
};
var result = from d in qResult.AsEnumerable()
select new DepartmentSummary()
{
Department = d.Department,
Employees = e.Employees.ToList()
};

Sounds like you're looking to get around lazy loading?
DataLoadOptions dlo = new DataLoadOptions();
dlo.LoadWith<Department>(d => d.Employees);
using (var dba = new MyDataContext())
{
dba.LoadOptions = dlo;
var result = from d in dba.Department
select d;
}
Now, if you don't have a relationship defined between Department and Employees (the Linq2Sql designer will do this for you if you have database relationships setup) then you should look into doing that. It makes it all dramatically easier. In fact, you don't even need your campaign summary.

This problem is due to the nature of the query. When you join Department to Employee, you'll get back one record for every Employee. This means that your ToList() statement is expecting multiple employees per department, but due to the join, always getting one.
Change your query to
var result =
from d in dba.Department
select new tCampaignSummary
{
Department = d,
Employees = dba.Employee.Where(e => e.DepartmentID ==
d.DepartmentID).ToList()
};
I've tested this and it works.
What it does differently is selects only one record per Department (not per employee) then it gets the zero to many corresponding employees for each dept and converts them to a list.
Good luck!
EDIT
As requested, here is the generated SQL:
SELECT [t0].*, [t1].*
(
SELECT COUNT(*)
FROM [dbo].[Employee] AS [t2]
WHERE [t2].[DepartmentID] = [t0].[DepartmentID]
) AS [value]
FROM [dbo].[Department] AS [t0]
LEFT OUTER JOIN [dbo].[Employee] AS [t1]
ON [t1].[DepartmentID] = [t0].[DepartmentID]
ORDER BY [t0].[DepartmentID], [t1].[IndexID]
The only modification is that LINQ will not do [t0].*, instead it will enumerate each field. Since I had to guess at the fields, I left them out to make the SQL clearer.