We Keep Coding

Select all columns from main table and only 1 column from JOINed table

I have two tables:
Table1
Id ArticleName ArticleTypeId
1 Blah Blah 3
2 Helo Blah 5
and
Table2
ArticleTypeId TypeName
3 Business
5 Construction
I'm trying to Join TableA and TableB on ArticleTypeId and basically return everything from Table1 and TypeName from Table2
Here's what I'm trying to do, but I'm not sure to to edit the SELECT in the statement to include the TypeName
var articles = (from s in _context.Articles
join b in _context.ArticleTypes on s.ArticleTypeId equals b.ArticleTypeId
select s).ToList();
Or is there an easier way to do this?
Goal:
Id ArticleName TypeName
1 Blah Blah Business
2 Helo Blah Construction

So you have two tables, a table with Articles (Table1), and a table with ArticleTypes (Table2). I decided to give your tables names that are meaningful, this makes discussion easier.
There is a one to many relationship between Articles and ArticleTypes: Every Article has exactly one ArticleType, namely the Article type that the foreign key ArticleTypeId refers to. Every ArticleType has zero or more Articles that refer to it.
You are using entity framework. If you've followed Entity Framework Coding Conventions, you'll have classes similar to the following.
class Article
{
public int Id {get; set;}
public string Name {get; set;}
// every Article has one ArticleType, namely the one that the foreign key refers to
public int ArticleTypeId {get; set;}
public virtual ArticleType ArticleType {get; set;}
}
class ArticleType
{
public int Id {get; set;}
public string TypeName {get; set;}
// every ArticleType has zero or more Articles referring to it (one-to-many)
public virtual ICollection<Article> Articles {get; set;}
}
In entity framework the non-virtual properties refer to columns of the tables; the virtual properties refer to the relations between the tables (one-to-many, many-to-many, ...)
The foreign key ArticleTypeId is a real column, so the property is non-virtual. Property ArticleType is virtual, because it represents the one-to-many relation.
For completeness your DbContext:
class MyWarehouse : DbContext
{
public DbSet<Article> Articles {get; set;}
public DbSet<ArticleType> ArticleTypes {get; set;}
}
I'm trying to Join TableA and TableB on ArticleTypeId and basically return everything from Table1 and TypeName from Table2
After you've defined your classes, your query is easy. The easiest method is using the virtual properties.
Use the virtual properties
Requirement Give me the Id and Name of all Articles, each Article with its TypeName.
using (var wareHouse = new MyWareHouse(...))
{
var requestedArticles = wareHouse.Articles.Select(article => new
{
// Select only the Article Properties that you plan to use
Id = article.Id,
Name = article.Name,
TypeName = article.ArticleType.TypeName,
});
// Process the requested Articles before disposing the wareHouse
}
In words: from every Article in the table of Articles fetch the Id, the Name and the one and only TypeName it has.
Entity Framework knows the relation between Articles and ArticleTypes. Because you use the virtual Property Article.ArticleType it knows which join to perform.
Using the virtual properties you can also get each ArticleType together with all Articles that have this ArticleTypes
var constructionArticles = wareHouse.ArticleTypes
.Where(articleType => articleType.TypeName == "construction")
.Select(articleType => new
{
Id = articleType.Id,
TypeName = articleType.TypeName,
// fetch all articles that have this TypeName
Articles = articleType.Articles.Select(article => new
{
Id = article.Id,
Name = article.Name,
// no need to fetch the foreign key, you already got this value
// ArticleTypeId = article.ArticleTypeId,
})
.ToList(),
})
.ToList();
Entity framework knows the relation and will do the proper (Group-)join for you.
Did you notice how natural using the virtual properties feel?
Do the Join yourself
Some people don't want to use the virtual properties, they prefer to do the (Group-)joins themselves.
Use the overload of method Join that has a parameter resultSelector, so you can specify the desired result.
// Join Articles with ArticleTypes
var requestedArticles = wareHouse.Articles.Join(wareHouse.ArticleTypes,
// from every Article take the foreign key
article => articleTypeId,
// from every ArticleType take the primary key
articleType => articleType.Id,
// parameter resultSelector:
// take each article and its one and only matching ArticleType to make one new
(article, articleType) => new
{
Id = article.Id,
Name = article.Name
TypeName = articleType.TypeName,
});
If you have a one-to-many relation, like Schools with their Students, Customers with their Orders, or ArticleTypes with their Articles, use GroupJoin and start at the 'one' side. If you want Students, each Student with the School he attends, use Join, and start at the 'many' side.
var schoolsWithTheirStudents = dbContext.Schools
.Where(school => school.City == ...) // if you don't want all Schools
.GroupJoin(dbContext.Students,
// from every School take the primary key
school => school.Id,
// from every Student take the foreign key to the School he attends
student => student.SchoolId,
// resultSelector: take each Schools with its matching Students to make one ned
(school, studentsWhoAttendThisSchool) => new
{
// Select only the School properties that you plan to use:
Id = school.Id,
Name = school.Name,
Address = school.Address,
...
// studentsWhoAttendThisSchool is a queryable sequence,
// so you can use LINQ on it:
Students = studentsWhoAttendThisSchool.Select(student => new
{
Id = student.Id,
Name = student.Name,
...
})
.ToList(),
});
})

What is the correct way to do many to many entity relation update in Entity framework core 6?

public class Book
{
public int Id { get; set; }
public string Name{ get; set; }
public string ISBN { get; set; }
public ICollection<Category> Categories { get; set; }
}
public class Category
{
public int Id { get; set; }
public string Name{ get; set; }
public ICollection<Book> Books { get; set; }
}
I a using Entity Framework core 6 with .NET 6.
I am trying to update the Categories of a specific Book.
For example, If one book has categories like .NET, C# then I want to update categories into .NET, EF Core, SqlServer, I think you get it now.
Do I need to add a Join entity for only the Update operation? As you can see I have not created any Join entity like BookCategories though I managed to Insert categories while creating Book for the first time.
But when trying to update the book with new categories I am getting two issues.
The old category is not deleted.
And getting Duplicate Error Key while trying to update with existing category, in this case, .NET.
Please kindly show the proper way of updating related entities in Entity Framework Core 6 in .NET6.

Many-to-Many relationships need a bit of configuration depending on what you want out of the relationship. If you just want the linking table to manage the link and nothing else:
[BookCategories]
BookId (PK, FK)
CategoryId (PK, FK)
Then you can set up the relationship to either use an entity definition or a shadow entity. In both cases this is typically preferable since your Book can have a collection of Categories, and the Category can have a collection of books. With Code-First and Migrations I believe EF can and will set up this linking table automatically. Otherwise you can use OnModelCreating or an EntityTypeConfiguration to configure what Table and Columns to use for the relationship.
This can be done either with an Entity declared for BookCategory, or without one:
With entity:
modelBuilder.Entity<Book>()
.HasMany(x => x.Categories)
.WithMany(x => Books);
.UsingEntity<BookCategory>(
l => l.HasOne<Book>().WithMany().HasForeignKey(x => x.BookId),
r => r.HasOne<Category>().WithMany().HasForeignKey(x => x.CategoryId),
j =>
{
j.HasKey("BookId", "CategoryId");
j.ToTable("BookCategories");
});
Without entity: (See Scaffolding many-to-many relationships - https://learn.microsoft.com/en-us/ef/core/what-is-new/ef-core-6.0/whatsnew)
modelBuilder.Entity<Book>()
.HasMany(x => x.Categories)
.WithMany(x => Books);
.UsingEntity<Dictionary<string, object>>(
"BookCategories",
l => l.HasOne<Book>().WithMany().HasForeignKey("BookId"),
r => r.HasOne<Category>().WithMany().HasForeignKey("CategoryId"),
j =>
{
j.HasKey("BookId", "CategoryId");
j.ToTable("BookCategories");
});
Alternatively, if the joining table needs to contain additional relevant details, for example if you are using a soft-delete system and want to mark deleted relationships as inactive rather than deleting those rows, then you have to adopt an indirect relationship using a BookCategory entity where Book has a collection of BookCategories, as does Category. (See Join entity type configuration - https://learn.microsoft.com/en-us/ef/core/modeling/relationships?tabs=fluent-api%2Cfluent-api-simple-key%2Csimple-key)
Once you have your relationships set up, it is important to treat these relationships as associations, not copies of data. This means you should ensure that your collections are initialized on construction, and never reset. You can add items to the collection or remove items from the collection, but you should never have code that resets the collection. (I.e. no code that does stuff like book.Categories = new List<Category>() or book.Categories = myUpdatedCategories etc.) While EF is tracking entities, it is relying on proxies to help with change tracking to know when data needs to be added, removed, or updated. This also means if you want to "change" a book's category, this is a remove and add, not an update.
For instance to change a book's category from "Java" to ".Net", you don't want to do something like:
var book = context.Books.Include(x => x.Categories).Single(x => x.BookId == bookId);
var category = book.Categories.SingleOrDefault(x => x.CategoryName == "Java");
if (category != null)
category.CategoryName = ".Net"; // or category.CategoryId = dotNetCategoryId;
This would attempt to modify the Category record to change it's Name (likely not intended) or attempt to change it's PK. (illegal)
Instead, you want to change the association:
var dotNetCategory = context.Categories.Single(x => x.CategoryId == dotNetCategoryId);
var book = context.Books.Include(x => x.Categories).Single(x => x.BookId == bookId);
var category = book.Categories.SingleOrDefault(x => x.CategoryName == "Java");
if (category != null)
{
book.Categories.Remove(category);
book.Categories.Add(dotNetCategory);
}
Behind the scenes, EF will delete the BookCategory linking the book to Java category, and insert a BookCategory with the new .Net association. If you have a joining entity then you will just need to remove, add, or update the BookCategory entity specifically based on the relationship changes you want to make.

I need to do a query on a returned list of users from another query, by Id

Currently, I have a service that grabs user information from the User table. The users can be created by admins or an employee and all of these employees have their own Id. So with that in mind, there is a column called CreatedBy which holds the id of this admin, or the user, that of which's name I have to return. So far I've pulled the user model but now I need to create the part where I pull the user's name with the user.Id in the CreatedBy
This is what I have pulling from my database tables Users and Company and the query parameters are just a name or company name
public async Task<List<ApplicationUser>> SearchUsers(UserSearchDto userSearchDto)
{
userSearchDto.FirstName ??= string.Empty;
userSearchDto.LastName ??= string.Empty;
userSearchDto.CompanyName ??= string.Empty;
return await _locationDbContext.Users
.Include(nameof(Company))
.Where(user => user.FirstName.Contains(userSearchDto.FirstName)
&& user.LastName.Contains(userSearchDto.LastName)
&& user.Company.Company_Name.Contains(userSearchDto.CompanyName))
.ToListAsync();
}
So within this list that I am returning I'm trying to do another query to grab more user information based on the CreatedBy id's returned in the first service to bring back the name of those users with the id's in CreatedBy.
var userDtos = _mapper.Map<List<ApplicationUser>, List<UserDetailsDto>>(users);
foreach (UserDetailsDto user in userDtos)
{
user.CreatedByName = await _userService
.SearchUsers(userDtos.Where(user.Id == user.CreatedBy))
}
I feel like this is a possible solution so far but I'm not sure where or how I would pull that because this solution is giving me an error at the point where I use the ".Where" statement. Maybe if I could create another service that would return the user by Id instead and use the createdby Id to pull the name but nothing like that exists yet. The model I'd like to return is also a bit different from the model representing the Users table as ApplicationUser has the CreatedBy which is an Id but the returned model, userDetailsDto will have a name string property as well that I will try and assign here in automapper. If I can think of how I can assign the name by the Id.
CreateMap<ApplicationUser, UserDetailsDto>()
.ForMember(dest => dest.CompanyName,
opts => opts.MapFrom(src => src.Company.Company_Name));

Ideally this is something that you should be able to resolve using navigation properties. If your User table uses CreatedBy to represent the CreatedBy User ID then you could adjust your mapping to facilitate a CreatedBy navigation property:
public class User
{
public class UserId { get; set; }
// ...
public virtual User CreatedBy { get; set; }
}
Then in the mapping use a shadow property for the FK association: (in OnModelCreating or using an EntityTypeConfiguration)
EF Core
.HasOne(x => x.CreatedBy)
.WithMany()
.HasForeignHey("CreatedBy") // Property on Users table
.Required();
EF 6
.HasRequired(x => x.CreatedBy)
.WithMany()
.Map(x => x.MapKey("CreatedBy")) // Property on Users table
Alternatively if you want the CreatedBy FK accessible in the User table, map it as something like CreatedByUserId:
public class User
{
public int UserId { get; set; }
// ...
[ForeignKey("CreatedBy"), Column("CreatedBy")]
public int CreatedByUserId { get; set; }
public virtual User CreatedBy { get; set; }
}
Now when you go to search for your users, you can project your CreatedBy user ID and Name in one go.
When it comes to optional search parameters you should keep the conditionals (if/else/ null checks etc ) outside of the Linq wherever possible. This helps compose more efficient queries rather than embedding conditional logic into the SQL.
public async Task<List<ApplicationUserViewModel>> SearchUsers(UserSearchDto userSearchDto)
{
var query = _locationDbContext.Users.AsQueryable();
if (!string.IsNullOrEmpty(userSearchDto.FirstName))
query = query.Where(x => x.FirstName.Contains(userSearchDto.FirstName));
if (!string.IsNullOrEmpty(userSearchDto.LastName))
query = query.Where(x => x.LastName.Contains(userSearchDto.LastName));
if (!string.IsNullOrEmpty(userSearchDto.CompanyName))
query = query.Where(x => x.Company.Name.Contains(userSearchDto.CompanyName));
return await query.ProjectTo<ApplicationUserViewModel>(_config)
.ToListAsync();
}
Where _config reflects an automapper MapperConfiguration containing the details on how to map a User to your desired view model. If you're not leveraging Automapper you can accomplish this using Select to project the values. Other considerations there would be to consider using StartsWith instead of Contains, perhaps offering an option to perform a more expensive Contains search... Also adding things like minimum search length checks (I.e. 3+ characters) and pagination or result row limits (I.e. Take(50)) to avoid outrageous search requests from hammering your system. (I.e. searching for users with "e" in the first name)
That view model might have UserId, UserName, CompanyName, then things like CreatedByUserId, CreatedByName. To resolve the CreatedBy details you just reference u.CreatedBy.UserId and u.CreatedBy.Name either in the Automapper config or within your Select(u => new ApplicationUserViewModel { ... }) statement.

How to query IQueryable with Include - ThenInclude?

In .NET Core 2.2 I'm stuck with filtering IQueryable built as:
_context.Ports.Include(p => p.VesselsPorts)
.ThenInclude(p => p.Arrival)
.Include(p => p.VesselsPorts)
.ThenInclude(p => p.Departure)
.OrderBy(p => p.PortLocode);
in many-to-many relation. And the entity models are such as:
public class PortModel
{
[Key]
public string PortLocode { get; set; }
public double? MaxKnownLOA { get; set; }
public double? MaxKnownBreadth { get; set; }
public double? MaxKnownDraught { get; set; }
public virtual ICollection<VesselPort> VesselsPorts { get; set; }
}
public class VesselPort
{
public int IMO { get; set; }
public string PortLocode { get; set; }
public DateTime? Departure { get; set; }
public DateTime? Arrival { get; set; }
public VesselModel VesselModel { get; set; }
public PortModel PortModel { get; set; }
}
Based on this this SO answer I managed to create LINQ like that:
_context.Ports.Include(p => p.VesselsPorts).ThenInclude(p => p.Arrival).OrderBy(p => p.PortLocode)
.Select(
p => new PortModel
{
PortLocode = p.PortLocode,
MaxKnownBreadth = p.MaxKnownBreadth,
MaxKnownDraught = p.MaxKnownDraught,
MaxKnownLOA = p.MaxKnownLOA,
VesselsPorts = p.VesselsPorts.Select(vp => vp.Arrival > DateTime.UtcNow.AddDays(-1)) as ICollection<VesselPort>
}).AsQueryable();
BUT what I need is to find all port records, where:
VesselsPorts.Arrival > DateTime.UtcNow.AddDays(-1) quantity is greater than int x = 5 value (for the example). And I have no clue how to do it :/

Thanks to #GertArnold comment, I ended up with query:
ports = ports.Where(p => p.VesselsPorts.Where(vp => vp.Arrival > DateTime.UtcNow.AddDays(-1)).Count() > x);

When using entity framework people tend to use Include instead of Select to save them some typing. It is seldom wise to do so.
The DbContext holds a ChangeTracker. Every complete row from any table that you fetch during the lifetime of the DbContext is stored in the ChangeTracker, as well as a clone. You get a reference to the copy. (or maybe a reference to the original). If you change properties of the data you got, they are changed in the copy that is in the ChangeTracker. During SaveChanges, the original is compared to the copy, to see if the data must be saved.
So if you are fetching quite a lot of data, and use include, then every fetched items is cloned. This might slow down your queries considerably.
Apart from this cloning, you will probably fetch more properties than you actually plan to use. Database management systems are extremely optimized in combining tables, and searching rows within tables. One of the slower parts is the transfer of the selected data to your local process.
For example, if you have a database with Schools and Students, with the obvious one to many-relation, then every Student will have a foreign key to the School he attends.
So if you ask for School [10] with his 2000 Students, then every Student will have a foreign key value of [10]. If you use Include, then you will be transferring this same value 10 over 2000 times. What a waste of processing power!
In entity framework, when querying data, always use Select to select the properties, and Select only the properties that you actually plan to use. Only use Include if you plan to change the fetched items.
Certainly don't use Include to save you some typing!
Requirement: Give me the Ports with their Vessels
var portsWithTheirVessels = dbContext.Ports
.Where(port => ...) // if you don't want all Ports
.Select(port => new
{
// only select the properties that you want:
PortLocode = port.PortLoCode,
MaxKnownLOA = port.MaxKnownLOA,
MaxKnownBreadth = prot.MaxKnownBreadth,
MaxKnownDraught = ports.MaxKnownDraught,
// The Vessels in this port:
Vessels = port.VesselsPort.Select(vessel => new
{
// again: only the properties that you plan to use
IMO = vessel.IMO,
...
// do not select the foreign key, you already know the value!
// PortLocode = vessle.PortLocode,
})
.ToList(),
});
Entity framework knows your one-to-many relation, and knows that if you use the virtual ICollection that it should do a (Group-)Join.
Some people prefer to do the Group-Join themselves, or they use a version of entity framework that does not support using the ICollection.
var portsWithTheirVessels = dbContext.Ports.GroupJoin(dbContext.VesselPorts,
port => port.PortLocode, // from every Port take the primary key
vessel => vessel.PortLocode, // from every Vessel take the foreign key to Port
// parameter resultSelector: take every Port with its zero or more Vessels to make one new
(port, vesselsInThisPort) => new
{
PortLocode = port.PortLoCode,
...
Vessels = vesselsInThisPort.Select(vessel => new
{
...
})
.ToList(),
});
Alternative:
var portsWithTheirVessels = dbContext.Ports.Select(port => new
{
PortLocode = port.PortLoCode,
...
Vessels = dbContext.VesselPorts.Where(vessel => vessel.PortLocode == port.PortLocode)
.Select(vessel => new
{
...
}
.ToList(),
});
Entity framework will translate this also to a GroupJoin.

LINQ Left Outer join with group by count through extension methods

I have two tables. Categories and Ads.
Every Category can have many ads in Ads table.
I want all the categories details including how many ads do they have each.
Following is my Linq expression it fetches only categories that have ads may be bacause FirstOrDefault()
I like to know how I can achieve the result given that the condition plus only categories that have location id (let say: lid) of "7".
Following in my expression
var x1 = context.Categories
.GroupJoin(
context.ads,
cat => cat.id,
ad => ad.catid,
(cat, ad) => new { cats = cat, ads = ad })
.SelectMany(
a => a.ads.DefaultIfEmpty(),
(a, y) => new { catss = a.cats, adss = y })
.GroupBy(w => w.adss,ww=>new { cat=ww.catss,count=ww.catss.ads.Count()})
.Where(s=>s.FirstOrDefault().cat.lid==7);

Apparently there is a one-to-many relation between Categories and Ads: every Category has zero or more Ads, and every Ad belongs to exactly one Category.
In proper Entity Framework this would be modelled as follows:
class Category
{
public int Id {get; set;}
// every Category has zero or more Adds:
public virtual ICollection<Ad> Ads {get; set;}
...
}
class Ad
{
public int Id {get; set;}
// every Ad belongs to exactly one Category, using foreign key CategoryId:
public int CategoryId {get; set;}
public Category Category {get; set;}
...
}
If your classes are modelled like this, entity framework will assume the proper one-to-many relationship between Category and Add. It might be that you have different names for certain properties, and that you use Attributes and / or fluent API, but the structure of the classes will be similar.
Having done this, your query is easier than you thought:
var result = dbContext.Categories.Select(category => new
{
Category = Category,
AdCount = Category.Adds.Count(),
};
In words: for each Category in the collection of categories, create a new object of anonymous class, with two properties:
Category contains the considered Category
AdCount contains the number of Ads that the Category has.
The model of Entity Framework will understand that a join and a count is needed to do this.

You could try something like the following:
var result = context.Categories
.Where(category => category.lid == 7)
.GroupJoin(
context.ads
, category => category.id
, ad => ad.catid
, (c,a) => new
{
Category = category,
NumberOfAds = a.DefaultIfEmpty().Count(x => x!=null)
});