We Keep Coding

Combine two list using Linq and add data as needed from different tables

I need to change a process and have been struggling with it for a couple of days now.
The current task checks for all digits entered by the user in Table1. I don't have an issue with that since I can return it with this statement:
var itemsTable1 = db.Table1.Where(a =>
searchNumbers.Contains(a.Digit1) || searchNumbers.Contains(a.Digit2) || searchNumbers.Contains(a.Digit3) ||
searchNumbers.Contains(a.Digit4) || searchNumbers.Contains(a.Digit5) || _Digit6 == a.Digit6 && a.ValidFlag == 1
).ToList();
Now I need to look for the same digits on Table2 and make sure I bring those numbers as well. Although the tables will have the same columns for digits, they will not have the same number of columns in total. I could just right another statement as above for Table2, no problem there. However, I also need to bring the records that do not contain the digits but have the same Ids. So, my scenarios would be something like this:
Table1 = contains digits -> Table2 != contains digits
Table2 = contains digits -> Table1 != contains digits
Table1 = contains digits -> Table2 = contains digits
Finally, I need to display the data on either list in a descending order, which I assume, I'd would have to combine the two/three lists and return it to the model.
Is there a way of doing this with plain Linq? Or am I better off creating maybe a CTE in a stored procedure and pass the parameters there and then calling in the EF?

I assume you need this query:
var query =
from t1 in db.Table1
join t2 in db.Table2 on t1.Id equals t1.Id
let t1Contains = searchNumbers.Contains(t1.Digit1)
|| searchNumbers.Contains(t1.Digit2)
|| searchNumbers.Contains(t1.Digit3)
|| searchNumbers.Contains(t1.Digit4)
|| searchNumbers.Contains(t1.Digit5)
|| _Digit6 == t1.Digit6 && t1.ValidFlag == 1
let t2Contains = searchNumbers.Contains(t2.Digit1)
|| searchNumbers.Contains(t2.Digit2)
|| searchNumbers.Contains(t2.Digit3)
|| searchNumbers.Contains(t2.Digit4)
|| searchNumbers.Contains(t2.Digit5)
|| _Digit6 == t2.Digit6 && t2.ValidFlag == 1
where t1Contains != t2Contains || t1Contains && t2Contains
select
{
t1,
t2
};
Note, that you have not specified desired output and how to order result.

Following #Svyatoslav Danyliv suggestion. I have created the following:
//By using the list, we make sure that the search returns every single digit, regardless of position they occupy in the DB
var itemsT1 = db.Table1.Where(a => searchNumbers.Contains(a.Digit1) || searchNumbers.Contains(a.Digit2) || searchNumbers.Contains(a.Digit3) ||
searchNumbers.Contains(a.Digit4) || searchNumbers.Contains(a.Digit5) || _Digit6 == a.Digit6 && a.ValidDrawResults == 1);
var itemsT2 = db.Table2.Where(a => searchNumbers.Contains(a.Digit1) || searchNumbers.Contains(a.Digit2) || searchNumbers.Contains(a.Digit3) ||
searchNumbers.Contains(a.Digit4) || searchNumbers.Contains(a.Digit5) || _Digit6 == a.Digit6 && a.ValidDrawResults == 1);
//Create list to hold Ids from the records above
List<int?> t1Ids = new List<int?>();
List<int?> t2Ids = new List<int?>();
//Insert the Ids into the lists
foreach (var t1Id in t1Ids )
{
t1Ids.Add((int)t1Id.Id);
}
foreach (var t2Id in t2Ids)
{
t2Ids.Add((int)t2Id.Id);
}
//Get the records from opposite table that contains same Ids
var resultT1 = db.Table1.Where(r => t1Ids.Contains(r.Id)
);
var resultT2 = db.Table2.Where(r => t2Ids.Contains(r.Id)
);
//Combine the lists to pass to the view
var groupedT1 = itemsT1.Concat(resultT1).Distinct();
var groupedT2 = itemsT2.Concat(resultT2).Distinct();
using (db)
{
var vmT1T2 = new ViewModelTables
{
getTable1 = groupedT2.ToList(),
getTable2 = groupedT2.ToList()
};
return View(vmT1T2);
}
It worked out perfectly as far as bring the records that I needed.
Once again, thank you #Svyatoslav Danyliv for pointing me in the right direction. I appreciate and hope this can help someone else as well.

Dynamic query execution in Entity Framework Core

I am developing a hospitality domain application (.Net Core 2.2) in which i am developing a reporting module. From a dashboard few filters are available to fetch records from Data Base.
Below is the DTO i am using to contains filters
public class SearchDto
{
public DateTime DateForm {get;set;}
public DateTime DateTo {get;set;}
public DateSearchType SearchType {get;set;}
public string RegionId {get;set;}
public OrdersStatus status {get;set;}
public string PaymentModeTypes {get;set;}
public string channel {get;set;}
}
Here DateSearchType is a enum with value
Start // service Start Date
End // service End Date
Creation // Order Creation Date
Also OrdersStatus (an enum) with values like All , Confirmed , Cancelled , PaymnetFailed etc
PaymentModeTypes can be a single string or comma seprated string for ex : "NetBanking, CreditCard, DebitCard, Cash"
RegionId is also a single string or comma seprated string as "101, 102, 102"
Same for Channel either "Web" or "Web, Mobile"
Curently ef core expressssion i am using is as follow
var v = Database.Orders.List(
x => ((SearchType == DateSearchType.Start) ? x.Services.Any(y => (y.MinStartTime >= DateForm.Date && y.MinStartTime <= DateTo.Date)) || DateForm.Date.Equals(DateTime.MinValue) : true)
&& ((SearchType == DateSearchType.End) ? x.Services.Any(y => y.MaxEndTime >= DateForm.Date && y.MaxEndTime <= DateTo.Date) : true)
&& ((SearchType == DateSearchType.Creation) ? x.BookingDate.Date >= DateForm.Date && x.BookingDate.Date <= DateTo.Date : true)
&& (RegionId.Length == 0 || RegionId.Contains(x.RegionId))
&& (status == OrdersStatus.All || x.Status == status)
&& (string.IsNullOrEmpty(PaymentModeTypes) || PaymentTypes.Contains(x.PaymentType))
&& (string.IsNullOrEmpty(channel) || channels.Contains(x.ChannelName)), //Where
x => x.Guests,
x => x.Services
);
Here Guests and Services are two another tables set as navigation property in Orders Model
This expression is working fine but taking too much time to execute , any good approach to optimize it or right way to rewrite this code ?
Also what is the best practice to exclude any filter if its value is not provided.
Few Filters are not mandatory, they can be supplied or not, so query execution has to be of dynamic nature. current implementation if a filter value is not supplied
&& (string.IsNullOrEmpty(PaymentModeTypes) || PaymentTypes.Contains(x.PaymentType))
Can anybody suggest some good material or any piece of code regarding this so i can optimize it.
Please ignore Typo as i am not habitual to use dark theme
Edit 1:Client Evaluation set as off
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder
.ConfigureWarnings(warnings => warnings.Throw(RelationalEventId.QueryClientEvaluationWarning));
}

There is some inefficiency in the generated SQL which might be causing the performance issue.
Before EF Core one is expected to conditionally chain multiple Where calls or use some predicate builder utility to build conditionally Where predicate having only the necessary conditions.
This normally is not needed in EF Core, because it tries to automatically eliminate such conditions. It does that for logical expressions (|| , &&), but failing to do so for the conditional expressions (? :). So the solution is to replace the later with the equivalent logical expressions.
You are doing that for most of your conditions, but not for the first 3. So replace
x => ((SearchType == DateSearchType.Start) ? x.Services.Any(y => (y.MinStartTime >= DateForm.Date && y.MinStartTime <= DateTo.Date)) || DateForm.Date.Equals(DateTime.MinValue) : true)
&& ((SearchType == DateSearchType.End) ? x.Services.Any(y => y.MaxEndTime >= DateForm.Date && y.MaxEndTime <= DateTo.Date) : true)
&& ((SearchType == DateSearchType.Creation) ? x.BookingDate.Date >= DateForm.Date && x.BookingDate.Date <= DateTo.Date : true)
with
x => (SearchType != DateSearchType.Start || x.Services.Any(y => y.MinStartTime >= DateForm.Date && y.MinStartTime <= DateTo.Date) || DateForm.Date.Equals(DateTime.MinValue))
&& (SearchType != DateSearchType.End || x.Services.Any(y => y.MaxEndTime >= DateForm.Date && y.MaxEndTime <= DateTo.Date))
&& (SearchType != DateSearchType.Creation || x.BookingDate.Date >= DateForm.Date && x.BookingDate.Date <= DateTo.Date)
and see if that helps. For sure the generated SQL will be optimal.
Noticed that you are using different variable names for comma separated strings and Contains filters. So I'm assuming you have something like this
public IEnumerable<string> PaymentTypes => (PaymentModeTypes ?? "").Split(", ");
public IEnumerable<string> channels => (channel ?? "").Split(", ");
which is good and will generate SQL IN (...) conditions when necessary.
You might consider doing the same for regions, e.g. add
public IEnumerable<string> RegionIds => (RegionId ?? "").Split(", ");
and replace
&& (RegionId.Length == 0 || RegionId.Contains(x.RegionId))
with
&& (string.IsNullOrEmpty(RegionId) || RegionIds.Contains(x.RegionId))

Excluding result based on length of a field

Below is my LINQ query:
list = (from i in context.Cars
.Where(c => terms.All(t => c.Code.Contains(t) || c.ShortDescription.Contains(t)
&& c.Code.Replace(" " , "").Length >3))
select new Model.Cars
{
CarId = i.CarId,
ShortDescription = i.ShortDescription,
Code = i.Code
}).Take(250).ToList();\
One of the business requirement is to exclude any record where code length is shorter than 3. A lot of these codes have whitespaces in them which is why I put in clause of replacing " " with "". This does not seem to work in my query. I am still getting results where code length is 3. I should only get results where code length is greater than3. It almost seems like the replace is not replacing whitespaces with no space. Everything else works. What am I doing wrong ?

Operator precedence strikes again.
.Where(c =>
terms.All(t =>
( c.Code.Contains(t) || c.ShortDescription.Contains(t) )
&& c.Code.Replace(" " , "").Length > 3
)
)
if (true || true && false)
MessageBox.Show("Gotcha!");

Why are you switching bewteen LINQ & lambda syntax? Had you stuck to the LINQ syntax, you probably would have spotted the precedence problem:
list = (from i in context.Cars
let code = i.Code.Trim()
where terms.All(t => code.Contains(t) || i.ShortDescription.Contains(t))
&& code.Length > 3
select new Model.Cars
{
CarId = i.CarId,
ShortDescription = i.ShortDescription,
Code = code
}).Take(250).ToList();

stating which conditions have been matched in linq where

I have a linq statement as such
dbContext.Items
.Where(
p =>
(p.Client.Contact != null && p.Client.Contact.Firstname.ToLower().Contains(searchText.ToLower()))
||
(p.Client.Contact != null && p.Client.Contact.Surname.ToLower().Contains(searchText.ToLower()))
||
(p.PolicyNumber != null && p.PolicyNumber.ToLower().Contains(searchText.ToLower()))
||
(
p.PolicyLivesAssureds
.Where(
pl =>
pl.Contact != null && pl.Contact.Firstname.ToLower().Contains(searchText.ToLower())
|| pl.Contact.Surname.ToLower().Contains(searchText.ToLower())
).Count() > 0
)
)
).OrderBy(p => p.IeUtem);
This is actually needed in an autocomplete. What I want to do is being able to know exactly which among my 5 conditions has been matched and display the particular item that has been matched. For example say that PolicyNumber has been matched i want to send only policynumber for that row and for others if name has been matched i want to send only the name for that row.
Is there a way to do this;

This is a bit more of a food for thought answer as it has flaws in it's approach, but I think it does solve your problem:
double[] items = { 1, 2, 3, 4, 5 };
IEnumerable<Tuple<double, int>> results = items.Select(x =>
{
int index = 0;
foreach (var condition in new Func<bool>[]
{
// TODO: Write conditions here.
() => x == 1,
() => x == 2
})
{
if (condition() == true)
return index;
else
index++;
}
return -1;
}).Zip(items, (matchedCondtion, item) => Tuple.Create(item, matchedCondtion))
.Where(x => x.Item2 != -1);
I've used a simple double array as an example of the collection to filter, but it's just an example, you can use anything.
The first select returns an integer for each element in the collection. If there is a condition match, it returns the index of the condition. If there is not match it returns -1.
It does this by enumerating over the Func collection and returning the index of the first true condition (emulating the short circuiting of the || operator). If no conditions match it simply returns -1 after evaluating all conditions.
These results are then zipped back up with the original collection (using a Tuple), mapping each element with the index of its matching condition (or -1).
So the example would return:
{ 1, 0 },
{ 2, 1 },
{ 3, -1 },
{ 4, -1 },
{ 5, -1 }
This result is then simply filtered using Where to remove any entries with -1, leaving you with a collection of elements that matched a condition and the index of the condition that matched (in the form of a Tuple).
So to customize this for your solution, you can remove the example conditions and place whatever number of conditions you want at:
// TODO: Write conditions here.

The question becomes how do you want to know which queries match. For example you could do something like this
class AutoCompleteItem {
String Text {get; set;}
Item Item {get; set;}
}
var firstNames = dbContext.Items.Select(p => new AutoCompleteItem { Name = p.Client.Contract.FirstName, Item = p})
var lastNames = dbContext.Items.Select(p => new AutoCompleteItem { Name = p.Client.Contract.SurName, Item = p})
var result = firstName.Union(lastNames).Where(p => p.Name.Contains(searchText)).OrderBy(a => a.Item.IeUtem);
Now AutcompleteItem is a class that contains the text you want (and possibly any other fields you need, like information which field it was that matched)
The Idea here is the MVVM patttern. You have your model (the items). Now you need to construct a viewModel (AutoCompleteItems) that actual aids you in displaying what you want.

Avoiding null exception in EF that is thrown by your query

I have a query like this :
result =
firstIdeaRepository.FindBy(
i => i.FirstIdeaState == FirstIdeaState && i.Date >= start && i.Date <= end)
.AsEnumerable()
.Select(j => new RptListOfCompanyBasedOnFirstIdeaState()
{
Name =
companyRepository.FindBy(i => i.UserId == j.UserId)
.FirstOrDefault()
DateOfMeeting =
calenderRepository.ConvertToPersianToShow(
meetingReposiotry.FindBy(s => s.FirstIdeaId == j.Id)
.FirstOrDefault()
.Date),
DateOfExit =
calenderRepository.ConvertToPersianToShow(j.DateOfExit.Value),
ReasonOfExit = j.ReasonOfExit,
}).ToList();
return result;
As you can see i use FirstOrDefault() and j.DateOfExit.Value and sometimes my Date doesn't have any values or sometime my other variables are null too because i use firstordefaut() like
companyRepository.FindBy(i => i.UserId == j.UserId).FirstOrDefault().
So my query throws a null exception and the result can't be created ,how can i handle this exception and for example if the .NET detects the null value ignores it by default or uses a default values for that ?
Best regards.

I would make the following changes:
result =
firstIdeaRepository.FindBy(
i => i.FirstIdeaState == FirstIdeaState && i.Date >= start && i.Date <= end)
.AsEnumerable()
.Select(j => new RptListOfCompanyBasedOnFirstIdeaState()
{
Name =
companyRepository.FindBy(i => i.UserId == j.UserId)
.FirstOrDefault()
DateOfMeeting =
callenderRepository.ConvertToPersianToShow(
meetingReposiotry.FindBy(s => s.FirstIdeaId == j.Id)
// project a new sequence first, before calling `FirstOrDefault`:
.Select(s => s.Date)
.FirstOrDefault(),
DateOfExit =
j.DateOfExit.HasValue ?
callenderRepository.ConvertToPersianToShow(j.DateOfExit.Value) :
null,
ReasonOfExit = j.ReasonOfExit,
}).ToList();
When you use FirstOrDefault, there's a possibility that you'll get null back (in the case of reference types), and so you need to plan for that in your code.
For example, when assigning DateOfMeeting, you could project the results (using .Select) before using .FirstOrDefault, so that you're not ever accessing the Date property on what could be a null value.
As for DateOfExit, I've used the conditional operator to determine whether to call the calendarRepository's method at all. This assumes that DateOfExit is nullable.
Unrelated: "Calendar" is spelled with one "l" and not two.

Since you're using a nullable date, you can try filtering by values that have date, something like:
.FindBy(s => s.FirstIdeaId == j.Id && s.Date.HasValue)
This will ensure that you don't get any records with null date.
As I mentioned in comments, other cases need to be handled on case-by-case basis. Judging by the code you've shown, maybe you can handle Name as:
Name = companyRepository.FindBy(i => i.UserId == j.UserId).FirstOrDefault() ?? "anonymous";
and so on.
Another example:
If you do want to get the record even if DateOfMeeting is null, then add a check for HasValue in subsequent part or default it to some date:
DateOfExit = j.DateOfExit.HasValue ?
callenderRepository.ConvertToPersianToShow(j.DateOfExit.Value)
: (DateTime)null, // you need to make `DateOfExit` nullable and then handle that downstream
// or (default with current date)
DateOfExit = j.DateOfExit.HasValue ?
callenderRepository.ConvertToPersianToShow(j.DateOfExit.Value)
: callenderRepository.ConvertToPersianToShow(DateTime.Now),
// or (default with empty date)
DateOfExit = j.DateOfExit.HasValue ?
callenderRepository.ConvertToPersianToShow(j.DateOfExit.Value)
: callenderRepository.ConvertToPersianToShow(new DateTime()),
Moral of the story: figure out what the default value should be in case of null and then substitute that accordingly in the query when calling FirstOrDefault().

The broadest solution would be to use the idea of an null object with the DefaultIfEmpty<T>(T DefaultValue) method in your query. An example would be:
var defaultMeeting = new Meeting() { Date = new DateTime() };
var dateOfMeeting = meetingRepository.FindBy(s => s.FirstIdeaId == j.Id)
.DefaultIfEmpty(defaultMeeting)
.FirstOrDefault()
.Date;