We Keep Coding

EF Core & LINQ - How to build a dynamic Where clause using a string variable at run time? [duplicate]

This question already has answers here:
Creating dynamic queries with entity framework
(4 answers)
Closed 3 years ago.
I've read many different variations to this question and I cannot believe the solution I need is so complicated that warrants using additional libraries and crazy tricks, hopefully not!
At runtime, the LINQ query in my project needs to change dynamically depending on how many columns in the DB table the user wants to filter by. In my example, I first show a working LINQ Query which is hard coded.
The next example uses a list that is built at runtime, all i need to figure out is how to insert the string variable (whereClause) into the LINQ Query without compile errors?
Working example (hard coded)
logs = _context.Logs.Where(s => s.Level == LogLevel & s.LogEventCategory == EventCategory)
.Select(s => new Logs()
{
TimeStamp = s.TimeStamp,
Level = s.Level,
Exception = s.Exception,
LogEventCategory = s.LogEventCategory,
LogEventType = s.LogEventType,
LogEventSource = s.LogEventSource,
LogEventName = s.LogEventName,
LogUserName = s.LogUserName,
LogForename = s.LogForename,
LogSurname = s.LogSurname,
LogData = s.LogData
});
Example Two - The solution I want to fix and use...
First create a list, the contents of the list will change each time a new query is run, strings passed as variables through the parent OnGet method will either contain a value and be used in the string join concatenation, or will be null and therefore not added to the list and used in the concatenation.
Second example is where I get compilation errors.
var filtersList = new List<string>();
if (LogLevel != null)
{
filtersList.Add("s.LogLevel == LogLevel");
}
if (EventCategory != null)
{
filtersList.Add("s.EventCategory == EventCategory");
}
var whereClause = string.Join(" & ", filtersList.ToArray());
logs = _context.Logs.Where(s => whereClause) // *HERE I WANT TO USE THE STRING VARIABLE! not working
.Select(s => new Logs()
{
TimeStamp = s.TimeStamp,
Level = s.Level,
Exception = s.Exception,
LogEventCategory = s.LogEventCategory,
LogEventType = s.LogEventType,
LogEventSource = s.LogEventSource,
LogEventName = s.LogEventName,
LogUserName = s.LogUserName,
LogForename = s.LogForename,
LogSurname = s.LogSurname,
LogData = s.LogData
});
The error I get says 'Cannot convert lambda expression to intended delegate type because some of the return types in the block are not implicitly convertible to the delegate return type' blah blah blah

You just add the .Where() calls to the end of your query, before materializing:
query = _context.Logs.Select(s => new ..);
if (EventCategory != null) query = query.Where(e => e.EventCategory == EventCategory);
if (LogLevel != null) query = query.Where(e => e.LogLevel == LogLevel);
var items = query.ToList();

Linq - EntityFramework NotSupportedException

I have a query that looks like this:
var caseList = (from x in context.Cases
where allowedCaseIds.Contains(x => x.CaseId)
select new Case {
CaseId = x.CaseId,
NotifierId = x.NotifierId,
Notifier = x.NotifierId.HasValue ? new Notifier { Name = x.Notifier.Name } : null // This line throws exception
}).ToList();
A Case class can have 0..1 Notifier
The query above will result in the following System.NotSupportedException:
Unable to create a null constant value of type 'Models.Notifier'. Only entity types, enumeration types or primitive types are supported
in this context.
At the moment the only workaround I found is to loop the query result afterwards and manually populate Notifierlike this:
foreach (var c in caseList.Where(x => x.NotifierId.HasValue)
{
c.Notifier = (from x in context.Notifiers
where x.CaseId == c.CaseId
select new Notifier {
Name = x.Name
}).FirstOrDefault();
}
But I really don't want to do this because in my actual scenario it would generate hundreds of additional queries.
Is there any possible solution for a situation like this?.

I think you need to do that in two steps. First you can fetch only the data what you need with an anonymous type in a single query:
var caseList = (from x in context.Cases
where allowedCaseIds.Contains(x => x.CaseId)
select new {
CaseId = x.CaseId,
NotifierId = x.NotifierId,
NotifierName = x.Notifier.Name
}).ToList();
After that, you can work in memory:
List<Case> cases = new List<Case>();
foreach (var c in caseList)
{
var case = new Case();
case.CaseId = c.CaseId;
case.NotifierId = c.NotifierId;
case.NotifierName = c.NotifierId.HasValue ? c.NotifierName : null;
cases.Add(case);
}

You could try writing your query as a chain of function calls rather than a query expression, then put an .AsEnumerable() in between:
var caseList = context.Clases
.Where(x => allowedCaseIds.Contains(x.CaseId))
.AsEnumerable() // Switch context
.Select(x => new Case() {
CaseId = x.CaseId,
NotifierId = x.NotifierId,
Notifier = x.NotifierId.HasValue
? new Notifier() { Name = x.Notifier.Name }
: null
})
.ToList();
This will cause EF to generate an SQL query only up to the point where you put the .AsEnumerable(), further down the road, LINQ to Objects will do all the work. This has the advantage that you can use code that cannot be translated to SQL and should not require a lot of changes to your existing code base (unless you're using a lot of let expressions...)

Linq to entities - SQL Query - Where list contains object with 2 properties (or more)

Having the following example:
var myIds = db.Table1.Where(x=>x.Prop2 == myFilter).Select(x=>x.Id).ToList();
var results = db.Table2.Where(x=> myIds.Contains(x.T1)).ToList();
This part is straight forward.
However, now I am facing a "slight" change where my "filter list" has 2 properties instead of only one:
// NOTE: for stackoverflow simplification I use a basic query to
// get my "myCombinationObject".
// In reality this is a much more complex case,
// but the end result is a LIST of objects with two properties.
var myCombinationObject = db.Table3.Where(x=>x.Prop3 == myFilter)
.Select(x=> new {
Id1 = x.T1,
Id2 = x.T2
}).ToList();
var myCombinationObjectId1s = myCombinationObject.Select(x=>xId1).ToList();
var myCombinationObjectId2s = myCombinationObject.Select(x=>xId2).ToList();
// step#1 - DB SQL part
var resultsRaw = db.Tables.Where( x=>
myCombinationObjectId1s.Contains(x.Prop1)
|| myCombinationObjectId2s.Contains(x.Prop2))
.ToList();
// step#2 - Now in memory side - where I make the final combination filter.
var resultsFiltered = resultsRaw.Where( x=>
myCombinationObject.Contains(
new {Id1 = x.Prop1, Id2 = x.Prop2 }
).ToList();
My question: is it even possible to merge the step#2 in the step#1 (query in linq to entities) ?

I've managed once to do what you want, however it is pretty hard and requires changing entity model a bit. You need an entity to map type
new {Id1 = x.Prop1, Id2 = x.Prop2 }
So you need enity having 2 properties - Id1 and Id2. If you have one - great, if not then add such entity to your model:
public class CombinationObjectTable
{
public virtual Guid Id1 { get; set; }
public virtual Guid Id2 { get; set; }
}
Add it to your model:
public DbSet<CombinationObjectTable> CombinationObjectTable { get; set; }
Create new migration and apply it database (database will have now additional table CombinationObjectTable). After that you start to build a query:
DbSet<CombinationObjectTable> combinationObjectTable = context.Set<CombinationObjectTable>();
StringBuilder wholeQuery = new StringBuilder("DELETE * FROM CombinationObjectTable");
foreach(var obj in myCombinationObject)
{
wholeQuery.Append(string.Format("INSERT INTO CombinationObjectTable(Id1, Id2) VALUES('{0}', '{1}')", obj.Id1, obj.Id2);
}
wholeQuery.Append(
db.Tables
.Where( x=>
myCombinationObjectId1s.Contains(x.Prop1)
|| myCombinationObjectId2s.Contains(x.Prop2))
.Where( x=>
combinationObjectTable.Any(ct => ct.Id1 == x.Id1 && ct.Id2 == x.Id2)
).ToString();
);
var filteredResults = context.Tables.ExecuteQuery(wholeQuery.ToString());
Thanks to this your main query stays written in linq. If you do not want to add new table to your db this is as well achievable. Add new class CombinationObjectTable to model, generate new migration to add it and afterwards remove code creating that table from migration code. After that apply migration. This way the db schema won't be changed but EF will think that there is CombinationObjectTable in database. Instead of it you will need to create a temporary table to hold data:
StringBuilder wholeQuery = new StringBuilder("CREATE TABLE #TempCombinationObjectTable(Id1 uniqueidentifies, Id2 uniqueidentifier);");
And when you invoke ToString method on your linq query change CombinationObjectTable to #TempCombinationObjectTable:
...
.ToString()
.Replace("CombinationObjectTable", "#TempCombinationObjectTable")
Other thing worth considering would be using query parameters to pass values in INSERT statements instead of just including them in query yourself - this is of course achievable with EF as well. This solution is not fully ready to apply, rather some hint in which direction you may go for the solution.

Can you do something like this:
var result=
db.Tables
.Where(t=>
db.Table3
.Where(x=>x.Prop3 == myFilter)
.Any(a=>a.T1==t.Prop1 || a.T2==t.Prop2)
).ToList();

If you simply want to avoid the intermediate result (and also creating a second intermediary list) you can do the following
var resultsFiltered = db.Tables.Where( x=>
myCombinationObjectId1s.Contains(x.Prop1)
|| myCombinationObjectId2s.Contains(x.Prop2))
.AsEnumerable() // everything past that is done in memory but isn't materialized immediately, keeping the streamed logic of linq
.Where( x=>
myCombinationObject
.Contains(new {Id1 = x.Prop1, Id2 = x.Prop2 })
.ToList();

Linq select from CRM public view

I have searching for a while now, but couldn't find how to query from public view. For example, I have predefined public view called Active Accounts and I want data from it.
So far I only know this way, but that not include any views:
using (var xrm = new XrmServiceContext("Xrm"))
{
var activeAccounts = from a in xrm.AccountSet
where a.StateCode == 0
select new { a.Id, a.Name };
// TODO ...
}
But I would like to do it like this (not working, ActiveAccountsView not exist, it's pseudo):
using (var xrm = new XrmServiceContext("Xrm"))
{
var activeAccounts = from a in xrm.ActiveAccountsView
select new { a.Id, a.Name };
// TODO ...
}
Is this even possible?

The query definitions of public views are stored in the savedquery entity, that can be queried using common techniques.
Out-of-the-box views are stored with a fixed ID, so querying Active Accounts on the OrganizationServiceContext object could be done in the following way:
private static IEnumerable<Entity> GetActiveAccounts(OrganizationServiceContext serviceContext)
{
string fetchXml = serviceContext
.CreateQuery("savedquery")
.Where(sq =>
sq.GetAttributeValue<Guid>("savedqueryid") == new Guid("00000000-0000-0000-00AA-000010001002"))
.Select(sq => sq.GetAttributeValue<string>("fetchxml"))
.First();
var request = new RetrieveMultipleRequest
{
Query = new FetchExpression(fetchXml)
};
var response = (RetrieveMultipleResponse) serviceContext.Execute(request);
return response.EntityCollection.Entities;
}
It is not possible to use LINQ here. LINQ relies on the QueryExpression class, but does not implement all its capabilities (OUTER JOIN is a painful omission for example). So, while it is possible to convert a LINQ query to a QueryExpression, the other way around is not.
Paging can be applied by editing the Fetch XML string, but if that is too much hassle, you can also consider to convert the Fetch XML to a QueryExpression and apply paging on that object:
private IEnumerable<Entity> GetActiveAccounts(int pageNumber)
{
string fetchXml = _serviceContext
.CreateQuery("savedquery")
.Where(sq =>
sq.GetAttributeValue<Guid>("savedqueryid") == new Guid("00000000-0000-0000-00AA-000010001002"))
.Select(sq => sq.GetAttributeValue<string>("fetchxml"))
.First();
var conversionRequest = new FetchXmlToQueryExpressionRequest
{
FetchXml = fetchXml
};
var response = (FetchXmlToQueryExpressionResponse)_serviceContext.Execute(conversionRequest);
response.Query.PageInfo = new PagingInfo { Count = 1, PageNumber = pageNumber };
var queryRequest = new RetrieveMultipleRequest
{
Query = response.Query
};
var result = (RetrieveMultipleResponse) _serviceContext.Execute(queryRequest);
return result.EntityCollection.Entities;
}
Additional advantage of the QueryExpression vs. Fetch XML is that it is processed in a bit more efficient way.
The very same can be done with user defined views; these views are stored in the userquery entity. The only difference here is you cannot rely on a fixed view ID. Instead you would need to filter your query on querytype, name, returnedtypecode, ownerid and/or other criteria.
Dynamics CRM also has an OrganizationRequest that allows you to execute the savedquery immediately. However, it returns its result as a resultset XML string, so you would still need to deserialize the response. (A nice example can be found here.) Also, I am not sure if it is possible to limit the result set to a specific page when using the ExecuteByIdSavedQueryRequest:
var request = new ExecuteByIdSavedQueryRequest
{
EntityId = new Guid("00000000-0000-0000-00AA-000010001002")
};
var response = (ExecuteByIdSavedQueryResponse)serviceContext.Execute(request);
string resultset = response.String;

String.Split in a Linq-To-SQL Query?

I have a database table that contains an nvarchar column like this:
1|12.6|18|19
I have a Business Object that has a Decimal[] property.
My LINQ Query looks like this:
var temp = from r in db.SomeTable select new BusinessObject {
// Other BusinessObject Properties snipped as they are straight 1:1
MeterValues = r.MeterValues.Split('|').Select(Decimal.Parse).ToArray()
};
var result = temp.ToArray();
This throws an NotSupportedException: Method 'System.String[] Split(Char[])' has no supported translation to SQL.
That kinda sucks :) Is there any way I can do this without having to add a string property to the business object or selecting an anonymous type and then iterating through it?
My current "solution" is:
var temp = from r in db.SomeTable select new {
mv = r.MeterValues,
bo = new BusinessObject { // all the other fields }
};
var result = new List<BusinessObject>();
foreach(var t in temp) {
var bo = t.bo;
bo.MeterValues = t.mv.Split('|').Select(Decimal.Parse).ToArray();
result.Add(bo);
}
return result.ToArray(); // The Method returns BusinessObject[]
That's kinda ugly though, with that temporary list.
I've tried adding a let mv = r.MeterValues.Split('|').Select(Decimal.Parse).ToArray() but that essentially leads to the same NotSupportedException.
This is .net 3.5SP1 if that matters.

You need to force the select clause to run on the client by calling .AsEnumerable() first:
var result = db.SomeTable.AsEnumerable().Select(r => new BusinessObject {
...
MeterValues = r.MeterValues.Split('|').Select(Decimal.Parse).ToArray()
}).ToList();

You can't use split, but in this scenario you can do the following:
// Database value is 1|12.6|18|19
string valueToFind = "19";
var temp = from r in db.SomeTable.Where(r => ("|" + r.MeterValues + "|").Contains("|" + valueToFind + "|"));
This code adds outer pipes (|) to the database value on the fly inside the query so you can do start, middle, and end value matches on the string.
For example, the above code looks for "|19|" inside "|1|12.6|18|19|", which is found and valid. This will work for any other valueToFind.

You don't need to use a temporary list:
var query = from r in db.SomeTable
select new
{
r.Id,
r.Name,
r.MeterValues,
...
};
var temp = from x in query.AsEnumerable()
select new BusinessObject
{
Id = x.Id,
Name = x.Name,
MeterValues = x.mv.Split('|').Select(Decimal.Parse).ToArray(),
...
};
return temp.ToArray();

Unfortunately its the IQueryable you are using (Linq to SQL) that is not supporting the Split function.
You are really only left with the IEnumerable (Linq to Objects) support for it in this case. You second code snippet is what you need to do, or something like...
var temp = (from r in db.SomeTable select new {
mv = r.MeterValues,
bo = new BusinessObject { // all the other fields }
}).AsEnumerable().Select(blah, blah) ;