Cannot perform LINQ complex object search in Entity Framework - c#

I have entities in the DB which each contain a list of key value pairs as metadata. I want to return a list of object by matching on specified items in the metadata.
Ie if objects can have metadata of KeyOne, KeyTwo and KeyThree, I want to be able to say "Bring me back all objects where KeyOne contains "abc" and KeyThree contains "de"
This is my C# query
var objects = repository.GetObjects().Where(t =>
request.SearchFilters.All(f =>
t.ObjectChild.Any(tt =>
tt.MetaDataPairs.Any(md =>
md.Key.ToLower() == f.Key.ToLower() && md.Value.ToLower().Contains(f.Value.ToLower())
)
)
)
).ToList();
and this is my request class
[DataContract]
public class FindObjectRequest
{
[DataMember]
public IDictionary<string, string> SearchFilters { get; set; }
}
And lastly my Metadata POCO
[Table("MetaDataPair")]
public class DbMetaDataPair : IEntityComparable<DbMetaDataPair>
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public long Id { get; set; }
[Required]
public string Key { get; set; }
public string Value { get; set; }
}
The error I get is
Error was Unable to create a constant value of type
'System.Collections.Generic.KeyValuePair`2[[System.String, mscorlib,
Version=4.0.0.0, Culture=neutral,
PublicKeyToken=b77a5c561934e089],[System.String, mscorlib,
Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]'.
Only primitive types or enumeration types are supported in this
context.

So it looks like the f variable in your query is a KeyValuePair<string, string>. What you need to do is save them as local variables before you use them as a KVP cannot be converted to SQL.
var filters = request.SearchFilters.Select(kvp => new[] { kvp.Key, kvp.Value }).ToArray();
var objects = repository.GetObjects().Where(t =>
filters.All(f =>
t.ObjectChild.Any(tt =>
tt.MetaDataPairs.Any(md =>
md.Key.ToLower() == f[0] && md.Value.ToLower().Contains(f[1])
)
)
)
).ToList();
What you have to remember is that anything you do in LINQ in EF when it is still of the type IQueryable<T> - which is before you call ToList(), ToArray(), ToDictionary() and maybe even AsEnumerable() (I have honestly never tried AsEnumerable()) - must be able to be represented as SQL. That means that only SQL types (string, int, long, byte, date, etc..) and entity types defined in your DbContext can be used in the query. Everything else needs to be broken apart into one of those forms.
EDIT:
You could also try coming from the other side of the query, but you are going to need a few things first..
The Model ...
[Table("MetaDataPair")]
public class DbMetaDataPair : IEntityComparable<DbMetaDataPair>
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public long Id { get; set; }
[Required]
public string Key { get; set; }
public string Value { get; set; }
// this is the navigation property back up to the ObjectChild
public virtual ObjectChild ObjectChild { get;set; }
}
public ObjectChild
{
...
public ICollection<MetaDataPair> MetaDataPairs { get; set; }
// this is the navigation property back up to the "Object"
public virtual Object Object { get; set; }
...
}
Now for the query...
public IEnumerable<Object> GetObjectsFromRequest(FindObjectRequest request)
{
foreach(var kvp in request.SearchFilters)
{
var key = kvp.Key;
var value = kvp.Value;
yield return metaDataRepository.MetaDataPairs
.Where(md => md.Key.ToLower() == key && md.Value.ToLower().Contains(value))
.Select(md => md.ObjectChild.Object)
}
}
This should execute 'n' number of SQL queries for the number of meta pairs you need to match. The better option would be trying to union them somehow but without some code to play with that might be a mission.
As a side note: I don't really know the names of your classes so I have used > > what I can work out. Obviously Object is not the name of the model..

Related

Many to Many in Linq using Dapper

I have Places, each place can have many tags. Each tag can be assigned to many places.
public class Place {
public int Id { get; set; }
public string PlaceName { get; set; }
public IEnumerable<Tag> Tags { get; set; }
}
public class Tag {
public int Id { get; set; }
public string TagName { get; set; }
}
public class TagPlace {
public int Id { get; set; }
public PlaceId { get; set; }
public TagId { get; set; }
}
The database has equivalent tables with foreign keys as appropriate.
I want to get a collection of Places, and I want each Place to have an appropriate colleciton of Tags. I guess using Linq might be required.
I've found various articles on this, but they aren't quite the same / deal with a list of ints rather than two collections of objects.
eg
https://social.msdn.microsoft.com/Forums/en-US/fda19d75-b2ac-4fb1-801b-4402d4bd5255/how-to-do-in-linq-quotselect-from-employee-where-id-in-101112quot?forum=linqprojectgeneral
LINQ Where in collection clause
What's the best way of doing this?
The classical approach with Dapper is to use a Dictionary to store the main objects while the query enumerates the records
public IEnumerable<Place> SelectPlaces()
{
string query = #"SELECT p.id, p.PlaceName, t.id, t.tagname
FROM Place p INNER JOIN TagPlace tp ON tp.PlaceId = p.Id
INNER JOIN Tag t ON tp.TagId = t.Id";
var result = default(IEnumerable<Place>);
Dictionary<int, Place> lookup = new Dictionary<int, Place>();
using (IDbConnection connection = GetOpenedConnection())
{
// Each record is passed to the delegate where p is an instance of
// Place and t is an instance of Tag, delegate should return the Place instance.
result = connection.Query<Place, Tag, Place(query, (p, t) =>
{
// Check if we have already stored the Place in the dictionary
if (!lookup.TryGetValue(p.Id, out Place placeFound))
{
// The dictionary doesnt have that Place
// Add it to the dictionary and
// set the variable where we will add the Tag
lookup.Add(p.Id, p);
placeFound = p;
// Probably it is better to initialize the IEnumerable
// directly in the class
placeFound.Tags = new List<Tag>();
}
// Add the tag to the current Place.
placeFound.Tags.Add(t);
return placeFound;
}, splitOn: "id");
// SplitOn is where we tell Dapper how to split the record returned
// in the two instances required, but here SplitOn
// is not really needed because "Id" is the default.
}
return result;
}

Entity.HasRequired returning items with NULL property

I have two related entities built and linked with Fluent API.
public class EDeal : IEntityBase
{
public int ID { get; set; }
public string Customer_id { get; set; }
public virtual ECustomer Customer { get; set; }
...etc
}
public class ECustomer : IEntityBase
{
public int ID { get; set; }
public string Customer_id { get; set; }
public string Customer_name { get; set; }
public virtual ICollection<EDeal> Deals { get; set; }
...etc
}
linked with
modelBuilder.Entity<ECustomer>().HasKey(c => c.Customer_id);
modelBuilder.Entity<EDeal>().HasRequired<ECustomer>(s => s.Customer)
.WithMany(r => r.Deals)
.HasForeignKey(s => s.Customer_id);
I recognize that this is inefficient linking but I had to link it in this way because I don't have control over the db structure.
The important thing to note is that the EDeal requires an ECustomer (.HasRequired). The database contains many rows in EDeal that have a null Customer_id field and I do not want to ever pull those lines when I query the entity.
I thought that the .HasRequired would make sure that I never got back any EDeals that do not have ECustomers associated with them but that doesn't seem to be the case. Instead, it only seems to ignore those lines with NULL Customer_id values when I try to order by a property in the Customer. And even then, returning the .Count() of the query behaves strangely.
var count1 = db.Set<EDeal>().Count(); //returns 1112
var count2 = db.Set<EDeal>().ToList().Count(); //returns 1112
var count3 = db.Set<EDeal>().OrderBy(c => c.Customer.Customer_name).Count(); //returns 1112
var count4 = db.Set<EDeal>().OrderBy(c => c.Customer.Customer_name).ToList().Count(); //returns 967
I know I can add a .Where(c => c.Customer.Customer_id != Null) to make sure I only get back what I'm looking for, but I'm hoping for a solution in the Entity's configuration because I have many generic functions acting on my IEntityBase class that build dynamic queries on generic Entities and I don't want to use a workaround for this case.
Questions:
1) Is there a way to limit the entity to only return those EDeals that have a corresponding ECustomer?
2) In my example above, why do count3 and count4 differ?
Thanks in advance.

Expression Tree Null VisitMember

I am converting an Expression<T, bool> to an Expression<Y, bool> where T and Y are different entities not related in any way other than through an Automapper mapping. Essentially, I have a Model object that my code uses:
public class Store
{
public string StoreId { get; set; }
public string Name { get; set; }
public List<Phone> Phones { get; set; }
public Address Address { get; set; }
public Account Account { get; set; }
public Status Status { get; set; }
}
that I am mapping to an entity object to store in my mongo database:
public class Store : MongoEntity
{
public string AccountId { get; set; }
public string Name { get; set; }
public List<string> UserIds { get; set; }
public List<Phone> PhoneNumbers { get; set; }
public Address Address { get; set; }
}
public abstract class MongoEntity : IMongoEntity
{
[BsonId]
public ObjectId Id { get; set; }
public Status Status { get; set; }
}
I used the answer in this question to work out how to convert between expressions (Question), and that got me 90% there. I was able to modify the code to grab the AutoMapper mappings between my Model store and my entity store and grab the destination property from from the source property:
private Expression<Func<TNewTarget, bool>> TransformPredicateLambda<TOldTarget, TNewTarget>(
Expression<Func<TOldTarget, bool>> predicate)
{
var lambda = (LambdaExpression)predicate;
if (lambda == null)
{
throw new NotSupportedException();
}
//Modified here to get automapper mappings
var maps = Mapper.FindTypeMapFor<TOldTarget, TNewTarget>();
var mutator = new ExpressionTargetTypeMutator(t => typeof(TNewTarget), maps);
var explorer = new ExpressionTreeExplorer();
var converted = mutator.Visit(predicate.Body);
return Expression.Lambda<Func<TNewTarget, bool>>(
converted,
lambda.Name,
lambda.TailCall,
explorer.Explore(converted).OfType<ParameterExpression>());
}
protected override Expression VisitMember(MemberExpression node)
{
var dataContractType = node.Member.ReflectedType;
var activeRecordType = _typeConverter(dataContractType);
PropertyMap prop = null;
foreach (var propertyMap in _maps)
{
var source = propertyMap.SourceMember;
var dest = propertyMap.DestinationProperty;
if (source != null && source.Name == node.Member.Name)
{
prop = propertyMap;
}
}
if (prop == null)
{
return base.VisitMember(node);
}
var propertyName = prop.DestinationProperty.Name;
var property = activeRecordType.GetProperty(propertyName);
var converted = Expression.MakeMemberAccess(
base.Visit(node.Expression),
property
);
return converted;
}
The problem is, my entity object doesn't have all of the same properties as my Model object (Account object versus AccountId for example). When the Transformer gets to the Account property on the Model object, I get an Exception (because there is no matching property on my Entity object). I cannot return null from VisitMember, and new Expression() is not allowed either. How can I handle ignoring properties on my Model object that do not exist on my Entity object?
Updating with info from Comments
So, to be a little more clear, I am using Automapper to map from a Models.Store to an Entity.Store. My entity.Store only has an AccountId (because I don't want to duplicate all of the account data), but my Models.Store needs the whole account object (which I would get by querying the Accounts collection).
Automapper is bascially converting my Account object to just an AccountId on my entity. Therefore, when I search for x => x.Account.AccountId == abcd1234 (where x is a models.Store), I need my expression to convert to x => x.AccountId == abcd1234 (where x is an Entity.Store).
I have that part working (changing mS => mS.Account.AccountId == 1234 to mE => mE.AccountId == 1234). The problem I am having now is that after doing the AccountId property, VisitMember is called with Account as the node. Since there is no Account in my Entity.Store object, I get the exception.
It's rather hard to test a solution without testable/runnable code. But here's a guess
Given the following expression mS => mS.Account.AccountId == 1234 and looking to transform MemberExpressions, you'll get the following calls:
VisitMember(mS.Account.AccountId
VisitMember(mS.Account)
You want to transform the second one into mE.AccountId. This involves two transformations: One, changing the property access from (EntityType).(AccountType).AccountId to (MongoStoreType).AccountId, and also changing the underlying object. If you're already handling the parameter transformations in other methods of your ExpressionVisitor, probably VisitParameter and VisitLambda, you'll be fine there. You then just need to skip looking at the parent MemberAccess, and jump straight to the grandparent:
var converted = Expression.MakeMemberAccess(
base.Visit(node.Expression),
property
);
return converted;
becomes something like this:
var parentMember = node.Expression as MemberExpression;
if (parentMember != null)
{
var grandparent = parentMember.Expression;
var converted = Expression.MakeMemberAccess(
base.Visit(grandparent),
property
);
return converted;
}
else
{
var converted = Expression.MakeMemberAccess(
base.Visit(node.Expression),
property
);
return converted;
}

Entity Framework - set model property value dynamically from a query

I have the following models:
public class A_DTO
{
[Key]
public string Id { get; set; }
**public virtual B_DTO B { get; set; }**
public virtual List<B_DTO> Bs { get; set; }
}
public class B_DTO
{
[Key]
public string Id { get; set; }
public string AId { get; set; }
public string UserId {get; set; }
[ForeignKey("AId"]
public virtual A_DTO A { get; set; }
[ForeignKey("UserId"]
public virtual User User { get; set; }
}
I am trying to get a list of object A_DTO but also including property B:
using AutoMapper.QueryableExtensions;
public IQueryable<A_DTO> GetAllA_DTO()
{
string userId = "8b6e9332-7c40-432e-ae95-0ac052904752";
return context.A_DTO
.Include("Bs")
.Include("B")
.Project().To<A_DTO>()
.Where(a => a.Bs.Any(b => b.UserId == userId));
}
How do I dynamically set this property according to set UserId and A_DTO.Id?
Here is a bag of observations in which you may be lucky enough to find your solution:
The B property in a code first model will result in there being a foreign key in the database table for A_DTOs that contains a reference to the B_DTOs table. Entity Framework will expect to own the responsibility for filling the B navigation property with an object populated with the data from the referenced row in the B_DTOs table, hence you would not be able to change it dynamically.
There is no need to use the Automapper Project method if your source type and destination type are the same. In your example they would both appear to be A_DTO. Are you sure you don't actually intend to have an entity "A" that is included in the context and "A_DTO" that is mapped from "A" via Automapper? If that is what you really want then you could have code in a .Select call mapping A.Bs.FirstOrDefault(b => b.UserId == userId) to A_DTO.B. However, you would not be able to apply filtering on the basis of the userId in an Automapper map.
Without seeing any of the Automapper Map setup code, it is difficult to get an idea of intent here.
As an aside, when using .Include it is better, in my opinion, to use the overload that takes an expression. In your case the includes would be rewritten:
.Include(a => a.B)
.Include(a => a.Bs)
Using this overload ensures that you will get compile time errors if you rename a property but fail to update the string in the .Include statement.

Order Collection by child child collection

I have the following objects
public class ObjectA{
public string Name { get; set; }
public ICollection<ObjectB> ObjectBCollection { get; set; }
}
public class ObjectB{
public string Name { get; set; }
public ICollection<ObjectC> ObjectCCollection { get; set; }
}
public class ObjectC{
public string Name { get; set; }
public DateTime Date { get; set; }
public InternalType Type { get; set; }
}
public enum InternalType {
TypeA,
TypeB,
TypeC
}
Now i want to order a List of ObjectA by the Dates in ObjectC that are closests to the current date. To make things a little more interesting, I also want it sorted by the InteralType. But I want TypeB have priority over TypeA and TypeC comes last.
I was thinking of creating an extra value that presents the integer value of the timespan between the current date and the Date property and multiply that by the Type property, but I can't figure out how to actually do that.
First of all if you want specific ordering in Enum you can do this:
public enum InternalType : int
{
TypeA = 2,
TypeB = 1,
TypeC = 3
}
Next if I understood your question correctly you have:
var collection = new List<ObjectA>();
which you need to sort by ALL dates in child elements. You can use Linq expressions for this:
List<KeyvaluePair<DateTime, ObjectA>> collectionWithDates = collection
.Select
(
objectA => new KeyValuePair<DateTime, ObjectA>
(
objectA
.SelectMany(a => a.ObjectBCollection)
.SelectMany(b => b.ObjectCCollection)
.OrderBy(c => c.Date).ThenBy(c => (int)c.Type)
.Last()
.Date,
objectA
)
)
.ToList();
To get ordered list of ObjectA you just need to:
var orderedCollection = collectionWithDates
.OrderBy(d => d.Key)
.Select(d => d.value)
.ToList();
I believe this should work. However I didn't tested it.
Correct me in comments if I misunderstood requirements.
Last thing to add - as far as I know, Linq expressions are not the fastest way to sort collections.

Categories