I have a question for you regarding the creation of a Dynamic select query in Entity Framework.
I already have a dynamic query for the select based on rights etc. But for each table I get 30+ fields that I have to parse via the .GetType().GetProperties().
Its complex and its quite costly in terms of resource due to the amount of data we have.
I have a service that tells me which fields I should select for each table. I would like to find a way to transform that into the query but I can't find something that is really dynamic.
That is not dynamic but manual:
using (var context = new StackOverflowContext())
{
var posts = context.Posts
.Where(p => p.Tags == "<sql-server>")
.Select(p => new {p.Id, p.Title});
// Do something;
}
I need to say, select only those fields but only the fields with this names.
I have the field list in a list of string but that could be changed.
Could you please help me?
Here is a .Net Fiddle code (made by msbendtsen) that allows to dynamically select columns (properties).
https://dotnetfiddle.net/3IMR1r
The sample is written for linq to objects but it should work with entity frameworks.
The key section is:
internal static IQueryable SelectProperties<T>(this IQueryable<T> queryable, IEnumerable<string> propertyNames)
{
// get propertyinfo's from original type
var properties = typeof(T).GetProperties().Where(p => propertyNames.Contains(p.Name));
// Create the x => expression
var lambdaParameterExpression = Expression.Parameter(typeof(T));
// Create the x.<propertyName>'s
var propertyExpressions = properties.Select(p => Expression.Property(lambdaParameterExpression, p));
// Creating anonymous type using dictionary of property name and property type
var anonymousType = AnonymousTypeUtils.CreateType(properties.ToDictionary(p => p.Name, p => p.PropertyType));
var anonymousTypeConstructor = anonymousType.GetConstructors().Single();
var anonymousTypeMembers = anonymousType.GetProperties().Cast<MemberInfo>().ToArray();
// Create the new {} expression using
var anonymousTypeNewExpression = Expression.New(anonymousTypeConstructor, propertyExpressions, anonymousTypeMembers);
var selectLambdaMethod = GetExpressionLambdaMethod(lambdaParameterExpression.Type, anonymousType);
var selectBodyLambdaParameters = new object[] { anonymousTypeNewExpression, new[] { lambdaParameterExpression } };
var selectBodyLambdaExpression = (LambdaExpression)selectLambdaMethod.Invoke(null, selectBodyLambdaParameters);
var selectMethod = GetQueryableSelectMethod(typeof(T), anonymousType);
var selectedQueryable = selectMethod.Invoke(null, new object[] { queryable, selectBodyLambdaExpression }) as IQueryable;
return selectedQueryable;
}
I have large yielded collection and I would like to get distinct values of each property independently:
IEnumerable<MyClass> collection = ...;
var prop1Values = collection.Select(i => i.Prop1).Distinct();
var prop2Values = collection.Select(i => i.Prop2).Distinct();
var prop3Values = collection.Select(i => i.Prop3).Distinct();
How to get it without enumerating the collection multiple times? Looking for most intuitive solution :)
You can try do it in a single foreach with a help of HashSet<T>s:
//TODO: put the right types for TypeOfProp1, TypeOfProp2, TypeOfProp3
var prop1Values = new HashSet<TypeOfProp1>();
var prop2Values = new HashSet<TypeOfProp2>();
var prop3Values = new HashSet<TypeOfProp3>();
foreach (var item in collection) {
prop1Values.Add(item.Prop1);
prop2Values.Add(item.Prop2);
prop3Values.Add(item.Prop3);
}
I am using the Linq.Dynamic Library and EF6. I am attempting to select only the fields from a database that my user selects.
However, all that is returned is a List<object>, I have attempted to Cast<dynamic>, and every way I can think of, but no matter what the object has 0 fields.
I have also tried explicitly declaring as an IEnumerable and that too was unsuccessful, and was unable to call .ToList(), without first calling Cast<T> which too was unsuccessful.
When converting one of the objects to string I get: "{filenumber=345400, custom2=, custom3=, custom6=4076995332, custom8=4072121417}".
The data is being returned I simply cannot cast it to the appropriate type.
var query = cmax.dbases
.Where(w => statuses.Any(a => w.statusname == a) && portfolios.Any(a => w.portfolio == a))
.Select(string.Format("new ({0})", string.Join(",", fields)))
.Take(Math.Min((int) takeAmount, count - taken));
var take = await query.ToListAsync();
take.ForEach(data => {
var type = take.GetType();
var properties = type.GetProperties();
var propCount = properties.Count();
properties.ToList().ForEach(prop => {
var name = prop.Name;
});
});
In one of my use cases I have converted the results to List<string[]> through an extension for IQueryable. As I know my column order in Select("New (columnNames...) "), I could easily figure out which one is which.
so here is the code of the extension
public static IList<string[]> ToStringArray(this IQueryable queryFinal)
{
var query = (IQueryable<dynamic>)queryFinal;
IList<dynamic> data = query.ToList();
System.Reflection.PropertyInfo[] props = null;
if (data.Count > 0) props = data[0].GetType().GetProperties();
if (props == null) return new List<string[]>();
/*I do other things using reflection here*/
return data.Select(d => props.Select(p => (p.GetValue(d, null) ?? string.Empty).ToString()).OfType<string>().ToArray()).ToList();
}
use it as
var result = query.ToStringArray();
The only viable solution I found was to manually parse the data in the string:
take.ForEach(data =>
{
var dbtrData = new DebtorData();
var cleaned = data.ToString().Replace("{", "").Replace("}", "");
var pairs = cleaned.Split(',');
pairs.ToList().ForEach(pair =>
{
var key = pair.Split('=')[0].Replace(" ", "");
var value = pair.Split('=')[1];
//USEDATA
}
});
I am trying to create a generic repository to access my database. In EF6 I was able to do that in order to get a specific entity:
protected IDbSet<T> dbset;
public T Get(object id)
{
return this.dbset.Find(id);
}
DbSet in EF7 is missing a Find method. Is there a way to implement the above piece of code?
Here's a very crude, incomplete, and untested implementation of .Find() as an extension method. If nothing else, it should get you pointed in the right direction.
The real implementation is tracked by #797.
static TEntity Find<TEntity>(this DbSet<TEntity> set, params object[] keyValues)
where TEntity : class
{
var context = ((IAccessor<IServiceProvider>)set).Service.GetService<DbContext>();
var entityType = context.Model.GetEntityType(typeof(TEntity));
var key = entityType.GetPrimaryKey();
var entries = context.ChangeTracker.Entries<TEntity>();
var i = 0;
foreach (var property in key.Properties)
{
var keyValue = keyValues[i];
entries = entries.Where(e => e.Property(property.Name).CurrentValue == keyValue);
i++;
}
var entry = entries.FirstOrDefault();
if (entry != null)
{
// Return the local object if it exists.
return entry.Entity;
}
// TODO: Build the real LINQ Expression
// set.Where(x => x.Id == keyValues[0]);
var parameter = Expression.Parameter(typeof(TEntity), "x");
var query = set.Where((Expression<Func<TEntity, bool>>)
Expression.Lambda(
Expression.Equal(
Expression.Property(parameter, "Id"),
Expression.Constant(keyValues[0])),
parameter));
// Look in the database
return query.FirstOrDefault();
}
In case you are using EF 7.0.0-rc1-final, below you find a small update for the code presented by #bricelam in the previous answer. By the way, thank you very much #bricelam - your code was extremely useful for me.
Here are my dependencies under "project.config":
"dependencies": {
"EntityFramework.Commands": "7.0.0-rc1-final",
"EntityFramework.MicrosoftSqlServer": "7.0.0-rc1-final",
"Microsoft.Framework.Configuration.Json": "1.0.0-beta8",
"Microsoft.Framework.ConfigurationModel": "1.0.0-beta4",
"Microsoft.Framework.ConfigurationModel.Json": "1.0.0-beta4",
"Microsoft.Framework.DependencyInjection": "1.0.0-beta8"
}
And below is the extension method for DbSet.Find(TEntity):
using Microsoft.Data.Entity;
using Microsoft.Data.Entity.Infrastructure;
using System;
using System.Linq;
using System.Linq.Expressions;
namespace Microsoft.Data.Entity.Extensions
{
public static class Extensions
{
public static TEntity Find<TEntity>(this DbSet<TEntity> set, params object[] keyValues) where TEntity : class
{
var context = ((IInfrastructure<IServiceProvider>)set).GetService<DbContext>();
var entityType = context.Model.FindEntityType(typeof(TEntity));
var key = entityType.FindPrimaryKey();
var entries = context.ChangeTracker.Entries<TEntity>();
var i = 0;
foreach (var property in key.Properties)
{
entries = entries.Where(e => e.Property(property.Name).CurrentValue == keyValues[i]);
i++;
}
var entry = entries.FirstOrDefault();
if (entry != null)
{
// Return the local object if it exists.
return entry.Entity;
}
// TODO: Build the real LINQ Expression
// set.Where(x => x.Id == keyValues[0]);
var parameter = Expression.Parameter(typeof(TEntity), "x");
var query = set.Where((Expression<Func<TEntity, bool>>)
Expression.Lambda(
Expression.Equal(
Expression.Property(parameter, "Id"),
Expression.Constant(keyValues[0])),
parameter));
// Look in the database
return query.FirstOrDefault();
}
}
}
Can't comment because of reputation, but if you use RC2 (or later?) you should use
var context = set.GetService<ICurrentDbContext>().Context;
instead of
var context = set.GetService<DbContext>();
I've taken some of the previously provided answers and tweaked them to fix a couple of problems:
Implicitly captured closure
Key shouldn't be hard coded to "Id"
public static TEntity Find<TEntity>(this DbSet<TEntity> set, params object[] keyValues) where TEntity : class
{
var context = set.GetService<DbContext>();
var entityType = context.Model.FindEntityType(typeof(TEntity));
var key = entityType.FindPrimaryKey();
var entries = context.ChangeTracker.Entries<TEntity>();
var i = 0;
foreach (var property in key.Properties)
{
var i1 = i;
entries = entries.Where(e => e.Property(property.Name).CurrentValue == keyValues[i1]);
i++;
}
var entry = entries.FirstOrDefault();
if (entry != null)
{
// Return the local object if it exists.
return entry.Entity;
}
var parameter = Expression.Parameter(typeof(TEntity), "x");
var query = set.AsQueryable();
i = 0;
foreach (var property in key.Properties)
{
var i1 = i;
query = query.Where((Expression<Func<TEntity, bool>>)
Expression.Lambda(
Expression.Equal(
Expression.Property(parameter, property.Name),
Expression.Constant(keyValues[i1])),
parameter));
i++;
}
// Look in the database
return query.FirstOrDefault();
}
Find finally arrives into Entity Framework core.
So...the above find methods worked great, but if you don't have a column named "Id" in your model, the whole thing is going to fail on the following line. I'm not sure why the OP would have put a hardcoded value into this spot
Expression.Property(parameter, "Id"),
Here's a revision that will fix it for those that name our Id columns appropriately. :)
var keyCompare = key.Properties[0].Name;
// TODO: Build the real LINQ Expression
// set.Where(x => x.Id == keyValues[0]);
var parameter = Expression.Parameter(typeof(TEntity), "x");
var query = set.Where((Expression<Func<TEntity, bool>>)
Expression.Lambda(
Expression.Equal(
Expression.Property(parameter, keyCompare),
//Expression.Property(parameter, "Id"),
Expression.Constant(keyValues[0])),
parameter));
// Look in the database
return query.FirstOrDefault();
}
This STILL very well could fail if you have more than one Key setup on the entity object and the key you're looking up by isn't the first, but it should be quite a bit btter this way.
Not enough reputation to comment, but there is a bug in #Roger-Santana answer when using it in a console app/seperate assembly:
var i = 0;
foreach (var property in key.Properties)
{
entries = entries.Where(e => e.Property(property.Name).CurrentValue == keyValues[i]);
i++;
}
var entry = entries.FirstOrDefault();
The value of 'i' is captured in the foreach so that when entries.FirstOrDefault() is called, keyValues[i] has the value of (at least) keyValues[i++], which in my case crashed with an out of index error.
A fix would be to copy the value of 'i' through the loop:
var i = 0;
foreach (var property in key.Properties)
{
var idx =i;
entries = entries.Where(e => e.Property(property.Name).CurrentValue == keyValues[idx]);
i++;
}
var entry = entries.FirstOrDefault();
I use linq; instead of Find method you can use:
var record = dbSet.SingleOrDefault(m => m.Id == id)
Let me contribute a revision that includes building the expression.
I'll confess I didn't actually test this ;-)
public static TEntity Find<TEntity>(this DbSet<TEntity> dbSet, params object[] keyValues) where TEntity : class
{
// Find DbContext, entity type, and primary key.
var context = ((IInfrastructure<IServiceProvider>)dbSet).GetService<DbContext>();
var entityType = context.Model.FindEntityType(typeof(TEntity));
var key = entityType.FindPrimaryKey();
// Build the lambda expression for the query: (TEntity entity) => AND( entity.keyProperty[i] == keyValues[i])
var entityParameter = Expression.Parameter(typeof(TEntity), "entity");
Expression whereClause = Expression.Constant(true, typeof(bool));
uint i = 0;
foreach (var keyProperty in key.Properties) {
var keyMatch = Expression.Equal(
Expression.Property(entityParameter, keyProperty.Name),
Expression.Constant(keyValues[i++])
);
whereClause = Expression.And(whereClause, keyMatch);
}
var lambdaExpression = (Expression<Func<TEntity,bool>>)Expression.Lambda(whereClause, entityParameter);
// Execute against the in-memory entities, which we get from ChangeTracker (but not filtering the state of the entities).
var entries = context.ChangeTracker.Entries<TEntity>().Select((EntityEntry e) => (TEntity)e.Entity);
TEntity entity = entries.AsQueryable().Where(lambdaExpression).First(); // First is what triggers the query execution.
// If found in memory then we're done.
if (entity != null) { return entity; }
// Otherwise execute the query against the database.
return dbSet.Where(lambdaExpression).First();
}
here is what I use.
Not a find method, but works like a charm
var professionalf = from m in _context.Professionals select m;
professionalf = professionalf.Where(s => s.ProfessionalId == id);
Professional professional = professionalf.First();
An edit was proposed to change ".First()" to ".FirstOrDefault()" in the very last line of my earlier post. The edit was voted down, but I agree with it. I would expect the function to return null if the key was not found. I would not want it to throw an exception. In most cases I would want to know if the key existed in the set, and handling an exception is a very slow way of figuring that out.
I got a problem with Moq and Automapper regarding IDataReader.
I found an example on stackoverflow and modified the code.
public static IDataReader MockIDataReader<T>(List<T> ojectsToEmulate) where T : class
{
var moq = new Mock<IDataReader>();
// This var stores current position in 'ojectsToEmulate' list
var count = 0;
moq.Setup(x => x.Read())
// Return 'True' while list still has an item
.Returns(() => count < ojectsToEmulate.Count)
// Go to next position
.Callback(() => count++);
var properties = typeof (T).GetProperties();
foreach (PropertyInfo t in properties)
{
var propName = t.Name;
moq.Setup(x => x[propName]).Returns(() => ojectsToEmulate[count].GetType().GetProperty(propName).GetValue(ojectsToEmulate[count],null));
}
return moq.Object;
}
}
My mapping:
Mapper.Configuration.CreateMap(typeof(IDataReader), typeof(IEnumerable<T>));
var result = Mapper.Map<IDataReader, IEnumerable<T>>(reader);
The problem I got here is that my result has 1 result a cityModel but all it's properties are null. If I check the value from my mocked reader like reader["name"] I got the "Alingsås" value so the mocking is correct but Automapper seams to have the problem.
I use a List of objects that I pass to my method that mocks it all.
var cityModel = new CityModel();
cityModel.Name = "Alingsås";
cityModel.Id = "SE";
cityModel.CountryId = "SE";
var cityModels = new List<CityModel>();
cityModels.Add(cityModel);
_fakeReader = MockTester.MockIDataReader(cityModels);
The code works fine, no exception is thrown, but the mapper gives me an object without the
valules. I can see in the debugger my reflection code works but it seams like my
x["Name"] aren't the method Automapper call from IDataReader? Or is it?
What can be wrong here?
Automapper internally uses the int indexer of IDataReader so you need to call Setup on that instead of Item[String].
Checking the Automapper's source you need to setup some additional methods to make it work:
//...
var index = 0;
foreach (PropertyInfo t in properties)
{
var propName = t.Name;
int index1 = index; // avoid access to modified closure
moq.Setup(x => x.GetFieldType(index1)).Returns(t.PropertyType);
moq.Setup(x => x.GetName(index1)).Returns(propName);
moq.Setup(x => x[index1])
.Returns(ojectsToEmulate[count]
.GetType()
.GetProperty(propName).GetValue(ojectsToEmulate[count], null));
index++;
}
moq.Setup(x => x.FieldCount).Returns(properties.Length);
//...