We Keep Coding

Generating Dynamic Linq For Jquery DataTables

I'm attempting to build an adapter to make using JQuery Datables with Entity Framework considerably simpler. The adapter is very similar to the one built by Telerik in their Kendo UI Extensions. I've managed to get most of the logic working, but the final piece that's giving me some trouble is getting the dynamically generated Linq to work.
I've looked into both LinqKit and Dynamic Expressions, and I'm a bit torn on how to approach this. I'm currently using the Dynamic Linq extension with limited success. It appears to work fine with Varchar and Int fields, but it stumbles with dates.
I'm using it like so:
public class Search
{
[DataMember(Name = "value")]
public string Value { get; set; }
[DataMember(Name = "regex")]
public string Regex { get; set; }
public string ToExpression(IList<Column> columns)
{
var list = columns.Select(column => $"{column.Data}.Contains.(#{columns.IndexOf(column)})").ToList();
return string.Join("or", list);
}
}
public class FilterExpression
{
public string Filter { get; }
public IEnumerable<object> Values { get; }
public FilterExpression(IEnumerable<Column> columns, Search search)
{
if (search.Value == null) return;
var list = columns.Where(n => n.Searchable).ToList();
Filter = ToExpression(list);
Values = list.Select(n => search.Value);
}
private static string ToExpression(IEnumerable<Column> columns)
{
var colList = columns.Where(n => n.Searchable).ToList();
var list = colList.Select(column => $"{column.Data}.ToString().Contains(#{colList.IndexOf(column)})").ToList();
return string.Join(" or ", list);
}
}
public static IQueryable Where(this IQueryable source, FilterExpression filterExpression)
{
return filterExpression?.Values == null ? source : source.Where(filterExpression.Filter, filterExpression.Values.ToArray());
}
The above works for most cases, but again Dates are a bit of a problem. The goal is to prevent developers from having to manually write their Linq Where statements and instead allow the adapter to simply generate it.
Again, the above works for both INT fields and VARCHAR, but if I include a DATE field in the model, the error I get is: System.NotSupportedException: 'LINQ to Entities does not recognize the method 'System.String ToString()' method, and this method cannot be translated into a store expression.'

https://www.c-sharpcorner.com/blogs/exception-linq-to-entities-does-not-recognize-the-method-tostring
Check this one out.
Bascially when you work with linq to entites, it creats a query which is fired on sql server. ToString() works with in memory objects.
If the object was in memory ,it would have worked just fine. Please go through the link.

Fluent NHibernate mapping for DateTime with default value

I know that there are many threads about this and I have read most of them. However for me a couple of things remain unclear and still do not work.
If I have on my database schema a field of type DateTime and I like to assign it a default value I would do something like this:
create table [mySchema].[MyTable](
ObjectGuid uniqueidentifier CONSTRAINT Id PRIMARY KEY,
SomeTextToStore nvarchar(128) NULL,
CDate datetime NOT NULL DEFAULT GETDATE(),
CUser nvarchar(64) DEFAULT CURRENT_USER
);
GO
(Don't know if it is important: I am using SQL Server Express 2014. Fluent configuration is for SQL Server 2012.)
This works fine when doing an INSERT from ISQL, inserts a timestamp of the moment when the record was added.
Using fluent I would write something like this:
Domain:
public class MyObject
{
public virtual Guid Id {get; set}
public virtual string SomeTextToStore {get; set;}
public virtual DateTime? CDate {get; set;}
public virtual string CUser {get; set;}
}
NOTE: I made CDate nullable!
And a mapping class like this:
class MyObjectMap : ClassMap<MyObject>
{
public MyObjectMap()
{
Table("MySchema.MyTable");
Id(x => x.Id).GeneratedBy.GuidComb();
Map(x => x.SomeTextToStore).Length(128).Nullable();
Map(x => x.CDate).Not.Nullable().Default("getdate()");
Map(x => x.CUser).Not.Nullable().Default("CURRENT_USER);
}
}
In the program (in my case this is a library that can be called from several type of programs) I do something like:
public void EnterSomeText()
{
using (var session = sessionManager.OpenSession())
{
using (var transaction = session.BeginTransaction())
{
var myObj = new MyObject();
myObj.SomeTextToStore("bla bla bla");
session.SaveOrUpdate(myObj);
transaction.Commit();
}
session.Close();
}
}
This ends always in a DateTime overflow exception! (SqlDateTime overflow. Must be between 1/1/1753 12:00:00 AM and 12/31/9999 11:59:59 PM)
It looks like not passing a value to the file CDate is causing the problem. When I add the default in my library like such it works:
...
myObj.SomeTextToStore("bla bla bla");
myObj.CDate = DateTime.Now; // <---- Set the date here
session.SaveOrUpdate(myObj);
...
But this is not really the solution....
Questions:
What am I doing wrong / missing ?
What is the correct strategy when
defining defaults? Doing it on the database or just in code? (When
starting a plain new vanilla project, I would prefer doing
everything from C# code, even create and later update the schema...)
Regarding domain classes: Is it wise to create constructors,
that fill fields with defaults?
I do this for the field CUser because in CUser I like to add the current user context
public MyObject()
{
var o = System.Security.Principal.WindowsIdentity.GetCurrent();
if (o != null)
{
CUser = o.Name;
}
}
Instead of filling the CDate field with the current date in my DB-access layer library I could do it also in the constructor of the domain-class like such:
public MyObject()
{
var o = System.Security.Principal.WindowsIdentity.GetCurrent();
if (o != null)
{
CUser = o.Name;
}
CDate = DateTime.Now;
}
Many thanks for your help and comments!

Usually for this type of mapping I do the following in my mapping:
DynamicInsert();
DynamicUpdate();
This way if you have nullable types in C# and you don't set them to anything nhibernate will not include them in the insert or update statement. I never really like it when nhibernate is updating columns that weren't changed in the code anyway.
Furthermore when you specify .Not.Nullable(); and .Default("getdate()") in the mapping all this is used for is schema generation. It's not actually used by nhibernate to do any validation or defaulting. That is left up to the database.

You have defined this column as NOT NULL in the database and Nullable() in the mapping. FluentNHibernate is sending DbNull to the database, which is getting rejected, since the database column is not nullable.
You should decide where your logic resides. Is this your code or the database who is the master?
Setting default value in FlientNHibernate was discussed, for example, in this question.
Suggested code from there is:
Map(x => x.SubmitionDate).Default("getdate()").Not.Nullable();
In general, I would always advise to use NHibernate Profiler (paid tool) to see what queries go to the database and why they fail. Invaluable for optimisation, you are using ORM, be careful now!

Limit collection to retrieve only recent entries for readonly entity

The User entity can have thousands of UserOperations. Sometimes I don't want to retrieve (for readonly entity) all of them but only "the recent 10 OR not completed".
public class SimpleForm
{
public class User : EntityBase
{
// ...
private ISet<UserOperation> _recentOperations = new HashedSet<UserOperation>();
public virtual ISet<UserOperation> RecentOperations { get { return _recentOperations; } set { _recentOperations = value; } }
}
}
So how can I specify it? I think I could use mapping overrides?
I understand I could make this with a seperate query but can it be done by entity mapping?
Also I wonder if there is a possibility to do the some for non-readonly entity where I can modify the collection of operations?
UPDATE
I tried to use
DateTime dateTime = (DateTime.UtcNow - TimeSpan.FromDays(15));
mapping.HasMany(x => x.RecentOperations)
.Where(x => x.EndedAt == null || x.EndedAt < dateTime);
but it says "Unable to convert expression to SQL".
I replaced it with
mapping.HasMany(x => x.RecentOperations)
.Where(x => x.EndedAt == null);
and now it throws null reference exception inside
в FluentNHibernate.Utils.ExpressionToSql.Convert(Object value)
в FluentNHibernate.Utils.ExpressionToSql.Convert(ConstantExpression expression)
в FluentNHibernate.Utils.ExpressionToSql.Convert[T](Expression`1 expression, UnaryExpression body)

There are 2 general ways how to filter mapped collections.
The first is a bit rigid, fixed, in a mapping defined where="" clause:
6.2. Mapping a Collection (...in fluent .Where(bool expr) or .Where(Sql statement string)
The second and maybe really suitable in this scenario, is dynamic version called filter:
18.1. NHibernate filters
NHibernate adds the ability to pre-define filter criteria and attach those filters at both a class and a collection level. A filter criteria is the ability to define a restriction clause very similiar to the existing "where" attribute available on the class and various collection elements. Except these filter conditions can be parameterized. The application can then make the decision at runtime whether given filters should be enabled and what their parameter values should be. Filters can be used like database views, but parameterized inside the application....
The implementation in fluent would look like this:
public class RecentFilter : FilterDefinition
{
public RecentFilter()
{
WithName("RecentFilter")
.WithCondition("( :EndedAtDate IS NULL OR EndedAt < :EndedAtDate )")
.AddParameter("EndedAtDate",NHibernate.NHibernateUtil.DateTime);
}
}
this is the filter, and here is its usage in a fluent mapping:
mapping
.HasMany(x => x.RecentOperations)
...
.ApplyFilter<RecentFilter>();
In runtime, we can turn filter on/off on the ISession level:
session.EnableFilter("RecentFilter")
.SetParameter("EndedAtDate",DateTime.Now.AddDays(-15));
See also:
property filter with fluent nHibernate automapping
Syntax to define a NHibernate Filter with Fluent Nhibernate?
Is it possible to use NHibernate Filters to filter through references?

Calculated field in model class gives exception

I'm getting the exception when executing Linq query:
The specified type member 'Active' is not supported in LINQ to
Entities. Only initializers, entity members, and entity navigation
properties are supported.
Model class
public class User : Entity
{
//Many properties skipped
[NotMapped]
public bool Active
{
get
{
return Orders.Any(c =>
c.Active && (c.TransactionType == TransactionType.Order ||
c.TransactionType == TransactionType.Subscription));
}
}
}
The Linq query which gives exception
public IEnumerable<User> GetInactiveUsersForSuspensionNotification()
{
return _userRepository.GetAll()
.Include(i=>i.Orders)
.Where(w => w.Active == false);
}
Orders is a related table to Users.

When using LINQ to Entities the LINQ expression is converted to a SQL query to be sent to the database. This is so that the whole table is not pulled into memory.
Your problem is that because Active is not mapped the database knows nothing about it and therefore cannot make the calculation.
You will either need to move Active into your DB or change the LINQ statement to only query on columns that are in your DB

Entity Framework 5 wrong data type in query

We are using EF 5.0 as our ORM of choice in our business solution, structured in a n-layer fashion with everything decoupled and a nice composition root with ninject.
Lately, we've been building a database that uses partitioning underneath, and we have some important indexes on DATE columns.
The columns are correctly declared on Sql Server 2008. We also added the correct data type in the EF mappings, with the HasColumnType("Date") instruction.
Still, when querying the table through Linq to Entities, the parameters that we filter the dates on are created of type DateTime2 and even the columns are cast to DateTime2 in the queries so the type matches the parameters.
This behaviour has several problems. First of all, if I'm telling EF engine that the column on the database is DATE why should it cast it to DateTime2?
Second, this cast is making the database ignore the indexes, thus not using partitioning. We have one year per phisical partitioning, and if I ask a date range, let's say, february 2013 to march 2013 the scan should happend only on one physical partition. It works correctly if manually using the correct data type DATE but with the cast to DateTime2 all the partitions get scanned, reducing performance drastically.
Now, I'm sure I'm missing out something, because it would be rather stupid that Microsoft ORM doesn't work well on Microsoft Sql Server.
I've been unable to find any documentation on how have EF use the correct data types in queries, so I'm asking here. Any help will be appreciated.
Thanks.

I don't believe that this is possible in Entity Framework. This requested enhancement would probably do what you need. This MSDN page shows the mapping between SQL Server types and CLR types. Note that date is supported and is mapped to DateTime, but since several SQL types map to the same CLR type, EF is evidently picking one SQL type as the preferred eqivalent of the CLR type.
Could you wrap your selection code in a stored procedure? If so, this would seem to be a reasonable solution. You could use DbSet{T}.SqlQuery to materialize objects from executing the sp.
Code sample
The following short console application demonstrates the concept. Note how the related entities are successfully lazy-loaded.
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.ComponentModel.DataAnnotations;
using System.ComponentModel.DataAnnotations.Schema;
using System.Data.Entity;
using System.Data.SqlClient;
using System.Linq;
namespace ConsoleApplication1
{
[Table("MyEntity")]
public class MyEntity
{
private Collection<MyRelatedEntity> relatedEntities;
[Key]
public virtual int MyEntityId { get; set; }
[DataType(DataType.Date)]
public virtual DateTime MyDate { get; set; }
[InverseProperty("MyEntity")]
public virtual ICollection<MyRelatedEntity> RelatedEntities
{
get
{
if (this.relatedEntities == null)
{
this.relatedEntities = new Collection<MyRelatedEntity>();
}
return this.relatedEntities;
}
}
public override string ToString()
{
return string.Format("Date: {0}; Related: {1}", this.MyDate, string.Join(", ", this.RelatedEntities.Select(q => q.SomeString).ToArray()));
}
}
public class MyRelatedEntity
{
[Key]
public virtual int MyRelatedEntityId { get; set; }
public virtual int MyEntityId { get; set; }
[ForeignKey("MyEntityId")]
public virtual MyEntity MyEntity { get; set; }
public virtual string SomeString { get;set;}
}
public class MyContext : DbContext
{
public DbSet<MyEntity> MyEntities
{
get { return this.Set<MyEntity>(); }
}
}
class Program
{
const string SqlQuery = #"DECLARE #date date; SET #date = #dateIn; SELECT * FROM MyEntity WHERE MyDate > #date";
static void Main(string[] args)
{
Database.SetInitializer(new DropCreateDatabaseAlways<MyContext>());
using (MyContext context = new MyContext())
{
context.MyEntities.Add(new MyEntity
{
MyDate = DateTime.Today.AddDays(-2),
RelatedEntities =
{
new MyRelatedEntity { SomeString = "Fish" },
new MyRelatedEntity { SomeString = "Haddock" }
}
});
context.MyEntities.Add(new MyEntity
{
MyDate = DateTime.Today.AddDays(1),
RelatedEntities =
{
new MyRelatedEntity { SomeString = "Sheep" },
new MyRelatedEntity { SomeString = "Cow" }
}
});
context.SaveChanges();
}
using (MyContext context = new MyContext())
{
IEnumerable<MyEntity> matches = context.MyEntities.SqlQuery(
SqlQuery,
new SqlParameter("#dateIn", DateTime.Today)).ToList();
// The implicit ToString method call here invokes lazy-loading of the related entities.
Console.WriteLine("Count: {0}; First: {1}.", matches.Count(), matches.First().ToString());
}
Console.Read();
}
}
}

The range of DateTime type in .NET and SQL server is different.
.NET DateTime range is : 0000-Jan-01 to 9999-Dec-31
SQL DateTime range is: 1900-Jan-01, 2079-Jun-06
To match the range, EF convert your .NET DateTime to SQL server DateTime2 type which has same range as .NET DateTime range.
I think your issue only happens when you have date property that is not assigned and passed to SQL server via EF. When the date is not assigned with specific value, it is default to DateTime.Min which is 0000-Jan-01 and that is causing the conversion to DateTime2.
I think you can either make your DateTime property nullable --> DateTime? or write a helper to convert your DateTime.Min to meet SQL DateTime range.
Hopefully, this helps.

I don't have a solution. I've never seen a LINQ-to-Entites query with .NET DateTime parameters involved that had used a parameter type in the SQL query other than datetime2(7). I doubt that you can get rid of that. Just a try to explain why it is as it is:
Suppose you have an entity with a property SomeNumber of type int. What result would you expect for a query like this:
....Where(e => e.SomeNumber >= 7.3)....
Probably all entities where SomeNumber is 8 or greater. If the (floating point decimal) parameter 7.3 would be cast to the type int stored in the database you had to decide how to round 7.3 - to 7 (will lead to wrong result) or to 8? OK, you could say, because my query says >= and I know the type in the DB is a whole number, rounding to 8 must be correct. If I would use <=, then rounding to 7 must be correct. If I would use ==, oh... I must not round at all or I know that the result must be empty and I could directly translate this Where clause to false. And != to true. But a parameter of 7.0 is a special case. Etc....
Well, the dilemma in this example has an easy solution: Decide on client side what you want by using an int parameter (7 or 8) in the first place.
The solution with DateTime is not so simple because .NET does not have a Date type. Queries with DateTime parameters will always have the form...
DateTime dateTime = new DateTime(2013, 5, 13, 10, 30, 0);
....Where(e => e.SomeDateTime >= dateTime)....
...and if SomeDateTime is stored as date in SQL Server you have again the rounding dilemma. Do I have to cast to 2013.05.13 or 2013.05.14? For the query above the client would surely expect all entities with a date of 14th and later.
Well, you could do it smart, like: if the time portion of my DateTime parameter is midnight, cast to the date portion. If I use >= cast to the next day, etc., etc.... Or you could always cast to datetime2(7). Then the result of the query is always correct and as the (.NET) client expects it. Correct... but perhaps with suboptimal index usage.