I have a method query like this:
public IList<BusinessObject> GetBusinessObject(Guid? filterId)
{
using (var db = new L2SDataContext())
{
var result = from bo in db.BusinessObjects
where (filterId.HasValue)
? bo.Filter == filterId.value
: true
orderby bo.Name
select SqlModelConverters.ConvertBusinessObject(bo);
return result.ToList();
}
}
At runtime, this throws a System.InvalidOperationException: Nullable object must have a value.
Looking at the Debugger, the problem is my Where Clause: Linq To SQL tries to convert the entire thing to SQL, so even if filterId is NULL, it will still try to access filterId.value.
I thought/hoped the C# compiler/CLR would evaluate that where clause as a code block and only send one of the two branches to Linq To SQL, but that's not how it works.
My refactored version works, but is not very elegant:
public IList<BusinessObject> GetBusinessObject(Guid? filterId)
{
using (var db = new L2SDataContext())
{
var temp = from bo in db.BusinessObjects select bo;
if(filterId.HasValue) temp = temp.Where(t => t.Filter == filterId.Value);
var result = from t in temp
orderby t.Name
select SqlModelConverters.ConvertBusinessObject(bo);
return result.ToList();
}
}
I know that Lazy-Evaluation will make sure that only one query is really sent, but having that temp object in there isn't that great really.
Did you try:
where filterId == null || t.Filter == filterId
Your fix is exactly correct. You are effectively trying to build up a query dynamically, based on your function input. It's a good idea to omit the where clause instead of supplying WHERE TRUE anyway. If I were writing this query, I would go with your fixed version myself.
It's not as pretty as using the language keywords, but it's still the right way to approach the query in my opinion.
Related
Are these two blocks exactly the same? Does one have any advantage over the other?
using (var context = new TRANSITEntities())
{
var result = context.Table1.Where(c => c.UserCode == "123");
}
using (var context = new TRANSITEntities())
{
var result = from c in context.Table1
where c.UserCode == "123"
select c;
}
Exactly the same.
You can verify this yourself by looking at the ToString()
string query1String, query2String;
using (var context = new TRANSITEntities())
{
var result = context.Table1.Where(c => c.UserCode == "123");
query1String = result.Expression.ToString();
}
using (var context = new TRANSITEntities())
{
var result = from c in context.Table1
where c.UserCode == "123"
select c;
var query2String = result.Expression.ToString();
}
Assert.AreEqual(query1String, query2String);
It should also be noted that result is NOT actually a result. It's a non-executed/enumerated IQueryable. Meaning it's like a SQL statement that hasn't been run yet (sort of). If you were to do
var t1 = result.ToArray()
var t2 = result.ToArray()
then the query would actually be executed twice. t1 and t2 are the REAL results (an in memory array)...not result. In other words, result should really be named query And further, the example you have above will never work...because if you call ToArray on the result outside of the using block, it will fail...because you can't run the query once the context is disposed:
using (var context = new TRANSITEntities())
{
var result = context.Table1.Where(c => c.UserCode == "123");
}
// throws exception:
var array = result.ToArray();
They're exactly the same. The compiler converts the query syntax to the lambda expression version before it resolves any methods.
Well the first selects from context.Table1, the second selects from context.LINTE. So they select from totally different collections.
Also the first selects on a UserCode column and the second on a CodeUsager column..
Apart from that (is the first just a translation of the second and you forgot to translate the second) the code is the same. The C# compiler will just translate the second query into the first one and they will behave identically.
If you are trying to select from different tables then it is different.
But if the tables are same then it is just the representation different.
There is no difference it is just the preference which way you like to code.
All the linq queries are converted to Lambda before creating the sql representation.
From Julia Lermans' book Programming Entity Framework:
The MSDN documentation says, “In general, we recommend query syntax
because it is usually simpler and more readable; however, there is no
semantic difference between method syntax and query syntax.”*
Therefore, using one over the other is a matter of style and personal
choice.
I'm migrating some stuff from one mysql server to a sql server but i can't figure out how to make this code work:
using (var context = new Context())
{
...
foreach (var item in collection)
{
IQueryable<entity> pages = from p in context.pages
where p.Serial == item.Key.ToString()
select p;
foreach (var page in pages)
{
DataManager.AddPageToDocument(page, item.Value);
}
}
Console.WriteLine("Done!");
Console.Read();
}
When it enters into the second foreach (var page in pages) it throws an exception saying:
LINQ to Entities does not recognize the method 'System.String
ToString()' method, and this method cannot be translated into a store
expression.
Anyone know why this happens?
Just save the string to a temp variable and then use that in your expression:
var strItem = item.Key.ToString();
IQueryable<entity> pages = from p in context.pages
where p.Serial == strItem
select p;
The problem arises because ToString() isn't really executed, it is turned into a MethodGroup and then parsed and translated to SQL. Since there is no ToString() equivalent, the expression fails.
Note:
Make sure you also check out Alex's answer regarding the SqlFunctions helper class that was added later. In many cases it can eliminate the need for the temporary variable.
As others have answered, this breaks because .ToString fails to translate to relevant SQL on the way into the database.
However, Microsoft provides the SqlFunctions class that is a collection of methods that can be used in situations like this.
For this case, what you are looking for here is SqlFunctions.StringConvert:
from p in context.pages
where p.Serial == SqlFunctions.StringConvert((double)item.Key.Id)
select p;
Good when the solution with temporary variables is not desirable for whatever reasons.
Similar to SqlFunctions you also have the EntityFunctions (with EF6 obsoleted by DbFunctions) that provides a different set of functions that also are data source agnostic (not limited to e.g. SQL).
The problem is that you are calling ToString in a LINQ to Entities query. That means the parser is trying to convert the ToString call into its equivalent SQL (which isn't possible...hence the exception).
All you have to do is move the ToString call to a separate line:
var keyString = item.Key.ToString();
var pages = from p in context.entities
where p.Serial == keyString
select p;
Cast table to Enumerable, then you call LINQ methods with using ToString() method inside:
var example = contex.table_name.AsEnumerable()
.Select(x => new {Date = x.date.ToString("M/d/yyyy")...)
But be careful, when you calling AsEnumerable or ToList methods because you will request all data from all entity before this method. In my case above I read all table_name rows by one request.
Had a similar problem.
Solved it by calling ToList() on the entity collection and querying the list.
If the collection is small this is an option.
IQueryable<entity> pages = context.pages.ToList().Where(p=>p.serial == item.Key.ToString())
Hope this helps.
Upgrading to Entity Framework Version 6.2.0 worked for me.
I was previously on Version 6.0.0.
Hope this helps,
Change it like this and it should work:
var key = item.Key.ToString();
IQueryable<entity> pages = from p in context.pages
where p.Serial == key
select p;
The reason why the exception is not thrown in the line the LINQ query is declared but in the line of the foreach is the deferred execution feature, i.e. the LINQ query is not executed until you try to access the result. And this happens in the foreach and not earlier.
If you really want to type ToString inside your query, you could write an expression tree visitor that rewrites the call to ToString with a call to the appropriate StringConvert function:
using System.Linq;
using System.Data.Entity.SqlServer;
using System.Linq.Expressions;
using static System.Linq.Expressions.Expression;
using System;
namespace ToStringRewriting {
class ToStringRewriter : ExpressionVisitor {
static MethodInfo stringConvertMethodInfo = typeof(SqlFunctions).GetMethods()
.Single(x => x.Name == "StringConvert" && x.GetParameters()[0].ParameterType == typeof(decimal?));
protected override Expression VisitMethodCall(MethodCallExpression node) {
var method = node.Method;
if (method.Name=="ToString") {
if (node.Object.GetType() == typeof(string)) { return node.Object; }
node = Call(stringConvertMethodInfo, Convert(node.Object, typeof(decimal?));
}
return base.VisitMethodCall(node);
}
}
class Person {
string Name { get; set; }
long SocialSecurityNumber { get; set; }
}
class Program {
void Main() {
Expression<Func<Person, Boolean>> expr = x => x.ToString().Length > 1;
var rewriter = new ToStringRewriter();
var finalExpression = rewriter.Visit(expr);
var dcx = new MyDataContext();
var query = dcx.Persons.Where(finalExpression);
}
}
}
In MVC, assume you are searching record(s) based on your requirement or information.
It is working properly.
[HttpPost]
[ActionName("Index")]
public ActionResult SearchRecord(FormCollection formcollection)
{
EmployeeContext employeeContext = new EmployeeContext();
string searchby=formcollection["SearchBy"];
string value=formcollection["Value"];
if (formcollection["SearchBy"] == "Gender")
{
List<MvcApplication1.Models.Employee> emplist = employeeContext.Employees.Where(x => x.Gender == value).ToList();
return View("Index", emplist);
}
else
{
List<MvcApplication1.Models.Employee> emplist = employeeContext.Employees.Where(x => x.Name == value).ToList();
return View("Index", emplist);
}
}
I got the same error in this case:
var result = Db.SystemLog
.Where(log =>
eventTypeValues.Contains(log.EventType)
&& (
search.Contains(log.Id.ToString())
|| log.Message.Contains(search)
|| log.PayLoad.Contains(search)
|| log.Timestamp.ToString(CultureInfo.CurrentUICulture).Contains(search)
)
)
.OrderByDescending(log => log.Id)
.Select(r => r);
After spending way too much time debugging, I figured out that error appeared in the logic expression.
The first line search.Contains(log.Id.ToString()) does work fine, but the last line that deals with a DateTime object made it fail miserably:
|| log.Timestamp.ToString(CultureInfo.CurrentUICulture).Contains(search)
Remove the problematic line and problem solved.
I do not fully understand why, but it seems as ToString() is a LINQ expression for strings, but not for Entities. LINQ for Entities deals with database queries like SQL, and SQL has no notion of ToString(). As such, we can not throw ToString() into a .Where() clause.
But how then does the first line work? Instead of ToString(), SQL have CAST and CONVERT, so my best guess so far is that linq for entities uses that in some simple cases. DateTime objects are not always found to be so simple...
My problem was that I had a 'text' data type for this column (due to a migration from sqlite).
Solution: just change the data type to 'nvarchar()' and regenerate the table.
Then Linq accepts the string comparison.
I am working on retiring Telerik Open Access and replacing it with Entity Framework 4.0. I came across same issue that telerik:GridBoundColumn filtering stopped working.
I find out that its not working only on System.String DataTypes. So I found this thread and solved it by just using .List() at the end of my Linq query as follows:
var x = (from y in db.Tables
orderby y.ColumnId descending
select new
{
y.FileName,
y.FileSource,
y.FileType,
FileDepartment = "Claims"
}).ToList();
Just turn the LINQ to Entity query into a LINQ to Objects query (e.g. call ToArray) anytime you need to use a method call in your LINQ query.
I have a table called "test", which only has 1 column, "NullableInt" (nullable int type)
The records are: 1, 2, null
int? nullableInt = null;
var t = db.tests.Where(x => x.NullableInt == null).ToList(); // returns 1 record
var t2 = db.tests.Where(x => x.NullableInt == nullableInt).ToList(); // returns 0 records
For some reason, t2 returns 0 records, even tho it's using "nullableInt" variable, which has a value of null, just like t, which is comparing against "null"
Any help would be greatly appreciated!
Yep - it's a bug in LINQ-to-SQL / Entity Framework. IS NULL queries will only be generated if you hardcode null into the query, instead of a variable that happens to currently be null.
The second query will generate
SELECT .......
WHERE NullableInt == #someParam
WHERE #someParam is null.
Where the first will generate the appropriate IS NULL in the WHERE clause.
If you're using LINQ-to-SQL, you can log your queries to Console.Out to see for yourself, and if you're using EF, then ToTraceString() should show you the same info (or SQL Server profiler)
tl;dr
If you use DbContext in EF6 this is fixed.
If you're using EF5 (or ObjectContext in EF6) you need to set ObjectContext.ContextOptions.UseCSharpNullComparisonBehavior to true. To do that on DbContext use this:
((IObjectContextAdapter)db).ObjectContext.ContextOptions.UseCSharpNullComparisonBehavior = true;
.
More details
The root cause of this issue is a difference in how the database compares null values and how C# compares null values. Because you write your query in C# you want to use the semantics of C#.
In EF5 we introduced ObjectContext.ContextOptions.UseCSharpNullComparisonBehavior, which allowed you to opt in to using C# semantics instead of database semantics. The default is false (so that existing queries don't magically start returning different results when you upgrade to EF5). But you can set it to true and both your queries will return rows.
If you are using DbContext in EF5 you need to drop down to the ObjectContext to set it:
((IObjectContextAdapter)db).ObjectContext.ContextOptions.UseCSharpNullComparisonBehavior = true;
If you are using EF6, then it's already set to true on DbContext so you are good to go. We decided this causes so much confusion it was worth taking the potential impact on existing queries.
Queries could be built in this way:
var q = db.tests;
if(nullableInt.HasValue)
{
q = q.Where(x => x.NullableInt == nullableInt.Value);
}
else
{
q = q.Where(x => x.NullableInt == null);
}
var t2 = q.ToList();
There is another solution that will always work, albeit with a small caveat:
int? nullableInt = null;
var t2 = db.tests.Where(x => object.Equals(x.NullableInt, nullableInt)).ToList();
When the value is null you will get the proper IS NULL query, however when its not null you will get something like:
SELECT ...
WHERE ([t0].[NullableInt] IS NOT NULL) AND ([t0].[NullableInt] = #p0)
Obviously it has a condition extra (the source of which is kind of puzzling). That being said, SQL Server's query optimizer should detect that, since #p0 is a non-null value, the first condition is a superset and will cut the where clause.
Would doing:
var t2 = db.tests.Where(x => x.NullableInt == nullableInt ?? null).ToList();
Work?
It seems like utter madness though.
I know the following is possible with linq2db4o
from Apple a in db
where a.Color.Equals(Colors.Green)
select a
What I need however is something that allows me to build my query conditionally (like I can in other linq variants)
public IEnumerable<Apple> SearchApples (AppleSearchbag bag){
var q = db.Apples;
if(bag.Color != null){
q = q.Where(a=>a.Color.Equals(bag.Color));
}
return q.AsEnumerable();
}
In a real world situation the searchbag will hold many properties and building a giant if-tree that catches all possible combinations of filled in properties would be madman's work.
It is possible to first call
var q = (from Color c in db select c);
and then continue from there. but this is not exactly what I'm looking for.
Disclaimer: near duplicate of my question of nearly 11 months ago.
This one's a bit more clear as I understand the matter better now and I hope by now some of the db4o dev eyes could catch this on this:
Any suggestions?
Yes it's definitely possible to compose optimized LINQ queries using db4o. Granted that db is defined as follows:
IObjectContainer db;
Here is your query:
public IEnumerable<Apple> SearchApples (AppleSearchbag bag)
{
var query = db.Cast<Apple> ();
// query will be a Db4objects.Db4o.Linq.IDb4oLinqQuery<Apple>
if (bag.Color != null)
query = query.Where (a => a.Color == bag.Color);
return query;
}
In that case, the query will be executed whenever the returned enumerable is being iterated over.
Another possibility is to use the IQueryable mechanism, that has the advantage of being better recognized by developers:
public IQueryable<Apple> SearchApples (AppleSearchbag bag)
{
var query = db.AsQueryable<Apple> ();
// query will be a System.Linq.IQueryble<Apple>
if (bag.Color != null)
query = query.Where (a => a.Color == bag.Color);
return query;
}
In both cases, db4o will try to deduce an optimized query from the lambda expression upon execution, and if it fails, will fallback to LINQ to objects. The first one has the advantage of being more direct, by avoiding the queryable to LINQ to db4o transformation.
Can you return IQueryable which is composed of two or more different subclasses ? Here's my attempt to show what I mean. It throws the error:
System.NotSupportedException: Types in
Union or Concat have members assigned
in different order..
var a = from oi in db.OrderItems
where oi.OrderID == ID
&& oi.ListingID != null
select new OrderItemA {
// etc
} as OrderItem;
var b = from oi in db.OrderItems
where oi.OrderID == ID
&& oi.AdID != null
select new OrderItemB {
//etc
} as OrderItem;
return a.Concat<OrderItem>(b);
Try doing the concat on IEnumerable instead of IQueryable:
return a.AsEnumerable().Concat(b.AsEnumerable());
If you need an IQueryable result you could do this:
return a.AsEnumerable().Concat(b.AsEnumerable()).AsQueryable();
Doing this will force the concat to happen in-memory instead of in SQL, and any additional operations will also happen in-memory (LINQ To Objects).
However, unlike the .ToList() example, the execution should still be deferred (making your data lazy loaded).
My guess is that this is because you are using LINQ in an LINQ-to-SQL context.
So using Concat means that LINQ2SQL will need to join both query into a SQL UNION query which might be where the System.NotSupportedException originated from.
Can you try this:
return a.ToList().Concat<OrderItem>(b.ToList());
And see if it make any difference?
What the above does is that it executes the query twice and then concatenate them in-memory instead of hot-off-SQL as to avoid the query translation problem.
It might not be the ideal solution, but if this work, my assumption is probably correct, that it's a query translation problem:
More information about Union and Concat translation to SQL:
http://blog.benhall.me.uk/2007/08/linq-to-sql-difference-between-concat.html
http://msdn.microsoft.com/en-us/library/bb399342.aspx
http://msdn.microsoft.com/en-us/library/bb386979.aspx
Hope this helps.
Interestingly after reading your post and a bit of testing, I realized that what your actually doing does seem to work just fine for me given that the projection part you show as ellipsis in both of your queries match. You see, LINQ to SQL appears to construct the underlying projection for the SQL select command based off of the property assignment statements as opposed to the actual type being materialized so as long as both sides have the same number, type, and order (not sure about this) of member assignments the UNION query should be valid.
My solution that I've been working with is to create a property on my DataContext class which acts much like a SQL View in that it allows me to write a query (in my case a Union between two different tables) and then use that query as if it is itself like a table when composing read-only select statements.
public partial class MyDataContext
{
public IQueryable<MyView> MyView
{
get
{
var query1 =
from a in TableA
let default_ColumnFromB = (string)null
select new MyView()
{
ColumnFromA = a.ColumnFromA,
ColumnFromB = default_ColumnFromB,
ColumnSharedByAAndB = a.ColumnSharedByAAndB,
};
var query2 =
from a in TableB
let default_ColumnFromA = (decimal?)null
select new MyView()
{
ColumnFromA = default_ColumnFromA,
ColumnFromB = b.ColumnFromB,
ColumnSharedByAAndB = b.ColumnSharedByAAndB,
};
return query1.Union(query2);
}
}
}
public class MyView
{
public decimal? ColumnFromA { get; set; }
public string ColumnFromB { get; set; }
public int ColumnSharedByAAndB { get; set; }
}
Notice two key things:
First of all the projection formed by the queries which make up both halves of the Union have the same number, type, and order of columns. Now LINQ may require the order to be the same (not sure about this) but it is definitely true that SQL does for a UNION and we can be sure that LINQ will require at least the same type and number of columns and these "columns" are known by the member assignments and not from the properties of the type you are instantiating in your projection.
Secondly LINQ currently doesn't allow for multiple constants to be used within a projections for queries which formulate a Concat or Union and from my understanding this is mainly because these two separate queries are separately optimized before the Union operation is processed. Normally LINQ to SQL is smart enough to realize that if you have a constant value which is only being used in the projection, then why send it to SQL just to have it come right back the way it was instead of tacking it on as a post process after the raw data comes back from SQL Server. Unfortunately the problem here is that this is a case of LINQ to SQL being to smart for it's own good, as it optimizes each individual query too early in the process. The way I've found to work around this is to use the let keyword to form a range variable for each value in the projection which will be materialized by getting it's value from a constant. Somehow this tricks LINQ to SQL into carrying these constants through to the actual SQL command which keeps all expected columns in the resulting UNION. More on this technique can be found here.
Using this techinque I at least have something reusable so that no matter how complex or ugly the actual Union can get, especially with the range variables, that in your end queries you can write queries to these pseudo views such as MyView and deal with the complexity underneath.
Can you do the projection after the concat?
// construct the query
var a = from oi in db.OrderItems
where oi.OrderID == ID
&& oi.ListingID != null
select new {
type = "A"
item = oi
}
var b = from oi in db.OrderItems
where oi.OrderID == ID
&& oi.AdID != null
select new {
type = "B"
item = oi
}
var temp = a.Concat<OrderItem>(b);
// create concrete types after concatenation
// to avoid inheritance issue
var result = from oi in temp
select (oi.type == "A"
? (new OrderItemA {
// OrderItemA projection
} as OrderItem)
: (new OrderItemB {
// OrderItemB projection
} as OrderItem)
);
return result
Not sure if the ternary operator works in LINQ2SQL in the above scenario but that might help avoid the inheritance issue.