I'm trying to implement method Find that searches the database.
I forgot to mention that I'm using Postgresql, so I can't use built in LINQ to SQL.
I want it to be like that:
var user = User.Find(a => a.LastName == "Brown");
Like it's done in List class. But when I go to List's source code (thanks, Reflector), I see this:
public T Find(Predicate<T> match)
{
if (match == null)
{
ThrowHelper.ThrowArgumentNullException(ExceptionArgument.match);
}
for (int i = 0; i < this._size; i++)
{
if (match(this._items[i]))
{
return this._items[i];
}
}
return default(T);
}
How can I implement this thing? I need to get those parameters to make the search.
Solution
Okay, I understood now that I need to do LINQ to SQL to do all this good expressions stuff, otherwise I'd have to spend a lot of time reimplementeing the wheel.
Since I can't use LINQ to SQL, I implemented this easy method:
public static User Find(User match, string orderBy = "")
{
string query = "";
if (!String.IsNullOrEmpty(match.FirstName)) query += "first_name='" + match.FirstName + "'";
if (!String.IsNullOrEmpty(match.LastName)) query += "last_name='" + match.LastName+ "'";
return Find(query + (!String.IsNullOrEmpty(orderBy) ? orderBy : ""));
}
This is how to use it:
var user = User.Find(new User { FirstName = "Bob", LastName = "Brown" });
Your method should accept Expression<Func<User>>.
This will give you expression tree instead of delegate which you can analyze and serialize to SQL or convert to any other API call your database have.
If you want everything to be generic, you may wish to go on with implementing IQueryable interface. Useful information can be found here: LINQ Tips: Implementing IQueryable Provider
Although for a simple scenario I would suggest not to complicate everything and stick with using Expression Trees and returning plain IEnumerable<T> or even List<T>.
For your case first version of code could look like this:
public IEnumerable<T> Get(Expression<Func<T, bool>> condition)
{
if (condition.Body.NodeType == ExpressionType.Equal)
{
var equalityExpression = ((BinaryExpression)condition.Body);
var column = ((MemberExpression)equalityExpression.Left).Member.Name;
var value = ((ConstantExpression)equalityExpression.Right).Value;
var table = typeof(T).Name;
var sql = string.Format("select * from {0} where {1} = '{2}'", table, column, value);
return ExecuteSelect(sql);
}
return Enumerable.Empty<T>();
}
And it's complexity grows fast when you want to handle new and new scenarios so make sure you have reliable unit tests for each scenario.
C# Samples for Visual Studio 2008 contain ExpressionTreeVisualizer that will help you to dig into Expression Trees more easily to understand how to extract information you need from it.
And of course, if you can stick with using existing implementation of LINQ, I would suggest to do it. There are Linq to SQL for SQL Server databases, Linq to Entities for many different databases, Linq to NHibernate for NHbernate projects.
Many other LINQ providers can be found here: Link to Everything: A List of LINQ Providers. Amount of work to implement LINQ provider is not trivial so it's a good idea to reuse tested and supported solution.
Exactly the same way. Just replace this._items with your users collection.
Also replace the type parameter T with the type User.
A lambda expression in source code can be converted to either a compiled executable delegate or an expression tree upon compilation. Usually we associate lambda's with delegates but in your case since you say you want access to the parameters (in this case I assume you mean LastName and "Brown" then you want an expression tree.
Once you have an expression tree, you can parse it to see exactly what it is an translate it to whatever you actually need to do.
Here are a few questions about expression trees.
Expression trees for dummies?
Bit Curious to understand Expression Tree in .NET
Sounds like you're definitely reinventing a very complicated wheel though. I'm sure it'll be a useful learning experience, but you should look into LINQ to Entities or LINQ to SQL for real-world programming.
Maybe I just haven't understood the question, but there's already a method for doing what you want: Enumerable.Where.
If you need to find a single element then use SingleOrDefault or FirstOrDefault instead.
You could do it something like this:
public static IEnumerable<User> Find(Predicate<User> match)
{
//I'm not sure of the name
using (var cn = new NpgsqlConnection("..your connection string..") )
using (var cmd = new NpgsqlCommand("SELECT * FROM Users", cn))
using (var rdr = cmd.ExecuteReader())
{
while (rdr.Read())
{
var user = BuildUserObjectFromIDataRecord(rdr);
if (match(user)) yield return user;
}
}
}
And then you can call it like this
var users = User.Find(a => a.LastName == "Brown");
Note that this returns any number of users, you still have to implement the BuildUserObjectFromIDataRecord() function, and that it will always want to iterate over the entire users table. But it gives you the exact semantics you want.
Okay, I understood now that I need to do LINQ to SQL to do all this good expressions stuff, otherwise I'd have to spend a lot of time reimplementeing the wheel.
Since I can't use LINQ to SQL, I implemented this easy method:
public static User Find(User match, string orderBy = "")
{
string query = "";
if (!String.IsNullOrEmpty(match.FirstName)) query += "first_name='" + match.FirstName + "'";
if (!String.IsNullOrEmpty(match.LastName)) query += "last_name='" + match.LastName+ "'";
return Find(query + (!String.IsNullOrEmpty(orderBy) ? orderBy : ""));
}
This is how to use it:
var user = User.Find(new User { FirstName = "Bob", LastName = "Brown" });
One way would be to create an anonymous delegate, like so:
Predicate<User> Finder = delegate(User user)
{
return user.LastName == "Brown";
}
var User = User.Find(Finder);
Related
I have a list of objects in a hierarchical structure. I want to build complex LINQ queries against that object list based on "conditions" a client company sets and are stored in the database. So I need to build these at run time, but because they will be run repeatedly whenever the client's users update or refresh their data I would like to store the LINQ queries in objects rather than rebuild them each time.
I have looked at ScottGu's Blog about Dynamic LINQ.
Also this article about using expression trees.
Neither of these appear to provide an adequate solution, but I may not be understanding them adequately. I'm afraid that I'm trying to use LINQ when I should consider other options.
My object hierarchy:
WorkOrder[]
Field[]
Task[]
Field[]
Here is an example of a LINQ query that I would like to store and execute. I can reasonably build this format based on the database records that define the conditions.
var query =
from wo in WorkOrders
from woF in wo.Fields
from task in wo.Tasks
from taskF in task.Fields
from taskF2 in task.Fields
where woF.Name == "System Status"
&& woF.Value.Contains("SETC")
&& taskF.Name == "Material"
&& taskF.Value == "Y"
&& taskF2.Name == "Planner"
&& taskF2.Value == "GR5259"
select new
{
wo_id = wo.ID,
task_id = task.ID
};
A few considerations.
Depending on the complexity of the user defined conditions I may or may not need to pull from the different object lists: the "froms" are dynamic.
Note that in this example I pulled twice from the task.fields[] so I aliased it two times.
The example LINQ structure allows me to have complex ANDs, ORs, parenthesis, etc. that I don't believe is practical with Dynamic Chaining or Expression Trees.
In my code I envision:
//1) Retrieve business rules from DB. I can do this.
//2) Iterate through the business rules to build the linq queries.
foreach (BusinessRule br in BusinessRules) {
//Grab the criteria for the rule from the DB.
//Create a linq to object query based on the criteria just built.
//Add this query to a list for later use.
}
...Elsewhere in application.
//Iterate through and execute the linq queries in order to apply business rules to data cached in the application.
foreach (LinqQuery q in LinqQueries) {
//Execute the query
//Apply business rule to the results.
}
Thank you very much for your thoughts, effort and ideas.
You can technically achieve what you need using only LINQ, but the PredicateBuilder is a nice utility class:
public enum AndOr
{
And,
Or
}
public enum QueryableObjects
{
WorkOrderField,
TaskField
}
public class ClientCondition
{
public AndOr AndOr;
public QueryableObjects QueryableObject;
public string PropertyName;
public string PropertyValue;
}
public void PredicateBuilderExample()
{
var conditions = new List<ClientCondition> {
{
new ClientCondition { AndOr = LINQ.AndOr.And,
QueryableObject = QueryableObjects.WorkOrderField,
PropertyName = "System Status",
PropertyValue = "SETC"
}
},
{
new ClientCondition{AndOr = AndOr.And,
QueryableObject = QueryableObjects.TaskField,
PropertyName = "Material",
PropertyValue = "Y"
}
},
{
new ClientCondition{AndOr = AndOr.Or,
QueryableObject = QueryableObjects.TaskField,
PropertyName = "Planner",
PropertyValue = "GR5259"
}
}
};
//Obviously this WorkOrder object is empty so it will always return empty lists when queried.
//Populate this yourself.
var WorkOrders = new List<WorkOrder>();
var wofPredicateBuilder = PredicateBuilder.True<WorkOrderField>();
var tfPredicateBuilder = PredicateBuilder.True<TaskField>();
foreach (var condition in conditions)
{
if (condition.AndOr == AndOr.And)
{
if (condition.QueryableObject == QueryableObjects.WorkOrderField)
{
wofPredicateBuilder = wofPredicateBuilder.And(
wof => wof.Name == condition.PropertyName &&
wof.Value.Contains(condition.PropertyValue));
}
}
if (condition.AndOr == AndOr.Or)
{
if (condition.QueryableObject == QueryableObjects.TaskField)
{
tfPredicateBuilder = tfPredicateBuilder.Or(
tf => tf.Name = condition.PropertyName &&
tf.Value.Contains(condition.PropertyValue));
}
}
//And so on for each condition type.
}
var query = from wo in WorkOrders
from woF in wo.Fields.AsQueryable().Where(wofPredicateBuilder)
from task in wo.Tasks
from taskF in task.Fields.AsQueryable().Where(tfPredicateBuilder)
select new
{
wo_id = wo.ID,
task_id = task.ID
};
}
Note that I use the enums to limit the possible conditions your clients can send you. To have a truly dynamic queryable engine, you will need to use Reflection to ensure the object names you receive are valid. That seems like a rather large scope, and at that point I would recommend researching a different approach, such as ElasticSearch.
Also note that the order of And and Ors matters significantly. Essentially you are allowing your customers to build SQL queries against your data, and that usually ends in tears. It's your job to limit them to the proper set of conditions they should be querying.
Based on the discussion with Guillaume I would only suggest to pay attention to the type of the resulting query when playing around with advanced dynamic query generation. If you are changing the shape of what is being returned via Select, Aggregate, or one of the other methods you will expect your inner type to change accordingly. If you are just filtering with Where you can keep adding on as many additional cases you want unless you want OR behavior then things like PredicateBuilder helps. If you want to pull in more data via Join, Zip, ... then you are either doing so to filter, add to the rows returned, and possibly change the shape of the data.
I've done a lot of this in the past and had most success focusing on specific helper methods that allow for common cases that I need and then leaning on linq expression trees and patterns such as the visitor pattern to allow custom expression built at runtime.
I sum myself to the hapless lot that fumbles with custom methods in LINQ to EF queries. I've skimmed the web trying to detect a pattern to what makes a custom method LINQ-friendly, and while every source says that the method must be translatable into a T-SQL query, the applications seem very diverse. So, I'll post my code here and hopefully a generous SO denizen can tell me what I'm doing wrong and why.
The Code
public IEnumerable<WordIndexModel> GetWordIndex(int transid)
{
return (from trindex in context.transIndexes
let trueWord = IsWord(trindex)
join trans in context.Transcripts on trindex.transLineUID equals trans.UID
group new { trindex, trans } by new { TrueWord = trueWord, trindex.transID } into grouped
orderby grouped.Key.word
where grouped.Key.transID == transid
select new WordIndexModel
{
Word = TrueWord,
Instances = grouped.Select(test => test.trans).Distinct()
});
}
public string IsWord(transIndex trindex)
{
Match m = Regex.Match(trindex.word, #"^[a-z]+(\w*[-]*)*",
RegexOptions.IgnoreCase);
return m.Value;
}
With the above code I access a table, transIndex that is essentially a word index of culled from various user documents. The problem is that not all entries are actually words. Nubers, and even underscore lines, such as, ___________,, are saved as well.
The Problem
I'd like to keep only the words that my custom method IsWord returns (at the present time I have not actually developed the parsing mechanism). But as the IsWord function shows it will return a string.
So, using let I introduce my custom method into the query and use it as a grouping parameter, the is selectable into my object. Upon execution I get the omninous:
LINQ to Entities does not recognize the method
'System.String IsWord(transIndex)' method, and this
method cannot be translated into a store expression."
I also need to make sure that only records that match the IsWord condition are returned.
Any ideas?
It is saying it does not understand your IsWord method in terms of how to translate it to SQL.
Frankly it does not do much anyway, why not replace it with
return (from trindex in context.transIndexes
let trueWord = trindex.word
join trans in context.Transcripts on trindex.transLineUID equals trans.UID
group new { trindex, trans } by new { TrueWord = trueWord, trindex.transID } into grouped
orderby grouped.Key.word
where grouped.Key.transID == transid
select new WordIndexModel
{
Word = TrueWord,
Instances = grouped.Select(test => test.trans).Distinct()
});
What methods can EF translate into SQL, i can't give you a list, but it can never translate a straight forward method you have written. But their are some built in ones that it understands, like MyArray.Contains(x) for example, it can turn this into something like
...
WHERE Field IN (ArrItem1,ArrItem2,ArrItem3)
If you want to write a linq compatible method then you need to create an expresion tree that EF can understand and turn into SQL.
This is where things star to bend my mind a little but this article may help http://blogs.msdn.com/b/csharpfaq/archive/2009/09/14/generating-dynamic-methods-with-expression-trees-in-visual-studio-2010.aspx.
If the percentage of bad records in return is not large, you could consider enumerate the result set first, and then apply the processing / filtering?
var query = (from trindex in context.transIndexes
...
select new WordIndexModel
{
Word,
Instances = grouped.Select(test => test.trans).Distinct()
});
var result = query.ToList().Where(word => IsTrueWord(word));
return result;
If the number of records is too high to enumerate, consider doing the check in a view or stored procedure. That will help with speed and keep the code clean.
But of course, using stored procedures has disadvatages of reusability and maintainbility (because of no refactoring tools).
Also, check out another answer which seems to be similar to this one: https://stackoverflow.com/a/10485624/3481183
I don't think is possible but wanted to ask to make sure. I am currently debugging some software someone else wrote and its a bit unfinished.
One part of the software is a search function which searches by different fields in the database and the person who wrote the software wrote a great big case statement with 21 cases in it 1 for each field the user may want to search by.
Is it possible to reduce this down using a case statement within the Linq or a variable I can set with a case statement before the Linq statement?
Example of 1 of the Linq queries: (Only the Where is changing in each query)
var list = (from data in dc.MemberDetails
where data.JoinDate.ToString() == searchField
select new
{
data.MemberID,
data.FirstName,
data.Surname,
data.Street,
data.City,
data.County,
data.Postcode,
data.MembershipCategory,
data.Paid,
data.ToPay
}
).ToList();
Update / Edit:
This is what comes before the case statement:
string searchField = txt1stSearchTerm.Text;
string searchColumn = cmbFirstColumn.Text;
switch (cmbFirstColumn.SelectedIndex + 1)
{
The cases are then done by the index of the combo box which holds the list of field names.
Given that where takes a predicate, you can pass any method or function which takes MemberDetail as a parameter and returns a boolean, then migrate the switch statement inside.
private bool IsMatch(MemberDetail detail)
{
// The comparison goes here.
}
var list = (from data in dc.MemberDetails
where data => this.IsMatch(data)
select new
{
data.MemberID,
data.FirstName,
data.Surname,
data.Street,
data.City,
data.County,
data.Postcode,
data.MembershipCategory,
data.Paid,
data.ToPay
}
).ToList();
Note that:
You may look for a more object-oriented way to do the comparison, rather than using a huge switch block.
An anonymous type with ten properties that you use in your select is kinda weird. Can't you return an instance of MemberDetail? Or an instance of its base class?
How are the different where statements handled, are they mutually excluside or do they all limit the query somehow?
Here is how you can have one or more filters for a same query and materialized after all filters have been applied.
var query = (from data in dc.MemberDetails
select ....);
if (!String.IsNullOrEmpty(searchField))
query = query.Where(pr => pr.JoinDate.ToString() == searchField);
if (!String.IsNullOrEmpty(otherField))
query = query.Where(....);
return query.ToList();
I'm using a view returning Domains according to an id. The Domains column can be 'Geography' or can be stuffed domains 'Geography,History'. (In any way, the data returned is a VARCHAR)
In my C# code, I have a list containing main domains:
private static List<string> _mainDomains = new List<string>()
{
"Geography",
"Mathematics",
"English"
};
I want to filter my LINQ query in order to return only data related to one or many main Domain:
expression = i => _mainDomains.Any(s => i.Domains.Contains(s));
var results = (from v_lq in context.my_view
select v_lq).Where(expression)
The problem is I can't use the Any key word, nor the Exists keyword, since they aren't available in SQL. I've seen many solutions using the Contains keyword, but it doesn't fit to my problem.
What should I do?
You can use contains:
where i.Domains.Any(s => _mainDomains.Contains<string>(s.xxx))
Notice that the generic arguments are required (even if Resharper might tell you they are not). They are required to select Enumerable.Contains, not List.Contains. The latter one is not translatable (which I consider an error in the L2S product design).
(I might not have gotten the query exactly right for your data model. Please just adapt it to your needs).
I figured it out. Since I can't use the Any keyword, I used this function:
public static bool ContainsAny(this string databaseString, List<string> stringList)
{
if (databaseString == null)
{
return false;
}
foreach (string s in stringList)
{
if (databaseString.Contains(s))
{
return true;
}
}
return false;
}
So then I can use this expression in my Where clause:
expression = i => i.Domains.ContainsAny(_mainDomains);
Update:
According to usr, the query would return all the values and execute the where clause server side. A better solution would be to use a different approach (and not use stuffed/comma-separated values)
I'm having some trouble figuring out the best way to do this, and I would appreciate any help.
Basically, I'm setting up a filter that allows the user to look at a history of audit items associated with an arbitrary "filter" of usernames.
The datasource is a SQL Server data base, so I'm taking the IQueryable "source" (either a direct table reference from the db context object, or perhaps an IQueryable that's resulted from additional queries), applying the WHERE filter, and then returning the resultant IQueryable object....but I'm a little stumped as to how to perform OR using this approach.
I've considered going the route of Expressions because I know how to OR those, but I haven't been able to figure out quite how to do that with a "Contains" type evaluation, so I'm currently using a UNION, but I'm afraid this might have negative impact on performance, and I'm wondering if it may not give me exactly what I need if other filters (in addition to user name filtering shown here) are added in an arbirary order.
Here is my sample code:
public override IQueryable<X> ApplyFilter<X>(IQueryable<X> source)
{
// Take allowed values...
List<string> searchStrings = new List<string>();
// <SNIP> (This just populates my list of search strings)
IQueryable<X> oReturn = null;
// Step through each iteration, and perform a 'LIKE %value%' query
string[] searchArray = searchStrings.ToArray();
for (int i = 0; i < searchArray.Length; i++)
{
string value = searchArray[i];
if (i == 0)
// For first step, perform direct WHERE
oReturn = source.Where(x => x.Username.Contains(value));
else
// For additional steps, perform UNION on WHERE
oReturn = oReturn.Union(source.Where(x => x.Username.Contains(value)));
}
return oReturn ?? source;
}
This feels like the wrong way to do things, but it does seem to work, so my question is first, is there a better way to do this? Also, is there a way to do a 'Contains' or 'Like' with Expressions?
(Editted to correct my code: In rolling back to working state in order to post it, I apparently didn't roll back quite far enough :) )
=============================================
ETA: Per the solution given, here is my new code (in case anyone reading this is interested):
public override IQueryable<X> ApplyFilter<X>(IQueryable<X> source)
{
List<string> searchStrings = new List<string>(AllowedValues);
// <SNIP> build collection of search values
string[] searchArray = searchStrings.ToArray();
Expression<Func<X, bool>> expression = PredicateBuilder.False<X>();
for (int i = 0; i < searchArray.Length; i++)
{
string value = searchArray[i];
expression = expression.Or(x => x.Username.Contains(value));
}
return source.Where(expression);
}
(One caveat I noticed: Following the PredicateBuilder's example, an empty collection of search strings will return false (false || value1 || ... ), whereas in my original version, I was assuming an empty list should just coallesce to the unfiltered source. As I thought about it more, the new version seems to make more sense for my needs, so I adopted that)
=============================================
You can use the PredicateBuilder from the LINQkit to dynamically construct your query.