We Keep Coding

How do I create a boolean "OR" LINQ statement from a runtime set of conditions? [duplicate]

I'm getting an array of strings for which I want to see if a certain number of data fields in the domain object have all of those strings. I know the data fields at compile-time but I don't know the size of the array at compile-time.
Is there a way that I can compose a where clause at run-time so that I can do what I'm looking for in a single linq query.
If you're wondering why it's a single query: I want to reduce the round trips to the DB as much as possible.
public IEnumerable<domainObjects> GetObjectsWith(string[] data)
{
var results = from d in domainObjects
where
(d.Data.Contains(data[0]) && d.Data.Contains(data[1]) && ...)
||
(d.Data2.Contains(data[0]) && d.Data.Contains(data[1]) && ...)
.
.
. // Data fields known at compile-time
}
The end result is, given 2 objects:
domainObject { Address = "1st st", Description="has couch and bed" }
domainObject2 { Address = "1st rd", Description="has couch" }
A query for { "couch", "bed" } would only return domainobject 1 but a query for { "couch" } would return both.
Likeqise a query for { "1st", "couch", "bed" } would return both as well.

You should use PredicateBuilder, it is a free utility class that allows you to construct complex where clauses using And's and Or's at runtime. You can loop through your array and build your where clause that way.
http://www.albahari.com/nutshell/predicatebuilder.aspx

There's 3 main ways to do it, PredicateBuilder, Dynamic Linq library, or manipulating your own Expression Trees (which the first two help you do under the covers).
PredicateBuilder is your best bet if you know all the properties in advance. If they are dynamic (i.e. the user selects them, then Dynamic Linq is the best bet).
See also Is there a pattern using Linq to dynamically create a filter?.
link text

Whether to use AsEnumerable() in LINQ or not?

I understand AsEnumerable() is used to switch from "LINQ to SQL" to "LINQ to Object", so we can use some extra (mostly user defined) methods in our LINQ queries. But from my experience I've seen, using AsEnumerable() makes the query much slower. In that case I can enumerate the list later to apply my own methods, but still the result is pretty slow.
Can anyone suggest any better approach?
Here is some code sample of what I'm trying to do?
With AsEnumerable():
var Data = (from r in _context.PRD_ChemProdReq.AsEnumerable()
//where r.RecordStatus == "NCF"
orderby r.RequisitionNo descending
select new PRDChemProdReq
{
RequisitionID = r.RequisitionID,
RequisitionNo = r.RequisitionNo,
RequisitionCategory = DalCommon.ReturnRequisitionCategory(r.RequisitionCategory),
RequisitionType = DalCommon.ReturnOrderType(r.RequisitionType),
ReqRaisedOn = (Convert.ToDateTime(r.ReqRaisedOn)).ToString("dd'/'MM'/'yyyy"),
RecordStatus= DalCommon.ReturnRecordStatus(r.RecordStatus),
RequisitionFromName = DalCommon.GetStoreName(r.RequisitionFrom),
RequisitionToName = DalCommon.GetStoreName(r.RequisitionTo)
}).ToList();
without AsEnumerable():
var Data = (from r in _context.PRD_ChemProdReq
//where r.RecordStatus == "NCF"
orderby r.RequisitionNo descending
select new PRDChemProdReq
{
RequisitionID = r.RequisitionID,
RequisitionNo = r.RequisitionNo,
RequisitionCategory = r.RequisitionCategory,
RequisitionType = (r.RequisitionType),
ReqRaisedOnTemp = (r.ReqRaisedOn),
RecordStatus= (r.RecordStatus),
RequisitionFrom = (r.RequisitionFrom),
RequisitionTo = (r.RequisitionTo)
}).ToList();
foreach (var item in Data)
{
item.RequisitionCategory = DalCommon.ReturnRequisitionCategory(item.RequisitionCategory);
item.RequisitionType = DalCommon.ReturnOrderType(item.RequisitionType);
item.ReqRaisedOn = (Convert.ToDateTime(item.ReqRaisedOnTemp)).ToString("dd'/'MM'/'yyyy");
item.RecordStatus = DalCommon.ReturnRecordStatus(item.RecordStatus);
item.RequisitionFromName = DalCommon.GetStoreName(item.RequisitionFrom);
item.RequisitionToName = DalCommon.GetStoreName(item.RequisitionTo);
}

It looks like you confuse these two interfaces as two totally different things. In fact IQueryable is inherited from IEnumerable, so whatever worked for you using with latter, would work with former as well, so don't need to use AsEnumerable.
Behind the scenes though these interfaces are implemented quite differently - IEnumerable will process your collection in memory, and IQueryable would pass the query to the underlying data provider. You can imagine that if a database table contains millions of records and you try to sort it, DB server can do it very quickly (using indexes) so Queryable will shine. For IEnumerable, all the data need to be loaded to your computer memory and sorted there.
For a longer answers search for "IEnumerable IQueryable difference" on SO, you will see a plenty of details:
Random Link 1
Random Link 2
Update: If you remove call .ToList from your second example, then the result won't be automatically loaded to memory. At this point you need to decide which items you want to store in memory and call your functions for them only.
var Data = (from r in _context.PRD_ChemProdReq
orderby r.RequisitionNo descending
select new PRDChemProdReq
{
// do your initialization
});
var subsetOfData = Data.Take(100).ToList(); // Now it's loaded to memory
foreach (var item in subsetOfData)
{
item.RequisitionCategory = DalCommon.ReturnRequisitionCategory(item.RequisitionCategory);
item.RequisitionType = DalCommon.ReturnOrderType(item.RequisitionType);
item.ReqRaisedOn = (Convert.ToDateTime(item.ReqRaisedOnTemp)).ToString("dd'/'MM'/'yyyy");
item.RecordStatus = DalCommon.ReturnRecordStatus(item.RecordStatus);
item.RequisitionFromName = DalCommon.GetStoreName(item.RequisitionFrom);
item.RequisitionToName = DalCommon.GetStoreName(item.RequisitionTo);
}
Now if you actually need to assign these properties for all your data and data can be arbitrary large, you need to work out a strategy how you can do it. The very simple option is to save them to database to a new table, then the size of processed data will be only limited by the capacity of your database.

AsEnumerable() will be slower if you add any query elements after it.
Even though AsEnumerable() doesn't execute the query directly, applying a where or orderby after the AsEnumerable() means that the Sql will get all items and then apply filtering and ordering to the collection in memory.
In short:
Without AsEnumerable() = filtering and ordering done in SQL
With AsEnumerable() and then Where or orderby = applied to whole collection brought into memory.
You can only run user defined functions on a collection in memory (as your Linq to SQL will not be able to interpret your functions to SQL code). So your second code snippet (without AsEnumerable()) is probably best.
The only other alternative is to apply your user defined functions in SQL itself.

OK guys, I took notice of different points from all of you & came up with this:
var Data = (from r in _context.PRD_ChemProdReq.AsEnumerable()
//where r.RecordStatus == "NCF"
join rf in _context.SYS_Store on (r.RequisitionFrom==null?0: r.RequisitionFrom) equals rf.StoreID into requisitionfrom
from rf in requisitionfrom.DefaultIfEmpty()
join rt in _context.SYS_Store on (r.RequisitionTo == null ? 0 : r.RequisitionTo) equals rt.StoreID into requisitionto
from rt in requisitionto.DefaultIfEmpty()
orderby r.RequisitionNo descending
select new PRDChemProdReq
{
RequisitionID = r.RequisitionID,
RequisitionNo = r.RequisitionNo,
RequisitionCategory = DalCommon.ReturnRequisitionCategory(r.RequisitionCategory),
RequisitionType = r.RequisitionType == "UR" ? "Urgent" : "Normal",
ReqRaisedOn = (Convert.ToDateTime(r.ReqRaisedOn)).ToString("dd'/'MM'/'yyyy"),
RecordStatus = (r.RecordStatus=="NCF"? "Not Confirmed": "Approved"),
RequisitionFromName = (rf==null? null: rf.StoreName),
RequisitionToName = (rt == null ? null : rt.StoreName)
});
First of all I removed my ToList() which does nothing but executes the query which is already done when I called AsEnumerable(). No points to execute the same query twice. Also my custom method calls within the select block was also playing a major part slowing down things. I tried lessen the method calls, rather used join where possible. It makes things pretty faster. Thank You all.

Custom Method in LINQ Query

I sum myself to the hapless lot that fumbles with custom methods in LINQ to EF queries. I've skimmed the web trying to detect a pattern to what makes a custom method LINQ-friendly, and while every source says that the method must be translatable into a T-SQL query, the applications seem very diverse. So, I'll post my code here and hopefully a generous SO denizen can tell me what I'm doing wrong and why.
The Code
public IEnumerable<WordIndexModel> GetWordIndex(int transid)
{
return (from trindex in context.transIndexes
let trueWord = IsWord(trindex)
join trans in context.Transcripts on trindex.transLineUID equals trans.UID
group new { trindex, trans } by new { TrueWord = trueWord, trindex.transID } into grouped
orderby grouped.Key.word
where grouped.Key.transID == transid
select new WordIndexModel
{
Word = TrueWord,
Instances = grouped.Select(test => test.trans).Distinct()
});
}
public string IsWord(transIndex trindex)
{
Match m = Regex.Match(trindex.word, #"^[a-z]+(\w*[-]*)*",
RegexOptions.IgnoreCase);
return m.Value;
}
With the above code I access a table, transIndex that is essentially a word index of culled from various user documents. The problem is that not all entries are actually words. Nubers, and even underscore lines, such as, ___________,, are saved as well.
The Problem
I'd like to keep only the words that my custom method IsWord returns (at the present time I have not actually developed the parsing mechanism). But as the IsWord function shows it will return a string.
So, using let I introduce my custom method into the query and use it as a grouping parameter, the is selectable into my object. Upon execution I get the omninous:
LINQ to Entities does not recognize the method
'System.String IsWord(transIndex)' method, and this
method cannot be translated into a store expression."
I also need to make sure that only records that match the IsWord condition are returned.
Any ideas?

It is saying it does not understand your IsWord method in terms of how to translate it to SQL.
Frankly it does not do much anyway, why not replace it with
return (from trindex in context.transIndexes
let trueWord = trindex.word
join trans in context.Transcripts on trindex.transLineUID equals trans.UID
group new { trindex, trans } by new { TrueWord = trueWord, trindex.transID } into grouped
orderby grouped.Key.word
where grouped.Key.transID == transid
select new WordIndexModel
{
Word = TrueWord,
Instances = grouped.Select(test => test.trans).Distinct()
});
What methods can EF translate into SQL, i can't give you a list, but it can never translate a straight forward method you have written. But their are some built in ones that it understands, like MyArray.Contains(x) for example, it can turn this into something like
...
WHERE Field IN (ArrItem1,ArrItem2,ArrItem3)
If you want to write a linq compatible method then you need to create an expresion tree that EF can understand and turn into SQL.
This is where things star to bend my mind a little but this article may help http://blogs.msdn.com/b/csharpfaq/archive/2009/09/14/generating-dynamic-methods-with-expression-trees-in-visual-studio-2010.aspx.

If the percentage of bad records in return is not large, you could consider enumerate the result set first, and then apply the processing / filtering?
var query = (from trindex in context.transIndexes
...
select new WordIndexModel
{
Word,
Instances = grouped.Select(test => test.trans).Distinct()
});
var result = query.ToList().Where(word => IsTrueWord(word));
return result;
If the number of records is too high to enumerate, consider doing the check in a view or stored procedure. That will help with speed and keep the code clean.
But of course, using stored procedures has disadvatages of reusability and maintainbility (because of no refactoring tools).
Also, check out another answer which seems to be similar to this one: https://stackoverflow.com/a/10485624/3481183

LINQ to SQL where clause verifying string contains list element

I'm using a view returning Domains according to an id. The Domains column can be 'Geography' or can be stuffed domains 'Geography,History'. (In any way, the data returned is a VARCHAR)
In my C# code, I have a list containing main domains:
private static List<string> _mainDomains = new List<string>()
{
"Geography",
"Mathematics",
"English"
};
I want to filter my LINQ query in order to return only data related to one or many main Domain:
expression = i => _mainDomains.Any(s => i.Domains.Contains(s));
var results = (from v_lq in context.my_view
select v_lq).Where(expression)
The problem is I can't use the Any key word, nor the Exists keyword, since they aren't available in SQL. I've seen many solutions using the Contains keyword, but it doesn't fit to my problem.
What should I do?

You can use contains:
where i.Domains.Any(s => _mainDomains.Contains<string>(s.xxx))
Notice that the generic arguments are required (even if Resharper might tell you they are not). They are required to select Enumerable.Contains, not List.Contains. The latter one is not translatable (which I consider an error in the L2S product design).
(I might not have gotten the query exactly right for your data model. Please just adapt it to your needs).

I figured it out. Since I can't use the Any keyword, I used this function:
public static bool ContainsAny(this string databaseString, List<string> stringList)
{
if (databaseString == null)
{
return false;
}
foreach (string s in stringList)
{
if (databaseString.Contains(s))
{
return true;
}
}
return false;
}
So then I can use this expression in my Where clause:
expression = i => i.Domains.ContainsAny(_mainDomains);
Update:
According to usr, the query would return all the values and execute the where clause server side. A better solution would be to use a different approach (and not use stuffed/comma-separated values)

First SQL to Linq Conversion

I am new to LINQ, can somebody guide me that how can I convert the following SQL Query in Linq.
Select tblIns.InsID, tblProg.ProgName
from tblIns, tblProg
Where tblIns.InsID = tblProg.InsID
I am working on MVC2 Project, I have dataContext and Reposetories , please find below the code where I need this query:
public IQueryable<tblInstitute> InsRepeater()
{
return from Inst in _db.tblInstitutes
from Progs in _db.tblPrograms
Where Inst.InstituteID = Progs.InstituteID
Select Inst.InstituteID, Progs.ProgramName
}

The first thing you need is a data context which emulates your database. Here is an example of how to do this with linq to sql. You can also do it with entity framework (EF) or any other provider.
Once you have the tables created, the query then translates pretty straight forward:
var results = from insEntity in tablIns
from progEntity in tablProg
where insEntity.InsID equals progEntity.InsID
select new { insEntity.InsID, progEntity.ProgName };
With the question you have asked, this is as much as I think will be useful. In the future it's best to write questions explaining what you are trying to do, what you have tried, and then where you are stuck. The question should be specific enough to get you just over the next hump.
Per your edit: The query you have needs to have lowercase where and select and it needs to end the statement with a semi-colon (assuming it is c#). Then you select statement needs to select a new object. The results would look something like this:
public IQueryable<tblInstitute> InsRepeater()
{
return from Inst in _db.tblInstitutes
from Progs in _db.tblPrograms
where Inst.InstituteID equals Progs.InstituteID
select Inst; // for the current method header
//select new { Inst.InstituteID, Progs.ProgramName }; // to use this one you'll have to create a new type with the properties you want to return
}

First you need to
1. Create entity classes.
2. The data context.
3. Define relationships
4. Query
Reference
from I in tblIns
from P in tblProg
Where I.InsID = P.InsID
Select I.InsID, P.ProgName

Assuming you have your foreign keys set up properly in the database you can just do this, there is no need to to the joins yourself:
from x in db.tblProgs
select x.tblIns.id, x.Progname