We Keep Coding

Custom Method in LINQ Query

I sum myself to the hapless lot that fumbles with custom methods in LINQ to EF queries. I've skimmed the web trying to detect a pattern to what makes a custom method LINQ-friendly, and while every source says that the method must be translatable into a T-SQL query, the applications seem very diverse. So, I'll post my code here and hopefully a generous SO denizen can tell me what I'm doing wrong and why.
The Code
public IEnumerable<WordIndexModel> GetWordIndex(int transid)
{
return (from trindex in context.transIndexes
let trueWord = IsWord(trindex)
join trans in context.Transcripts on trindex.transLineUID equals trans.UID
group new { trindex, trans } by new { TrueWord = trueWord, trindex.transID } into grouped
orderby grouped.Key.word
where grouped.Key.transID == transid
select new WordIndexModel
{
Word = TrueWord,
Instances = grouped.Select(test => test.trans).Distinct()
});
}
public string IsWord(transIndex trindex)
{
Match m = Regex.Match(trindex.word, #"^[a-z]+(\w*[-]*)*",
RegexOptions.IgnoreCase);
return m.Value;
}
With the above code I access a table, transIndex that is essentially a word index of culled from various user documents. The problem is that not all entries are actually words. Nubers, and even underscore lines, such as, ___________,, are saved as well.
The Problem
I'd like to keep only the words that my custom method IsWord returns (at the present time I have not actually developed the parsing mechanism). But as the IsWord function shows it will return a string.
So, using let I introduce my custom method into the query and use it as a grouping parameter, the is selectable into my object. Upon execution I get the omninous:
LINQ to Entities does not recognize the method
'System.String IsWord(transIndex)' method, and this
method cannot be translated into a store expression."
I also need to make sure that only records that match the IsWord condition are returned.
Any ideas?

It is saying it does not understand your IsWord method in terms of how to translate it to SQL.
Frankly it does not do much anyway, why not replace it with
return (from trindex in context.transIndexes
let trueWord = trindex.word
join trans in context.Transcripts on trindex.transLineUID equals trans.UID
group new { trindex, trans } by new { TrueWord = trueWord, trindex.transID } into grouped
orderby grouped.Key.word
where grouped.Key.transID == transid
select new WordIndexModel
{
Word = TrueWord,
Instances = grouped.Select(test => test.trans).Distinct()
});
What methods can EF translate into SQL, i can't give you a list, but it can never translate a straight forward method you have written. But their are some built in ones that it understands, like MyArray.Contains(x) for example, it can turn this into something like
...
WHERE Field IN (ArrItem1,ArrItem2,ArrItem3)
If you want to write a linq compatible method then you need to create an expresion tree that EF can understand and turn into SQL.
This is where things star to bend my mind a little but this article may help http://blogs.msdn.com/b/csharpfaq/archive/2009/09/14/generating-dynamic-methods-with-expression-trees-in-visual-studio-2010.aspx.

If the percentage of bad records in return is not large, you could consider enumerate the result set first, and then apply the processing / filtering?
var query = (from trindex in context.transIndexes
...
select new WordIndexModel
{
Word,
Instances = grouped.Select(test => test.trans).Distinct()
});
var result = query.ToList().Where(word => IsTrueWord(word));
return result;
If the number of records is too high to enumerate, consider doing the check in a view or stored procedure. That will help with speed and keep the code clean.
But of course, using stored procedures has disadvatages of reusability and maintainbility (because of no refactoring tools).
Also, check out another answer which seems to be similar to this one: https://stackoverflow.com/a/10485624/3481183

Using variables to build a LinQ query?

I don't think is possible but wanted to ask to make sure. I am currently debugging some software someone else wrote and its a bit unfinished.
One part of the software is a search function which searches by different fields in the database and the person who wrote the software wrote a great big case statement with 21 cases in it 1 for each field the user may want to search by.
Is it possible to reduce this down using a case statement within the Linq or a variable I can set with a case statement before the Linq statement?
Example of 1 of the Linq queries: (Only the Where is changing in each query)
var list = (from data in dc.MemberDetails
where data.JoinDate.ToString() == searchField
select new
{
data.MemberID,
data.FirstName,
data.Surname,
data.Street,
data.City,
data.County,
data.Postcode,
data.MembershipCategory,
data.Paid,
data.ToPay
}
).ToList();
Update / Edit:
This is what comes before the case statement:
string searchField = txt1stSearchTerm.Text;
string searchColumn = cmbFirstColumn.Text;
switch (cmbFirstColumn.SelectedIndex + 1)
{
The cases are then done by the index of the combo box which holds the list of field names.

Given that where takes a predicate, you can pass any method or function which takes MemberDetail as a parameter and returns a boolean, then migrate the switch statement inside.
private bool IsMatch(MemberDetail detail)
{
// The comparison goes here.
}
var list = (from data in dc.MemberDetails
where data => this.IsMatch(data)
select new
{
data.MemberID,
data.FirstName,
data.Surname,
data.Street,
data.City,
data.County,
data.Postcode,
data.MembershipCategory,
data.Paid,
data.ToPay
}
).ToList();
Note that:
You may look for a more object-oriented way to do the comparison, rather than using a huge switch block.
An anonymous type with ten properties that you use in your select is kinda weird. Can't you return an instance of MemberDetail? Or an instance of its base class?

How are the different where statements handled, are they mutually excluside or do they all limit the query somehow?
Here is how you can have one or more filters for a same query and materialized after all filters have been applied.
var query = (from data in dc.MemberDetails
select ....);
if (!String.IsNullOrEmpty(searchField))
query = query.Where(pr => pr.JoinDate.ToString() == searchField);
if (!String.IsNullOrEmpty(otherField))
query = query.Where(....);
return query.ToList();

Making a SQL database table browser using Entity Framework in c# winforms

I'm just looking for some links or tips for some general direction. I'm writing a program where the user will have access to many different tables in a sql server database. For example, if a user clicks on the "Foo" button, it will bring up a gui dialog which displays all of the columns of the "Foo" table. It also has a textbox at the top for filtering through data in the columns. For the most part, what i've written so far works atleast decent, but there are times when the performance is really slow and I can't help but feel I'm doing something wrong. Generally my code goes like this.
//GUI Constructor will call a Load Function
private void Load()
{
//Context is a DbContext entity
var query = from q in context.Foo
select q;
datagrid.Datasource = query.ToList();
}
The part which I feel goes wrong is with the textbox filter searching. The way I do it right now is to basically re-query the database to get more specific rows. This function gets called on the TextBox TextChanged event. I know this would be bad to call every keypress, I was going to add a timer to wait for the user to stop typing before applying the filter but anyways this is the code though.
private void TextFilter()
{
var query = from q in context.Foo.Where( x => x.Name == FilterTextbox.Text )
select q;
datagrid.Datasource = query.ToList();
}
I would assume it'd be better to store the entire database from the load function into a list that way all of the info is in the programs memory already, but this actually was slower than just querying the database again. I also tried using Context.Foo.Local and querying off of that but it proved to be as slow as storing all the data in my own List
private void AlternateLoad()
{
context.Foo.Load();
datagrid.Datasource = context.Foo.Local.ToList();
}
private void AlternateTextSearch()
{
var query = from q in context.Foo.Local.Where( x => x.Name == FilterTextBox.Text )
select q;
datagrid.Datasource = query.ToList();
}
I've experimented with AsParallel() when using Local or my own List but it doesn't seem to make a difference. Anyways I just want to see how to speed this up. In one specific scenario, I was prefiltering the database before displaying the data, and the prefilter took about 19 seconds before it could display its 7 row result. The smaller tables are fine but the tables with 100k+ rows definitely reveal the weakness of the code. Any tips or just general links to on how to do this would be greatly appreciated. I've been searching all over and I have had no luck in finding anything.
Thanks very much!

It's really up to you to estimate what the best solution is for the application, but the following should be some solid pointers to continue:
Don't return data that you don't need: top 100 rows / Paging
LINQ query ==> .First() .Skip() .Take()
In memory filtering can be very fast too! Keep the entire list in memory and run linq query against this collection...
Create StoredProcedures with a parameter to filter on and
put an index on the columns you must likely want to filter on.

I would suggest adding a private member variable to the class(a list):
public MyClass
{
private List<Foo> myFoo = new List<Foo>();
//GUI Constructor will call a Load Function
private void Load()
{
//Context is a DbContext entity
var query = from q in context.Foo
select q;
// set the private member variable "myFoo" here
datagrid.Datasource = myFoo = query.ToList();
}
private void TextFilter()
{
// don't requery the DB, query the list myFoo
//var query = from q in context.Foo.Where( x => x.Name == FilterTextbox.Text )
//select q;
var query = myFoo.Where(x => x.Column == SearchCriteria).ToList();
datagrid.Datasource = query;
}
}
Note: The above code isn't tested or anything, hopefully point you in the right direction.
Also Important: Because you're not requering the database the data in the list could become out of date.

EF4 - Linq to SQLite and List.Contains strange behaviour

I have some issue querying my local sqlite-database. Below is the current code:
public static Dictionary<Guid, string> GetHashedComanyMembers(List<Guid> IdsToSearch)
{
using (var dbContext = new DbEntities(GetConnectionString()))
{
var test = dbContext.<myTable>.Where(cm => IdsToSearch.Contains(cm.IdToFind));
}
}
The variable test contains an emtpt dataset. And now to my question.
If is set a variable
var firstGuid = IdsToSearch[0]
and i change the next line to
var test = dbContext.<myTable>.Where(cm => cm.IdToFind == firstGuid);
test contains the two sets i want to have.
If i let me show the (first) generated statement and run it on the database directly it also fails to find something. But the (second) statement succeeds.
So does anyone have any idea where i'm going wrong or could point out the error?
Thanks in advance, Steve
The first version:
SELECT
[Extent1].[ValueId] AS [ValueId],
[Extent1].[ValueFirstName] AS [ValueFirstName],
[Extent1].[ValueLastName] AS [ValueLastName],
[Extent1].[ValueLastChanged] AS [ValueLastChanged],
[Extent1].[ValueLastChangedBy] AS [ValueLastChangedBy]
FROM [tblValue] AS [Extent1]
WHERE '28826de0-27ea-42ab-9104-234ee289de16' = [Extent1].[ValueId]
Second:
SELECT
[Extent1].[ValueId] AS [ValueId],
[Extent1].[ValueFirstName] AS [ValueFirstName],
[Extent1].[ValueLastName] AS [ValueLastName],
[Extent1].[ValueLastChanged] AS [ValueLastChanged],
[Extent1].[ValueLastChangedBy] AS [ValueLastChangedBy]
FROM [tblValue] AS [Extent1]
WHERE [Extent1].[ValueId] = #p__linq__0
Edit
If i call th ToList() method on my table before the Where() everything is fine because the whole table-content will be enumerated an i call the Contains on this list. But that can't be the solution to Select the whole table first.
So i think the problem is the conversation done by Linq because the generated statements checks agains the Guid as string but it is internally stored as Blob.
In the second statement the guid is not set directly. There is an placeholder that will be replaced by DBMS? And this query is alright.
So any ideas where to start?

C#, Linq2SQL - tricks to fetch a ViewModel object with relation data?

I don't know Linq2Sql so well yet and I was wondering if there is a trick for this probably common MVVM scenario. I have Linq2Sql data context containing Domain models, but I am fetching data for my customized ViewModel object from it.
var query = from ord in ctx.Table_Orders
select new OrderViewModel()
{
OrderId = ord.OrderId,
OrderSum = ord.OrderSum,
OrderCurrencyId = ord.OrderCurrencyId,
OrderCurrencyView = ord.Currency.CurrencyText
};
So i want my ViewModel to inculde both CurrencyId from domain object and the CurrencyText from related table to show it nicely in the View.
This code works great. It generates one DB call with join to fetch the CurrencyText. But the model is simplified, real one has many more fields. I want to make the code reusable because I have many different queries, that returns the same ViewModel. Now every minor change to OrderViewModel requires lots of maintainance.
So I moved the code to OrderViewModel itself as a constructor.
public OrderViewModel(Table_Order ord)
{
OrderId = ord.OrderId,
OrderSum = ord.OrderSum,
OrderCurrencyId = ord.OrderCurrencyId,
OrderCurrencyView = ord.Currency.CurrencyText
}
And call it like this.
var query = from ord in ctx.Table_Orders
select new OrderViewModel(ord);
The Problem: The join is gone DB query is no more optimised. Now I get 1+N calls to database to fetch CurrencyText for every line.
Any comments are welcome. Maybe I have missed different great approach.
This is how far i could get on my own, to get the code reusability. I created a function that does the job and has multiple parameters. Then I need to explicitly pass it everything that has crossed the line of entity.
var query = ctx.Table_Orders.Select(m =>
newOrderViewModel(m, m.Currency.CurrencyText));
The DB call is again optimized. But it still does not feel like I am there yet! What tricks do You know for this case?
EDIT : The final solution
Thanks to a hint by #Muhammad Adeel Zahid I arrived at this solution.
I created an extension for IQueryable
public static class Mappers
{
public static IEnumerable<OrderViewModel> OrderViewModels(this IQueryable<Table_Order> q)
{
return from ord in q
select new OrderViewModel()
{
OrderId = ord.OrderId,
OrderSum = ord.OrderSum,
OrderCurrencyId = ord.OrderCurrencyId,
OrderCurrencyView = ord.Currency.CurrencyText
};
}
}
Now i can do this to get all list
var orders = ctx.Table_Order.OrderViewModels().ToList();
or this to get a single item, or anything in between with Where(x => ..)
var order = ctx.Table_Order
.Where(x => x.OrderId == id).OrderViewModels().SingleOrDefault();
And that completely solves this question. The SQL generated is perfect and the code to translate objects is reusable. Approach like this should work with both LINQ to SQL and LINQ to Entities. (Not tested with the latter) Thank You again #Muhammad Adeel Zahid

Whenever we query the database, we mostly require either enumeration of objects (more than one records in db) or we want a single entity (one record in db). you can write your mapping code in method that returns enumeration for whole table like
public IEnumerable<OrderViewModel> GetAllOrders()
{
return from ord in ctx.Table_Orders
select new OrderViewModel()
{
OrderId = ord.OrderId,
OrderSum = ord.OrderSum,
OrderCurrencyId = ord.OrderCurrencyId,
OrderCurrencyView = ord.Currency.CurrencyText
};
}
Now you may want to filter these records and return another enumeration for example on currencyID
public IEnumerable<OrderViewModel> GetOrdersByCurrency(int CurrencyID)
{
return GetAllOrders().Where(x=>x.CurrencyId == CurrencyID);
}
Now you may also want to find single record out of all these view models
public OrderViewModel GetOrder(int OrderID)
{
return GetAllOrders().SingleOrDefault(x=>x.OrderId == OrderID);
}
The beauty of IEnumerable is that it keeps adding conditions to query and does not execute it until it is needed. so your whole table will not be loaded unless you really want it and you have kept your code in single place. Now if there are any changes in ViewModel Mapping or in query itself, it has to be done in GetAllOrders() method, rest of code will stay unchanged

You can avoid the N+1 queries problem by having Linq2SQL eagerly load the referenced entites you need to construct your viewmodels. This way you can build one list of objects (and some referenced objects) and use it to construct everything. Have a look at this blog post.
One word of warning though: This technique (setting LoadOptions for the Linq2SQL data context) can only be done once per data context. If you need to perform a second query with a different eager loading configuration, you must re-initalize your data context. I automated this with a simple wrapper class around my context.