I want to write a LINQ query which returns two streams of objects. In F# I would write a Seq expression which creates an IEnumerable of 2-tuples and then run Seq.unzip. What is the proper mechanism to do this in C# (on .NET 3.5)?
Cheers, Jurgen
Your best bet is probably to create a Pair<T1, T2> type and return a sequence of that. (Or use an anonymous type to do the same thing.)
You can then "unzip" it with:
var firstElements = pairs.Select(pair => pair.First);
var secondElements = pairs.Select(pair => pair.Second);
It's probably worth materializing pairs first though (e.g. call ToList() at the end of your first query) to avoid evaluating the query twice.
Basically this is exactly the same as your F# approach, but with no built-in support.
Due to the lack of tuples in C# you may create an anonymous type.
Semantics for this are:
someEnumerable.Select( inst => new { AnonTypeFirstStream = inst.FieldA, AnonTypeSecondStream = inst.FieldB });
This way you're not bound in the amount of streams you return, you can just add a field to the anonymous type pretty like you can add an element to a tuple.
Related
I have a list of anonymous objects, that each anonymous object has the same members, and I create this list by something like this :
var listWorthies = balanceWorthies.Select(w => new
{
OwnerName = w.OwnerOnInquery,
OwnerDocDate = w.OwnerDocDate,
}).ToList();
Now I just wanted to convert the OwnerDocDate member of each dynamic object.
something like this :
var listWorthies = balanceWorthies.Select(w => new
{
OwnerName = w.OwnerOnInquery,
OwnerDocDate = ConvertDate(w.OwnerDocDate),
}).ToList();
gives me the error
LINQ to Entities does not recognize the method 'System.String ConvertDate(System.DateTime)' method, and this method cannot be translated into a store expression.
after creating the list, I tried somethings like these ones too :
foreach (dynamic Worthy in listWorthies)
{
DateTime OwnerDocDate_2 = ConvertDate(Worthy.OwnerDocDate);
Worthy.AddProperty(OwnerDocDate_2);
}
but it gives me this error :
'<>f__AnonymousType8<string,string,System.DateTime,System.DateTime,System.Guid,string>' does not contain a definition for 'AddProperty'
UPDATE
the output of function ConvertDate is String, and I want to change the type of OwnerDocDate from datetime to string.
how can I solve this?
You forgot to tell us that balanceWorthies is an IQueryable<...>, not an IEnumerable<...>.
To understand why this matters, you must understand the difference between an IQueryable<...>, and an IEnumerable<...>.
IEnumerable
An object of a class that implements IEnumerable<...>, represents a sequence. You can get the first element, and once you've got an element, you can get the next element, as long as there are elements.
At its lowest level this is done using methode GetEnumerator() / MoveNext() / Current:
IEnumerable<Cusotmer> customers = ...
IEnumerator<Customer> enumerator = customers.GetEnumerator();
while (enumerator.MoveNext())
{
// There is still a Customer in the sequence:
Customer customer = enumerator.Current;
ProcessCustomer(customer);
}
foreach will deep inside do this.
If you look at the LINQ methods, you will see that there are two groups of methods: the ones that return IEnumerable<...> and the others. The first group won't enumerate the sequence. We say that they use lazy-execution or deferred-execution. In the description of these LINQ methods, you'll find this term in the remarks section.
Concatenating methods of this group isn't expensive: the query is not executed, only the Enumerator is adjusted.
The LINQ methods that return something else than IEnumerble<...> will execute the sequence. Deep inside GetEnumerator / MoveNext / Current is called to access the elements of the source sequence one by one.
IQueryable
An object of a class that implements IQueryable<...> doesn't represent an enumerable sequence, it represents the potential to fetch an enumerable sequence.
To do this, the class holds an Expression and a Provider. The Expression holds what data must be fetched in some generic format. The Provider knows where the data must be fetched (usually a Database Management System) and what language is used to communicate with this DBMS (usually SQL).
When you ask the IQueryable to get the enumerator, the Expression is sent to the Provider, who will translate the Expression into SQL and fetch the data at the DBMS. The fetched data is presented as an IEnumerator<...>. The caller can use MoveNext() / Current to access the fetched elements one by one.
Back to your question
The Provider doesn't know your own methods. Hence it doesn't know how to translate them into SQL. In fact, there are several standard LINQ methods that are not supported by entity framework. See Supported and unsupported LINQ methods
Your compiler doesn't know how smart your Provider is, so he can't complain. You'll get the error at runtime.
The easiest way to solve your problem is by transferring the data to your local process and let your local process execute the methods as if it was an IEnumerable<...>.
This is done using the method Enumerable.AsEnumerable. As transferring data is expensive, it is wise to limit the data being transferred to a minimum before you call AsEnumerable. So first do all your Where, (Group-)Join, etc. everything that limits the amount of transferred data.
var listWorthies = balanceWorthies.Select(w => new
{
OwnerName = w.OwnerOnInquery,
OwnerDocDate = w.OwnerDocDate,
})
// OwnerDocDate has the type of balanceWorthy.OwnerDocDate
// move the data to local process, so you can ConvertDate
.AsEnumerable()
// now you can call your own methods:
.Select(fetchedItem => new
{
OwnerName = fetchedItem.OwnderName,
OwnerDocDate = ConvertDate(fetchedItem.OwnerDocDate),
});
Database management systems are extremely optimized in selecting data. It seems that ConvertDate will only translate the data into a different format. So you won't lose a lot of efficiencies.
If in other cases the method will change the amount of selected data, try to change the expression such that the DBMS can handle it, especially if it is before a Where. If the DBMS must execute your code, and you can't translate the LINQ into something that your provider supports, you'll have to write an extension method that changes the Expression. How to do that is something for a different question.
I have some function (X) which return IQueryable with "OrderBy(x => x.Name)".
I want to use X function in new code, but in this code there is parameter which determine the order (asc/desc).
I prefer not to change (X) since it used in multiple places already.
there is option to get X(), then cancel it's "OrderBy", and apply new order?
(now it throws exception, since it is like Y.OrderBy(a => a.Name).OrderByDescending(a => a.Name))
"A column has been specified more than once in the order by list. Columns in the order by list must be unique.\r\nStatement(s) could not be prepared."
use
public void x(string order = string.empty)
{
Y.OrderBy(a => a.Name);
if (order == "desc")
{
Y = Y.Reverse();
}
}
I've tried to wiggle Expression object of IOrderedQueryable that your method is actually returning (am I wrong?) but so far no luck, and I can't see the option to manipulate System.Linq.Expression or System.Linq.EnumerableQuery easy enough to use it instead of changing X() method.
IMHO, you should choose from:
create somes kind of wrapper method using the Reverse() (#Eric Lizotte answer) or OrderByDescending() - it will be less time consuming I guess but for grouping, filtering etc. you will have to create brand new function.
remove OrderBy() from X() function body, and modify your old code - returning non-ordered result will give you more flexibility and opportunity to do what you want to do with the result via linq expressions in each scenario that you'll came across.
Can you clarify for me if you're talking about "function" you have in mind a sql function that is mapped via ORM (eg. Entity Framework) in your application or just a ordinary method which is using your custom provider to return IQueryable?
I am using the Entity Framework and want to put the elements of an IQueryable into an array of a converted type.
Having NormalType as first type and ConvertedType as the converted one, the "conversion" is done this way:
//Creates a NormalType object based on ConvertedType instance.
NormalType nt = new NormalType (ctInstance);
// Returns a new ConvertedType instance,
//based on the content of the NormalType instance.
ConvertedType ct = nt.getConvertedType();
Now, I want to convert a list of type NormalType into a ConvertedType array
(IQueryable<NormalType> -> ConvertedType[]).
My first thought is to classically iterate through the IQueryable<NormalType>, put the data into a List<ConvertedType> and then convert the second list to an array using ToArray() method:
//Rest of code....
List<ConvertedType> convertedTypeList = new List<ConvertedType>();
//normalTypeIQueryable is of type IQueryable<NormalType>
foreach(NormalType normalType in normalTypeIQueryable)
{
convertedTypeList.Add(normalType.getConvertedType());
}
ConvertedType[] convertedTypeArray = convertedTypeList.ToArray();
My second thought is to do it using ToList(), ConvertAll(...) and ToArray() in a one-liner:
normalTypeIQueryable.ToList().ConvertAll(n => n.GetConvertedType()).ToArray();
I think that the last one will lead into more than one loops through the conversions though.
Which of those two ways is better in terms of performance? Is there any even better alternative?
Thank you in advance.
UPDATE: The GetConvertedType() creates a new instance of a converted type based on the data the NormalType contains, in a "deep-copy" manner.
This is pretty straightforward. What you want to do is project the values inside the source sequence into different values, then put the resulting sequence into an array.
LINQ does projection with Select, so the simplest, most natural, and (at least as long as you are going to use LINQ) most performant way to write it is:
var result = input.AsEnumerable().Select(i => i.GetConvertedType()).ToArray();
This is effectively equivalent to the manual foreach loop in your example.
How can I sort an array of strings using the OrderBy function? I saw I need to implement some interfaces...
You can sort the array by using.
var sortedstrings = myStringArray.OrderBy( s => s );
This will return an instance of Ienumerable. If you need to retain it as a array, use this code instead.
myStringArray = myStringArray.OrderBy( s => s ).ToArray();
I'm not sure what you are referring to when you said that you have to implement some interfaces, but you do not have to do this when using the IEnumerable.OrderBy. Simply pass a Func<TSource, TKey> in the form of a lambda-expression.
OrderBy won't sort the existing array in place. If you need to do that, use Array.Sort.
OrderBy always returns a new sequence - which of course you can convert to an array and store a reference to in the original variable, as per Øyvind's answer.
To sort inside an existing array, call Array.Sort(theArray).
Re your comment on interfaces: you don't need to add any interfaces here, since string is well supported; but for custom types (of your own) you can implement IComparable / IComparable<T> to enable sorting. You can also do the same passing in an IComparer / IComparer<T>, if you want (or need) the code that provides the ordering to be separate to the type itself.
Linq has two (syntax) ways to sort an array of strings.
1:
string[] sortedStrings = unsortedStrings.OrderBy(s => s).ToArray();
This syntax is using a Lambda Expressions if you don't know what s => s means.
2:
sortedStrings = (from strings in unsortedStrings
orderby strings
select strings).ToArray();
This example looks a bit like a SQL statement and is probably easier to read if you are new with Linq.
ToArray() converts the IOrderedEnumerable<string> to as string[] in this case.
This is probably a stupid question, but here goes. I would like to be able to dynamically construct a predicate < T > from a string parsed from a database VARCHAR column, or any string, for that matter. For example, say the column in the database contained the following string:
return e.SomeStringProperty.Contains("foo");
These code/string values would be stored in the database knowing what the possible properties of the generic "e" is, and knowing that they had to return a boolean. Then, in a magical, wonderful, fantasy world, the code could execute without knowing what the predicate was, like:
string predicateCode = GetCodeFromDatabase();
var allItems = new List<SomeObject>{....};
var filteredItems = allItems.FindAll(delegate(SomeObject e) { predicateCode });
or Lambda-ized:
var filteredItems = allItems.FindAll(e => [predicateCode]);
I know it can probably never be this simple, but is there a way, maybe using Reflection.Emit, to create the delegate code dynamically from text and give it to the FindAll < T > (or any other anonymous/extension) method?
The C# and VB compilers are available from within the .NET Framework:
C# CodeDom Provider
Be aware though, that this way you end up with a separate assembly (which can only be unloaded if it's in a separate AppDomain). This approach is only feasible if you can compile all the predicates you are going to need at once. Otherwise there is too much overhead involved.
System.Reflection.Emit is a great API for dynamically emitting code for the CLR. It is, however, a bit cumbersome to use and you must learn CIL.
LINQ expression trees are an easy to use back-end (compilation to CIL) but you would have to write your own parser.
I suggest you have a look at one of the "dynamic languages" that run on the CLR (or DLR) such as IronPython. It's the most efficient way to implement this feature, if you ask me.
Check out the Dynamic Linq project it does all this and more!
http://weblogs.asp.net/scottgu/archive/2008/01/07/dynamic-linq-part-1-using-the-linq-dynamic-query-library.aspx
Great for simple stuff like user selected orderby's or where clauses
It is possible using emit, but you'd be building your own parser.
EDIT
I remember that in ScottGu's PDC keynote, he showed a feature using the CLI version of the .net framework that resembled Ruby's eval, but I can't find a URL that can corroborate this. I'm making this a commnity wiki so that anyone who has a good link can add it.
I stepped off the dynamic linq because it's limited in ways I want to search a collection, unless you prove me wrong.
My filter needs to be: in a list of orders, filter the list so that I have only the orders with in the collection of items in that order, an item with the name "coca cola".
So that will result to a method of: orders.Findall(o => o.Items.Exists(i => i.Name == "coca cola"))
In dynamic linq I didn't find any way to do that, so I started with CodeDomProvicer.
I created a new Type with a method which contains my dynamically built FindAll Method:
public static IList Filter(list, searchString)
{
// this will by dynamically built code
return orders.Findall(o => o.Items.Exists(i => i.Name == "coca cola"));
}
when I try to build this assembly:
CompilerResults results = provider.CompileAssemblyFromSource(parameters, sb.ToString());
I'm getting the error:
Invalid expression term ">"
Why isn't the compiler able to compile the predicate?