I get the JSON from another external vendor and I have simplyfied the same here.
Problem : I am searching for instrumentIdentifier in Holding in the json, have to extract it , it can appear at level 2 or level 3 or level 4 in the assetcategory.
I am not sure how to search it properly using linq or normal C# methods, I don't have latest newtonsoft to query based on JsonPath.
Stuck badly using Linq or even normal method,
.net version is 4.0 , newtonsoft 4.5
Use recursion:
public holdings FindHoldings(portfolio portfolio, string instrumentId)
{
return FindHoldingsRecursive(portfolio.assetTypes, instrumentId);
}
public holdings FindHoldingsRecursive(
IEnumerable<subAssetType> assetTypes,
string instrumentId)
{
if (assetTypes == null)
return null;
return assetTypes
.Select(a => FindHoldingsRecursive(a, instrumentId))
.FirstOrDefault(h => h != null);
}
public holdings FindHoldingsRecursive(
subAssetType assetType,
string instrumentId)
{
return
assetType.holdings.FirstOrDefault(h => h.instrumentIdentifier == instrumentId);
?? FindHoldingsRecursive(assetType.assetTypes, instrumentId);
}
This will do a depth-first search.
If you want a more generic solution to traversing a tree structure, I'd created these extension method for my own benefit:
public static class EnumerableExtensions
{
public static IEnumerable<T> OrEmpty<T>(this IEnumerable<T> collection)
{
return collection ?? Enumerable.Empty<T>();
}
public static IEnumerable<T> Recurse<T>(
this IEnumerable<T> collection,
Func<T, IEnumerable<T>> childrenSelector)
{
return collection.SelectMany(i => i.Recurse(childrenSelector));
}
public static IEnumerable<T> Recurse<T>(
this T parent,
Func<T, IEnumerable<T>> childrenSelector)
{
yield return parent;
var children = childrenSelector(parent).OrEmpty();
foreach (var descendant in children.Recurse(childrenSelector))
{
yield return descendant;
}
}
}
This will let you do this:
var theHolding = portfolio.assetTypes
.Recurse(a => a.assetTypes)
.SelectMany(a => a.holdings.OrEmpty())
.FirstOrDefault(h => h.instrumentIdentifier == "foo");
Related
I need to search a tree for data that could be anywhere in the tree. How can this be done with linq?
class Program
{
static void Main(string[] args) {
var familyRoot = new Family() {Name = "FamilyRoot"};
var familyB = new Family() {Name = "FamilyB"};
familyRoot.Children.Add(familyB);
var familyC = new Family() {Name = "FamilyC"};
familyB.Children.Add(familyC);
var familyD = new Family() {Name = "FamilyD"};
familyC.Children.Add(familyD);
//There can be from 1 to n levels of families.
//Search all children, grandchildren, great grandchildren etc, for "FamilyD" and return the object.
}
}
public class Family {
public string Name { get; set; }
List<Family> _children = new List<Family>();
public List<Family> Children {
get { return _children; }
}
}
That's an extension to It'sNotALie.s answer.
public static class Linq
{
public static IEnumerable<T> Flatten<T>(this T source, Func<T, IEnumerable<T>> selector)
{
return selector(source).SelectMany(c => Flatten(c, selector))
.Concat(new[] { source });
}
}
Sample test usage:
var result = familyRoot.Flatten(x => x.Children).FirstOrDefault(x => x.Name == "FamilyD");
Returns familyD object.
You can make it work on IEnumerable<T> source too:
public static IEnumerable<T> Flatten<T>(this IEnumerable<T> source, Func<T, IEnumerable<T>> selector)
{
return source.SelectMany(x => Flatten(x, selector))
.Concat(source);
}
Another solution without recursion...
var result = FamilyToEnumerable(familyRoot)
.Where(f => f.Name == "FamilyD");
IEnumerable<Family> FamilyToEnumerable(Family f)
{
Stack<Family> stack = new Stack<Family>();
stack.Push(f);
while (stack.Count > 0)
{
var family = stack.Pop();
yield return family;
foreach (var child in family.Children)
stack.Push(child);
}
}
Simple:
familyRoot.Flatten(f => f.Children);
//you can do whatever you want with that sequence there.
//for example you could use Where on it and find the specific families, etc.
IEnumerable<T> Flatten<T>(this T source, Func<T, IEnumerable<T>> selector)
{
return selector(source).SelectMany(c => Flatten(selector(c), selector))
.Concat(new[]{source});
}
So, the simplest option is to write a function that traverses your hierarchy and produces a single sequence. This then goes at the start of your LINQ operations, e.g.
IEnumerable<T> Flatten<T>(this T source)
{
foreach(var item in source) {
yield item;
foreach(var child in Flatten(item.Children)
yield child;
}
}
To call simply: familyRoot.Flatten().Where(n => n.Name == "Bob");
A slight alternative would give you a way to quickly ignore a whole branch:
IEnumerable<T> Flatten<T>(this T source, Func<T, bool> predicate)
{
foreach(var item in source) {
if (predicate(item)) {
yield item;
foreach(var child in Flatten(item.Children)
yield child;
}
}
Then you could do things like: family.Flatten(n => n.Children.Count > 2).Where(...)
I like Kenneth Bo Christensen's answer using stack, it works great, it is easy to read and it is fast (and doesn't use recursion).
The only unpleasant thing is that it reverses the order of child items (because stack is FIFO). If sort order doesn't matter to you then it's ok.
If it does, sorting can be achieved easily using selector(current).Reverse() in the foreach loop (the rest of the code is the same as in Kenneth's original post)...
public static IEnumerable<T> Flatten<T>(this T source, Func<T, IEnumerable<T>> selector)
{
var stack = new Stack<T>();
stack.Push(source);
while (stack.Count > 0)
{
var current = stack.Pop();
yield return current;
foreach (var child in selector(current).Reverse())
stack.Push(child);
}
}
Well, I guess the way is to go with the technique of working with hierarchical structures:
You need an anchor to make
You need the recursion part
// Anchor
rootFamily.Children.ForEach(childFamily =>
{
if (childFamily.Name.Contains(search))
{
// Your logic here
return;
}
SearchForChildren(childFamily);
});
// Recursion
public void SearchForChildren(Family childFamily)
{
childFamily.Children.ForEach(_childFamily =>
{
if (_childFamily.Name.Contains(search))
{
// Your logic here
return;
}
SearchForChildren(_childFamily);
});
}
I have tried two of the suggested codes and made the code a bit more clear:
public static IEnumerable<T> Flatten1<T>(this T source, Func<T, IEnumerable<T>> selector)
{
return selector(source).SelectMany(c => Flatten1(c, selector)).Concat(new[] { source });
}
public static IEnumerable<T> Flatten2<T>(this T source, Func<T, IEnumerable<T>> selector)
{
var stack = new Stack<T>();
stack.Push(source);
while (stack.Count > 0)
{
var current = stack.Pop();
yield return current;
foreach (var child in selector(current))
stack.Push(child);
}
}
Flatten2() seems to be a little bit faster but its a close run.
Some further variants on the answers of It'sNotALie., MarcinJuraszek and DamienG.
First, the former two give a counterintuitive ordering. To get a nice tree-traversal ordering to the results, just invert the concatenation (put the "source" first).
Second, if you are working with an expensive source like EF, and you want to limit entire branches, Damien's suggestion that you inject the predicate is a good one and can still be done with Linq.
Finally, for an expensive source it may also be good to pre-select the fields of interest from each node with an injected selector.
Putting all these together:
public static IEnumerable<R> Flatten<T,R>(this T source, Func<T, IEnumerable<T>> children
, Func<T, R> selector
, Func<T, bool> branchpredicate = null
) {
if (children == null) throw new ArgumentNullException("children");
if (selector == null) throw new ArgumentNullException("selector");
var pred = branchpredicate ?? (src => true);
if (children(source) == null) return new[] { selector(source) };
return new[] { selector(source) }
.Concat(children(source)
.Where(pred)
.SelectMany(c => Flatten(c, children, selector, pred)));
}
I'm trying to write an extension method that is supposed to traverse an object graph and return all visited objects.
I'm not sure if my approach is the best, so please do comment on that. Also yield is frying my brain... I'm sure the answer is obvious :/
Model
public class MyClass
{
public MyClass Parent {get;set;}
}
Method
public static IEnumerable<T> SelectNested<T>
(this T source, Func<T, T> selector)
where T : class
{
yield return source;
var parent = selector(source);
if (parent == null)
yield break;
yield return SelectNestedParents(parent, selector).FirstOrDefault();
}
Usage
var list = myObject.SelectNested(x => x.Parent);
The problem
It's almost working. But it only visits 2 objects. It self and the parent.
So given this graph c -> b -> a starting from c. c, b is returned which is not quite what I wanted.
The result I'm looking for is b, c
In the last line of SelectNested you only return the first parent:
yield return SelectNestedParents(parent, selector).FirstOrDefault();
You have to return all parents:
foreach (var p in SelectNestedParents(parent, selector))
return p;
Instead of using recursion you can use iteration which probably is more efficient:
public static IEnumerable<T> SelectNested<T>(this T source, Func<T, T> selector)
where T : class {
var current = source;
while (current != null) {
yield return current;
current = selector(current);
}
}
The following code should work as expected:
public static IEnumerable<T> SelectNested<T>()
{
if (source != null){
yield return source;
var parent = selector(source);
// Result of the recursive call is IEnumerable<T>
// so you need to iterate over it and return its content.
foreach (var parent in (SelectNested(selector(source))))
{
yield return parent;
}
}
}
Strictly speaking, your class looks to be a list, not a graph, since selector returns only one object not an enumeration of them. Thus recursion is not necessary.
public static IEnumerable<T> SelectNested<T>(this T source, Func<T, T> selector)
where T : class
{
while (source != null)
{
yield return source;
source = selector(source);
}
}
I have a utility method that processes a collection of strings, let's say:
public static string MyJoin(this IEnumerable<string> strings)
{
return string.Join(Environment.NewLine, strings);
}
I'd like to be able to also process IEnumerable<IEnumerable<string>> and IEnumerable<IEnumerable<IEnumerable<string>>> and so on, all of them in this way:
public static string MyJoin(this IEnumerable<IEnumerable<string>> strings)
{
return strings.Select(x => x.MyJoin()).MyJoin();
}
public static string MyJoin(this IEnumerable<IEnumerable<IEnumerable<string>>> strings)
{
return strings.Select(x => x.MyJoin()).MyJoin();
}
Is there any way I can elegantly express this without hardcoding all the possible levels I might want to use?
I've tried to do it with generics, but I just can't figure out the right constraints (if there are any).
I'd also like to avoid declaring a new type, if at all possible.
Performance, on the other hand, is not critical.
Based on Stefan's solution, simplified and compilation-fixed version
public static string MyJoin(this IEnumerable items)
{
if (items is IEnumerable<string>)
{
return string.Join(Environment.NewLine, (IEnumerable<string>)items);
}
if (items is IEnumerable<IEnumerable>)
{
return items.Cast<IEnumerable>().Select(x => x.MyJoin())).MyJoin();
}
throw new InvalidOperationException("Type is not a (nested) enumarable of strings");
}
Be careful with IEnumerable<T> and strings, because a string is an IEnumerable<T>, where T is char.
Static implementation using multiple overloads:
public static string MyJoin(this IEnumerable<IEnumerable<IEnumerable<string>>> items)
{
return items.SelectMany(x => x).MyJoin();
}
public static string MyJoin(this IEnumerable<IEnumerable<string>> items)
{
return items.SelectMany(x => x).MyJoin();
}
public static string MyJoin(this IEnumerable<string> strings)
{
return string.Join(Environment.NewLine, strings);
}
You cannot have a single generic method and "unwrap" the generic argument dynamically.
Implementation using reflection:
public static string MyJoin<T>(this IEnumerable<T> items)
{
if (typeof(T) == typeof(string)
{
return items.Cast<string>().MyStringJoin();
}
var innerIEnumerableType = typeof(T).GetInterfaces().FirstOrDefault(x => x.IsGeneric()
&& x.GetGenericType() == typeof(IEnumerable<>);
if (innerIEnumerableType != null)
{
// create generic method to
var method = typeof(ThisType).GetMethod("MyJoin", /* some flags */)
.MakeGenericMethod(innerIEnumerableType.GetGenericArguments().First())
// recursive call to generic method
return items.Select(x => (string)method.Invoke(null, x)).MyStringJoin();
}
throw new InvalidOperationException("Type is not a (nested) enumarable of strings")
}
public static string MyStringJoin(this IEnumerable<string> strings)
{
return string.Join(Environment.NewLine, strings);
}
To be honest, reflection isn't useful here. Why use generics, if it isn't typesafe anyway? At the end you could just as well use non-generic IEnumerable. It's much easier to implement. See Denis Itskovich's solution, he managed to write down what I actually tried to do.
Rather than having an IEnumerable of an unknown type, what you're really trying to do here is create a tree based structure. The most effective way of doing this is to create a Node class that can represent a value an its children.
public class Node
{
public Node(string value)
{
Value = value;
Children = Enumerable.Empty<Node>();
}
public Node(string value, IEnumerable<Node> children)
{
Value = value;
Children = children;
}
public string Value { get; private set; }
public IEnumerable<Node> Children { get; private set; }
}
This is now a much easier structure to traverse. Here is a general purpose tree traversal method which we can apply to this case:
public static IEnumerable<T> Traverse<T>(T item,
Func<T, IEnumerable<T>> childSelector)
{
var stack = new Stack<T>();
stack.Push(item);
while (stack.Any())
{
var next = stack.Pop();
yield return next;
foreach (var child in childSelector(next))
stack.Push(child);
}
}
var allValues = Traverse(rootNode, node => node.Children)
.Select(node => node.Value);
Partly inspired by Stefan's answer, I've reached a working solution that compiles without generics and (unfortunately) with mostly runtime type checking:
public static string MyJoin(this IEnumerable<string> strings)
{
return string.MyJoin(Environment.NewLine, strings);
}
public static string MyJoin(this IEnumerable<object> items)
{
if (items.All(x => x is string))
return items.Cast<string>().MyJoin();
if (items.All(x => x is IEnumerable<object>))
return items.Cast<IEnumerable<object>>().Select(x => x.MyJoin()).MyJoin();
throw new InvalidOperationException("The argument was not a nested enumerable of strings.");
}
Use the SelectMany extension method.
I need to search a tree for data that could be anywhere in the tree. How can this be done with linq?
class Program
{
static void Main(string[] args) {
var familyRoot = new Family() {Name = "FamilyRoot"};
var familyB = new Family() {Name = "FamilyB"};
familyRoot.Children.Add(familyB);
var familyC = new Family() {Name = "FamilyC"};
familyB.Children.Add(familyC);
var familyD = new Family() {Name = "FamilyD"};
familyC.Children.Add(familyD);
//There can be from 1 to n levels of families.
//Search all children, grandchildren, great grandchildren etc, for "FamilyD" and return the object.
}
}
public class Family {
public string Name { get; set; }
List<Family> _children = new List<Family>();
public List<Family> Children {
get { return _children; }
}
}
That's an extension to It'sNotALie.s answer.
public static class Linq
{
public static IEnumerable<T> Flatten<T>(this T source, Func<T, IEnumerable<T>> selector)
{
return selector(source).SelectMany(c => Flatten(c, selector))
.Concat(new[] { source });
}
}
Sample test usage:
var result = familyRoot.Flatten(x => x.Children).FirstOrDefault(x => x.Name == "FamilyD");
Returns familyD object.
You can make it work on IEnumerable<T> source too:
public static IEnumerable<T> Flatten<T>(this IEnumerable<T> source, Func<T, IEnumerable<T>> selector)
{
return source.SelectMany(x => Flatten(x, selector))
.Concat(source);
}
Another solution without recursion...
var result = FamilyToEnumerable(familyRoot)
.Where(f => f.Name == "FamilyD");
IEnumerable<Family> FamilyToEnumerable(Family f)
{
Stack<Family> stack = new Stack<Family>();
stack.Push(f);
while (stack.Count > 0)
{
var family = stack.Pop();
yield return family;
foreach (var child in family.Children)
stack.Push(child);
}
}
Simple:
familyRoot.Flatten(f => f.Children);
//you can do whatever you want with that sequence there.
//for example you could use Where on it and find the specific families, etc.
IEnumerable<T> Flatten<T>(this T source, Func<T, IEnumerable<T>> selector)
{
return selector(source).SelectMany(c => Flatten(selector(c), selector))
.Concat(new[]{source});
}
So, the simplest option is to write a function that traverses your hierarchy and produces a single sequence. This then goes at the start of your LINQ operations, e.g.
IEnumerable<T> Flatten<T>(this T source)
{
foreach(var item in source) {
yield item;
foreach(var child in Flatten(item.Children)
yield child;
}
}
To call simply: familyRoot.Flatten().Where(n => n.Name == "Bob");
A slight alternative would give you a way to quickly ignore a whole branch:
IEnumerable<T> Flatten<T>(this T source, Func<T, bool> predicate)
{
foreach(var item in source) {
if (predicate(item)) {
yield item;
foreach(var child in Flatten(item.Children)
yield child;
}
}
Then you could do things like: family.Flatten(n => n.Children.Count > 2).Where(...)
I like Kenneth Bo Christensen's answer using stack, it works great, it is easy to read and it is fast (and doesn't use recursion).
The only unpleasant thing is that it reverses the order of child items (because stack is FIFO). If sort order doesn't matter to you then it's ok.
If it does, sorting can be achieved easily using selector(current).Reverse() in the foreach loop (the rest of the code is the same as in Kenneth's original post)...
public static IEnumerable<T> Flatten<T>(this T source, Func<T, IEnumerable<T>> selector)
{
var stack = new Stack<T>();
stack.Push(source);
while (stack.Count > 0)
{
var current = stack.Pop();
yield return current;
foreach (var child in selector(current).Reverse())
stack.Push(child);
}
}
Well, I guess the way is to go with the technique of working with hierarchical structures:
You need an anchor to make
You need the recursion part
// Anchor
rootFamily.Children.ForEach(childFamily =>
{
if (childFamily.Name.Contains(search))
{
// Your logic here
return;
}
SearchForChildren(childFamily);
});
// Recursion
public void SearchForChildren(Family childFamily)
{
childFamily.Children.ForEach(_childFamily =>
{
if (_childFamily.Name.Contains(search))
{
// Your logic here
return;
}
SearchForChildren(_childFamily);
});
}
I have tried two of the suggested codes and made the code a bit more clear:
public static IEnumerable<T> Flatten1<T>(this T source, Func<T, IEnumerable<T>> selector)
{
return selector(source).SelectMany(c => Flatten1(c, selector)).Concat(new[] { source });
}
public static IEnumerable<T> Flatten2<T>(this T source, Func<T, IEnumerable<T>> selector)
{
var stack = new Stack<T>();
stack.Push(source);
while (stack.Count > 0)
{
var current = stack.Pop();
yield return current;
foreach (var child in selector(current))
stack.Push(child);
}
}
Flatten2() seems to be a little bit faster but its a close run.
Some further variants on the answers of It'sNotALie., MarcinJuraszek and DamienG.
First, the former two give a counterintuitive ordering. To get a nice tree-traversal ordering to the results, just invert the concatenation (put the "source" first).
Second, if you are working with an expensive source like EF, and you want to limit entire branches, Damien's suggestion that you inject the predicate is a good one and can still be done with Linq.
Finally, for an expensive source it may also be good to pre-select the fields of interest from each node with an injected selector.
Putting all these together:
public static IEnumerable<R> Flatten<T,R>(this T source, Func<T, IEnumerable<T>> children
, Func<T, R> selector
, Func<T, bool> branchpredicate = null
) {
if (children == null) throw new ArgumentNullException("children");
if (selector == null) throw new ArgumentNullException("selector");
var pred = branchpredicate ?? (src => true);
if (children(source) == null) return new[] { selector(source) };
return new[] { selector(source) }
.Concat(children(source)
.Where(pred)
.SelectMany(c => Flatten(c, children, selector, pred)));
}
Say I have a method like this:
IEnumerable<record> GetSomeRecords()
{
while(...)
{
yield return aRecord
}
}
Now, lets say I have a caller that also returns an enumerable of the same type, something like this
IEnumerable<record> ParentGetSomeRecords()
{
// I want to do this, but for some reason, my brain locks right here
foreach(item in someItems)
yield return GetSomeRecords();
}
That code gets syntax error error because yield return wants a type record, and I'm returning an IEnumerable of records
I want one "flat" IEnumerable that flattens a nested loop of enumerables. It's making me crazy, becuase I know I've done this before, but I can't seem to remember what it was. got any hints?
Is this what you are after?
IEnumerable<record> ParentGetSomeRecords()
{
foreach(var item in someItems)
foreach(var record in GetSomeRecords())
yield return record;
}
As noted, this will only work for a single level of children but is the most equivalent of your example code.
Update
Some people seem to believe you want the ability to flatten a hierarchical structure. Here is an extension method which performs breadth-first flattening (get the siblings before children):
Coming from a single item:
[Pure]
public static IEnumerable<T> BreadthFirstFlatten<T>(this T source, Func<T, IEnumerable<T>> selector)
{
Contract.Requires(!ReferenceEquals(source, null));
Contract.Requires(selector != null);
Contract.Ensures(Contract.Result<IEnumerable<T>>() != null);
var pendingChildren = new List<T> {source};
while (pendingChildren.Any())
{
var localPending = pendingChildren.ToList();
pendingChildren.Clear();
foreach (var child in localPending)
{
yield return child;
var results = selector(child);
if (results != null)
pendingChildren.AddRange(results);
}
}
}
This can be used like so:
record rec = ...;
IEnumerable<record> flattened = rec.BreadthFirstFlatten(r => r.ChildRecords);
This will result in an IEnumerable<record> containing rec, all of recs children, all of the childrens children, etc etc..
If you are coming from a collection of records, use the following code:
[Pure]
private static IEnumerable<T> BreadthFirstFlatten<T, TResult>(IEnumerable<T> source, Func<T, TResult> selector, Action<ICollection<T>, TResult> addMethod)
{
Contract.Requires(source != null);
Contract.Requires(selector != null);
Contract.Requires(addMethod != null);
Contract.Ensures(Contract.Result<IEnumerable<T>>() != null);
var pendingChildren = new List<T>(source);
while (pendingChildren.Any())
{
var localPending = pendingChildren.ToList();
pendingChildren.Clear();
foreach (var child in localPending)
{
yield return child;
var results = selector(child);
if (!ReferenceEquals(results, null))
addMethod(pendingChildren, results);
}
}
}
[Pure]
public static IEnumerable<T> BreadthFirstFlatten<T>(this IEnumerable<T> source, Func<T, IEnumerable<T>> selector)
{
Contract.Requires(source != null);
Contract.Requires(selector != null);
Contract.Ensures(Contract.Result<IEnumerable<T>>() != null);
return BreadthFirstFlatten(source, selector, (collection, arg2) => collection.AddRange(arg2));
}
[Pure]
public static IEnumerable<T> BreadthFirstFlatten<T>(this IEnumerable<T> source, Func<T, T> selector)
{
Contract.Requires(source != null);
Contract.Requires(selector != null);
Contract.Ensures(Contract.Result<IEnumerable<T>>() != null);
return BreadthFirstFlatten(source, selector, (collection, arg2) => collection.Add(arg2));
}
These two extension methods can be used like so:
IEnumerable<records> records = ...;
IEnumerable<record> flattened = records.BreadthFirstFlatten(r => r.ChildRecords);
Or from the reverse direction:
IEnumerable<record> records = ...;
IEnumerable<record> flattened = records.BreadthFirstFlatten(r => r.ParentRecords);
All of these extension methods are iterative so not limited by the stack size.
I have a whole host of these types of methods, including pre-order and post-order depth-first traversal, if you wish to see them, I will make a repo and upload them somewhere :)
How about:
IEnumerable<record> ParentGetSomeRecords()
{
var nestedEnumerable = <whatever the heck gets your nested set>;
// SelectMany with an identity flattens
// IEnumerable<IEnumerable<T>> to just IEnumerable<T>
return nestedEnumerable.SelectMany(rec => rec);
}
Inefficient, but you could use this:
List<Record> rcrdList = new List<Record>();
foreach (var item in someItems)
{
rcrdList.AddRange(item.GetSomeRecords());
}
return rcrdList;