How to combine two different GroupedStreams in Rx.NET? - c#

This question is similar, but it does not apply to my case, since the user needed the merge observable streams from the same IGroupedObservable, while I want to combine streams from different groups.
I have the following structures and streams:
type A = {
Id: int
Value: int
}
type B = {
Id: int
Value: int
}
//subjects to test input, just any source of As and Bs
let subjectA: Subject<A> = Subject.broadcast
let subjectB: Subject<B> = Subject.broadcast
//grouped streams
let groupedA: IObservable<<IGroupedObservable<int, A>> = Observable.groupBy (fun a -> a.Id) subjectA
let groupedB: IObservable<<IGroupedObservable<int, B>> = Observable.groupBy (fun b -> b.Id) subjectB
I want to somehow merge the internal observables of A and B when groupedA.Key = groupedB.Key, and get an observable of (A, B) pairs where A.Id = B.Id
The signature I want is something like
IObservable<IGroupedObservable<int, A>> -> IObservable<IGroupedObservable<int, B>> -> IObservable<IGroupedObservable<int, (A, B)>> where for all (A, B), A.Id = B.Id
I tried a bunch of combineLatest, groupJoin, filters and maps variations, but with no success.
I'm using F# with Rx.Net and FSharp.Control.Reactive, but if you know the answer in C# (or any language, really) please post it

Here is a custom operator GroupJoin that you could use. It is based on the Select, Merge, GroupBy and Where operators:
/// <summary>
/// Groups and joins the elements of two observable sequences, based on common keys.
/// </summary>
public static IObservable<(TKey Key, IObservable<TLeft> Left, IObservable<TRight> Right)>
GroupJoin<TLeft, TRight, TKey>(
this IObservable<TLeft> left,
IObservable<TRight> right,
Func<TLeft, TKey> leftKeySelector,
Func<TRight, TKey> rightKeySelector,
IEqualityComparer<TKey> keyComparer = null)
{
// Arguments validation omitted
keyComparer ??= EqualityComparer<TKey>.Default;
return left
.Select(x => (x, (TRight)default, Type: 1, Key: leftKeySelector(x)))
.Merge(right.Select(x => ((TLeft)default, x, Type: 2, Key: rightKeySelector(x))))
.GroupBy(e => e.Key, keyComparer)
.Select(g => (
g.Key,
g.Where(e => e.Type == 1).Select(e => e.Item1),
g.Where(e => e.Type == 2).Select(e => e.Item2)
));
}
Usage example:
var subjectA = new Subject<A>();
var subjectB = new Subject<B>();
IObservable<IGroupedObservable<int, (A, B)>> query = subjectA
.GroupJoin(subjectB, a => a.Id, b => b.Id)
.SelectMany(g => g.Left.Zip(g.Right, (a, b) => (g.Key, a, b)))
.GroupBy(e => e.Key, e => (e.a, e.b));

I'm not clear if this is what you want. So it may be helpful to clarify first with runner code. Assuming the following runner code:
var aSubject = new Subject<A>();
var bSubject = new Subject<B>();
var groupedA = aSubject.GroupBy(a => a.Id);
var groupedB = bSubject.GroupBy(b => b.Id);
//Initiate solution
solution.Merge()
.Subscribe(t => Console.WriteLine($"(Id = {t.a.Id}, AValue = {t.a.Value}, BValue = {t.b.Value} )"));
aSubject.OnNext(new A() { Id = 1, Value = 1 });
aSubject.OnNext(new A() { Id = 1, Value = 2 });
bSubject.OnNext(new B() { Id = 1, Value = 10 });
bSubject.OnNext(new B() { Id = 1, Value = 20 });
bSubject.OnNext(new B() { Id = 1, Value = 30 });
Do you want to see the following output:
(Id = 1, AValue = 1, BValue = 10)
(Id = 1, AValue = 2, BValue = 10)
(Id = 1, AValue = 1, BValue = 20)
(Id = 1, AValue = 2, BValue = 20)
(Id = 1, AValue = 1, BValue = 30)
(Id = 1, AValue = 2, BValue = 30)
If that's the case, you can get to solution as follows:
var solution = groupedA.Merge()
.Join(groupedB.Merge(),
_ => Observable.Never<Unit>(),
_ => Observable.Never<Unit>(),
(a, b) => (a, b)
)
.Where(t => t.a.Id == t.b.Id)
.GroupBy(g => g.a.Id);
I'll caution that there are memory/performance impacts here if this is part of a long-running process. This keeps all A and B objects in memory indefinitely, waiting to see if they can be paired off. To shorten the amount of time they're kept in memory, change the Observable.Never() calls to appropriate windows for how long to keep each object in memory.

As a start, this has the signature you want:
let cartesian left right =
rxquery {
for a in left do
for b in right do
yield a, b
}
let mergeGroups left right =
rxquery {
for (leftGroup : IGroupedObservable<'key, 'a>) in left do
for (rightGroup : IGroupedObservable<'key, 'b>) in right do
if leftGroup.Key = rightGroup.Key then
let merged = cartesian leftGroup rightGroup
yield {
new IGroupedObservable<_, _> with
member __.Key = leftGroup.Key
member __.Subscribe(observer) = merged.Subscribe(observer)
}
}
However, in my testing, the groups are all empty. I don't have enough Rx experience to know why, but perhaps someone else does.

Related

Using LINQ Lambda expression determining value by group by and where condition

So I have the following data table:
Region Class
Reg100 A
Reg100 B
Reg200 A
Reg300 B
Where I want to determine a region that has two classes A and B. In this case it would be Reg100. How could I write this in using lambda expression?
I have tried something like below but not getting what I want.
dt.Where(x => x.Class.Contains(listOfAandB).GroupBy(x=>x.Region).FirstOrDefault()
Lets define the input:
class Data
{
public string Region;
public string Class;
}
var dt = new[]
{
new Data { Region = "Reg100", Class = "A" },
new Data { Region = "Reg100", Class = "B" },
new Data { Region = "Reg200", Class = "A" },
new Data { Region = "Reg300", Class = "B" },
};
Now, using GroupBy we can group the input by Region.
dt.GroupBy(x => x.Region)
This yields { Reg100 (A, B), Reg200 A, Reg 300 B }. Now we look if we can find both A and B:
dt.GroupBy(x => x.Region)
.Where(g => g.Any(x => x.Class == "A") && g.Any(x => x.Class == "B"))
And finally as we are only interested in the region, we project to it:
dt.GroupBy(x => x.Region)
.Where(g => g.Any(x => x.Class == "A") && g.Any(x => x.Class == "B"))
.Select(g => g.Key);

Strange behavior of EqualityComparer with nullable fields

Assume there is this class:
public class Foo
{
public int Id { get; set; }
public int? NullableId { get; set; }
public Foo(int id, int? nullableId)
{
Id = id;
NullableId = nullableId;
}
}
I need to compare these objects by following rules:
If both objects have value for NullableId then we compare both Id
and NullableId
If some of the objects/both of them do not have NullableId then
ignore it and compare only Id.
To achieve it I have overwritten Equals and GetHashCode like this:
public override bool Equals(object obj)
{
var otherFoo = (Foo)obj;
var equalityCondition = Id == otherFoo.Id;
if (NullableId.HasValue && otherFoo.NullableId.HasValue)
equalityCondition &= (NullableId== otherFoo.NullableId);
return equalityCondition;
}
public override int GetHashCode()
{
var hashCode = 806340729;
hashCode = hashCode * -1521134295 + Id.GetHashCode();
return hashCode;
}
Further down I have two lists of Foo:
var first = new List<Foo> { new Foo(1, null) };
var second = new List<Foo> { new Foo(1, 1), new Foo(1, 2), new Foo(1, 3) };
Next, I want to join these lists. If I do it like this:
var result = second.Join(first, s => s, f => f, (f, s) => new {f, s}).ToList();
then the result would be as I expected and I will get 3 items.
But, if I change order and join first with second:
var result = first.Join(second, f => f, s => s, (f, s) => new {f, s}).ToList();
then the result would only have 1 item - new Foo(1, null) and new Foo(1 ,3)
I can not get what am I doing wrong. If try to put a break point in Equals method then I can see that it tries to compare items from same list (e. g. compare new Foo(1, 1) and new Foo(1 ,2)). For me it looks like that happens because of Lookup that is being created inside Join method.
Could someone clarify what happens there? What should I change to achieve desired behavior?
Your Equals method is reflexive and symmetric, but it is not transitive.
Your implementation doesn't meet the requirements specified in the docs:
If (x.Equals(y) && y.Equals(z)) returns true, then x.Equals(z) returns true.
from https://learn.microsoft.com/en-us/dotnet/api/system.object.equals?view=netframework-4.8
For example, suppose you have:
var x = new Foo(1, 100);
var y = new Foo(1, null);
var z = new Foo(1, 200);
You have x.Equals(y) and y.Equals(z) which implies that you should also have x.Equals(z), but your implementation does not do this. Since you don't meet the specification, you can't expect any algorithms reliant on your Equals method to behave correctly.
You ask what you can do instead. This depends on exactly what you need to do. Part of the problem is that it's not really clear what is intended in the corner-cases, if indeed they can appear. What should happen if one Id appears multiple times with the same NullableId in one or both lists? For a simple example, if new Foo(1, 1) exists in the first list three times, and the second list three times, what should be in the output? Nine items, one for each pairing?
Here's a naive attempt to solve your problem. This joins on only Id and then filters out any pairings that have incompatible NullableId. But you might not be expecting the duplicates when an Id appears multiple times in each list, as can be seen in the example output.
using System;
using System.Linq;
using System.Collections.Generic;
public class Foo
{
public int Id { get; set; }
public int? NullableId { get; set; }
public Foo(int id, int? nullableId)
{
Id = id;
NullableId = nullableId;
}
public override string ToString() => $"Foo({Id}, {NullableId?.ToString()??"null"})";
}
class MainClass {
public static IEnumerable<Foo> JoinFoos(IEnumerable<Foo> first, IEnumerable<Foo> second) {
return first
.Join(second, f=>f.Id, s=>s.Id, (f,s) => new {f,s})
.Where(fs =>
fs.f.NullableId == null ||
fs.s.NullableId == null ||
fs.f.NullableId == fs.s.NullableId)
.Select(fs => new Foo(fs.f.Id, fs.f.NullableId ?? fs.s.NullableId));
}
public static void Main (string[] args) {
var first = new List<Foo> { new Foo(1, null), new Foo(1, null), new Foo(1, 3) };
var second = new List<Foo> { new Foo(1, 1), new Foo(1, 2), new Foo(1, 3), new Foo(1, null) };
foreach (var f in JoinFoos(first, second)) {
Console.WriteLine(f);
}
}
}
Output:
Foo(1, 1)
Foo(1, 2)
Foo(1, 3)
Foo(1, null)
Foo(1, 1)
Foo(1, 2)
Foo(1, 3)
Foo(1, null)
Foo(1, 3)
Foo(1, 3)
It also might be too slow for you if you have tens of thousands of items with the same Id, because it builds up every possible pair with matching Id before filtering them out. If each list has 10,000 items with Id == 1 then that's 100,000,000 pairs to pick through.
My answer contains a program that I believe is better than the one proposed in Weeble's answer but first I would like to demonstrate how the Join method works and talk about problems I see in your approach.
As you can see here https://learn.microsoft.com/en-us/dotnet/api/system.linq.enumerable.join?view=netframework-4.8
the Join method
Correlates the elements of two sequences based on matching keys.
If the keys don't match then elements from both collections are not included. For example, remove your Equals and GetHashCode methods and try this code:
var first = new List<Foo> { new Foo(1, 1) };
var second = new List<Foo> { new Foo(1, 1), new Foo(1, 2), new Foo(1, 3) };
//This is your original code that returns no results
var result = second.Join(first, s => s, f => f, (f, s) => new { f, s }).ToList();
result = first.Join(second, s => s, f => f, (f, s) => new { f, s }).ToList();
//This code is mine and it returns in both calls of the Join method one element in the resulting collection; the element contains two instances of Foo (1,1) - f and s
result = second.Join(first, s => new { s.Id, s.NullableId }, f => new { f.Id, f.NullableId }, (f, s) => new { f, s }).ToList();
result = first.Join(second, s => new { s.Id, s.NullableId }, f => new { f.Id, f.NullableId }, (f, s) => new { f, s }).ToList();
But if you set your original data input that contains null with my code:
var first = new List<Foo> { new Foo(1, null) };
var second = new List<Foo> { new Foo(1, 1), new Foo(1, 2), new Foo(1, 3) };
var result = second.Join(first, s => new { s.Id, s.NullableId }, f => new { f.Id, f.NullableId }, (f, s) => new { f, s }).ToList();
result = first.Join(second, s => new { s.Id, s.NullableId }, f => new { f.Id, f.NullableId }, (f, s) => new { f, s }).ToList();
the result variable will be empty in both cases since the key { 1, null } doesn't match any other key, i.e. { 1, 1 }, { 1, 2 }, { 1, 3 }.
Now returning to your question. I would suggest you reconsider your entire approach in cases like this and here is why. Let us imagine that your implementation of the Equals and GetHashCode methods worked as you expected and you even didn't post your question. Then your solution creates the following outcomes, as I see it:
To understand how your code calculates its output the user of your code has to have access to the code of the Foo type and spend time reviewing your implementation of the Equals and GetHashCode methods (or reading documentation).
With such implementation of the Equals and GetHashCode methods, you are trying to change the expected behavior of the Join method. The user may expect that the first element of the first collection Foo(1, null) will not be considered equal to the first element of the second collection Foo(1, 1).
Let us imagine that you have multiple classes to join, each is written by some individual, and each class has its own logic in the Equals and GetHashCode methods. To figure out how actually your joining works with each type the user instead of looking into a joining method implementation only once would need to check the source code of all those classes trying to understand how each type handles its own comparison facing different variations of things like this with magic numbers (taken from your code):
public override int GetHashCode()
{
var hashCode = 806340729;
hashCode = hashCode * -1521134295 + Id.GetHashCode();
return hashCode;
}
It may don't seem a big problem but imagine you are a new person on the
the project, you have a lot of classes with logic like this and limited time
to complete your task, e.g. you have an urgent change request, huge sets
of data input, and no unit tests.
If someone inherites from your class Foo and put an instance of Foo1 to the collection among with Foo instances:
public class Foo1 : Foo
{
public Foo1(int id, int? nullableId) : base (id, nullableId)
{
Id = id;
NullableId = nullableId;
}
public override bool Equals(object obj)
{
var otherFoo1 = (Foo1)obj;
return Id == otherFoo1.Id;
}
public override int GetHashCode()
{
var hashCode = 806340729;
hashCode = hashCode * -1521134295 + Id.GetHashCode();
return hashCode;
}
}
var first = new List<Foo> { new Foo1(1, 1) };
var second = new List<Foo> { new Foo(1, 1), new Foo(1, 2), new Foo(1, 3)};
var result = second.Join(first, s => s, f => f, (f, s) => new { f, s }).ToList();
result = first.Join(second, s => s, f => f, (f, s) => new { f, s }).ToList();
then you have here a run-time exception in the Equals method of the type Foo1:
System.InvalidCastException, Message=Unable to cast object of type
'ConsoleApp1.Foo' to type 'ConsoleApp1.Foo1'. With the same input data, my code
would work fine in this situation:
var result = second.Join(first, s => s.Id, f => f.Id, (f, s) => new { f, s }).ToList();
result = first.Join(second, s => s.Id, f => f.Id, (f, s) => new { f, s }).ToList();
With your implementation of the Equals and GetHashCode methods when someone modifies the joining code like this:
var result = second.Join(first, s => new { s.Id, s.NullableId }, f => new { f.Id, f.NullableId }, (f, s) => new { f, s }).ToList();
result = first.Join(second, s => new { s.Id, s.NullableId }, f => new { f.Id, f.NullableId }, (f, s) => new { f, s }).ToList();
then your logic in the Equals and GetHashCode methods will be ignored and
you will have a different result.
In my opinion, this approach (with overriding Equals and GetHashCode methods) may be a source of multiple bugs. I think it is better when your code performing joining has an implementation that can be understood without any extra information, the implementation of the logic is concentrated within one method, the implementation is clear, predictable, maintainable, and it is simple to understand.
Please also note that with your input data:
var first = new List<Foo> { new Foo(1, null) };
var second = new List<Foo> { new Foo(1, 1), new Foo(1, 2), new Foo(1, 3) };
the code in the Weeble's answer generates the following output:
Foo(1, 1)
Foo(1, 2)
Foo(1, 3)
while as far as I understand you asked for an implementation that with the input produces output that looks like this:
Foo(1, null), Foo(1, 1)
Foo(1, null), Foo(1, 2)
Foo(1, null), Foo(1, 3)
Please consider updating your solution with my code since it produces a result in the format you asked for, my code is easier to understand, and it has other advantages as you can see:
using System;
using System.Collections.Generic;
using System.Linq;
namespace ConsoleApp40
{
public class Foo
{
public int Id { get; set; }
public int? NullableId { get; set; }
public Foo(int id, int? nullableId)
{
Id = id;
NullableId = nullableId;
}
public override string ToString() => $"Foo({Id}, {NullableId?.ToString() ?? "null"})";
}
class Program
{
static void Main(string[] args)
{
var first = new List<Foo> { new Foo(1, null), new Foo(1, 5), new Foo(2, 3), new Foo(6, 2) };
var second = new List<Foo> { new Foo(1, 1), new Foo(1, 2), new Foo(1, 3), new Foo(2, null) };
var result = second.Join(first, s=>s.Id, f=>f.Id, (f, s) => new { f, s })
.Where(o => !((o.f.NullableId != null && o.s.NullableId != null) &&
(o.f.NullableId != o.s.NullableId)));
foreach (var o in result) {
Console.WriteLine(o.f + ", " + o.s);
}
Console.ReadLine();
}
}
}
Output:
Foo(1, 1), Foo(1, null)
Foo(1, 2), Foo(1, null)
Foo(1, 3), Foo(1, null)
Foo(2, null), Foo(2, 3)

How to make linq lambda expression

select a.ObjectFieldID,
a.FieldName,
b.RelationName
from tblMNG_Framework_ObjectField as a left join
tblMNG_Framework_ObjectRelation as b on a.ObjectID = b.RelatedObjectID
and a.ObjectFieldID = b.RelatedKeyFieldID
where a.ObjectID = 2
and a.Deleted = 0
It's my quick code, I have no data to test so feedback me with your data sample if left join is not exact.
tblMNG_Framework_ObjectField.GroupJoin(tblMNG_Framework_ObjectRelation,
a => new { JoinCol1 = a.ObjectID, JoinCol2 = a.ObjectFieldID },
b => new { JoinCol1 = b.RelatedObjectID, JoinCol2 = b.RelatedKeyFieldID },
(a, b) => new { a, b }).Where(x => x.a.ObjectID == 2 && x.a.Deleted == 0)
.SelectMany(b => b.b.DefaultIfEmpty(),
(a, b) => new { a.a.ObjectID, a.a.FieldName, b.RelationName });
Hope it helps

Count object with same attributevalues in list of objects with GroupBy

I got a class:
public class result
{
public int A;
public int B;
public int C;
}
I make a list of it:
public static List<result> results = new List<result>();
I then fill that list with random data
Somthing like 10,000,000 enteries, where a, b and c will have a value of 0 to 24.
I would like to show in my console what combo has been found and how many
in SQL it would be somthing like:
SELECT A, B, C, COUNT(*) AS total
FROM results
GROUP BY A, B, C
And i have tried so many things, i think i can write a book about it.
some of the stuff i tried:
var query1 = results.GroupBy(x => new { x.A, x.B, x.C }).Select(group => new { Value = group.Key, Count = group.Count() });
var query2 = from r in results group r by new { r.A, r.B, r.C } into rGroup select rGroup;
var query3 = results.GroupBy(x => new { x.A, x.B, x.C }) .Where(g => g.Count() > 1).ToDictionary(x => x.Key, y => y.Count());
var query4 = from r in results
group r by new { r.A, r.B, r.C } into rGroup
select new { key = rGroup.Key, cnt = rGroup.Count() };
But nothing seems to work.
I would like to get back a list with the a,b,c values and a count of how many have been found.
Yet im unable to get it working, i tried hours of googleing and tried everything, at this point i am completly lost.
For completeness sake a full example.
Same solutions as nlawalker though, producing a dict.
public class result
{
public int A;
public int B;
public int C;
public result(int a, int b, int c)
{
A = a;
B = b;
C = c;
}
}
static void Main(string[] args)
{
Random r = new Random(23);
var data = new List<result>();
for (int i = 0; i < 100; i++)
data.Add(new result(r.Next(1, 3), r.Next(1, 3), r.Next(1, 3)));
var dic = data
.GroupBy(k => new { k.A, k.B, k.C })
.ToDictionary(g => g.Key, g => g.Count());
foreach (var kvp in dic)
Console.WriteLine($"({kvp.Key.A},{kvp.Key.B},{kvp.Key.C}) : {kvp.Value}");
Console.ReadLine();
}
Output:
(2,2,2) : 13
(1,2,2) : 11
(2,1,1) : 9
(1,1,1) : 16
(1,2,1) : 14
(1,1,2) : 15
(2,1,2) : 7
(2,2,1) : 15
You got the GroupBy part right - you just need to select the groups into another object that has the A, B and C values of the group, along with the count of the group:
results.GroupBy(x => new { x.A, x.B, x.C })
.Select(g => new { g.Key.A, g.Key.B, g.Key.C, Count = g.Count()})

How to calculate employees reporting to a manager recursively using LINQ?

I have a dictionary that contains mapping of employee and his/her manager like this
Dictionary<string, string> employees = new Dictionary<string, string>()
{
{ "A","C" },
{ "B","C" },
{ "C","F" },
{ "D","E" },
{ "E","F" },
{ "F","F" }
};
I want to get no of employees under each manager in the hierarchy not just their direct reports but down the hierarchy chain.
In the above dictionary the root node/ceo is listed as reporting to himself. This is the only node that is guaranteed to have this self relationship.
How can I find total no of employees reporting to each manager. Output should be
A - 0
B - 0
C - 2
D - 0
E - 1
F - 5
This is what I tried so far but it only gives counts of direct reports not all reports in the chain
var reports = employees
.GroupBy(e => e.Value, (key, g) => new { employee = key, reports = g.Count() });
The problem you describe is virtually identical to the problem described in this blog post.
Your spec could be written as (this is a trivial adaptation from the quoted text):
The complete set of reports upon which an Employee depends is the transitive closure of the directly-reports-to relationship.
The post then proceeds to provide the following code to compute the transitive closure of a particular relationship:
static HashSet<T> TransitiveClosure<T>(
this Func<T, IEnumerable<T>> relation,
T item)
{
var closure = new HashSet<T>();
var stack = new Stack<T>();
stack.Push(item);
while (stack.Count > 0)
{
T current = stack.Pop();
foreach (T newItem in relation(current))
{
if (!closure.Contains(newItem))
{
closure.Add(newItem);
stack.Push(newItem);
}
}
}
return closure;
}
So all that's left is to provide the code for the directly-reports-to relationship.
This can be easily computed by creating a lookup from your dictionary mapping each employee to their reports:
var directReportLookup = employees.ToLookup(pair => pair.Value, pair => pair.Key);
Func<string, IEnumerable<string>> directReports =
employee => directReportLookup[employee];
employees.Select(pair => new
{
Employee = pair.Key,
Count = directReports.TransitiveClosure(pair.Key).Count,
});
You might be interested in a recursion in the anonymous methods. There is one interesting approach from the functional languages: fixed-point combinator.
It looks like this:
public static class Combinator
{
public static Func<TInput, TResult> Y<TInput, TResult>(Func<Func<TInput,TResult>, TInput, TResult> function)
{
return input => function(Y(function), input);
}
}
And can be used like this:
var result = employees
.Select(employee => Combinator.Y<string, int>
(
(f, e) => employees.Where(x => x.Value == e && x.Value != x.Key)
.Aggregate(employees.Count(x => x.Value == e && x.Value != x.Key), (current, next) => current + f(next.Key))
)
.Invoke(employee.Key))
.ToList();
Of course, it will be more useful for simler tasks, like this:
var fact = Combinator.Y<int, int>((f, n) => n > 1 ? n * f(n - 1) : 1);
var fib = Combinator.Y<uint, int>((f, n) => n > 2 ? f(n - 1) + f(n - 2) : (n == 0 ? 0 : 1));

Categories