How to split an Observable stream in chunks, dependent on second stream? - c#

I thought that this is easy, but my brain is melting right now..
The problem
Given the following IObservable<int> Stream:
1
1
0
0
0
1
0
0
1
0
1
I want to split it into an IObservable<IEnumerable<int>> Stream of the form
1
1 0 0 0
1 0 0
1 0
1
so whenever there is a 0, it just gets added to the IEnumerable, and when a 1 occurs, a new List is started; This is a bit cleaner definition to what my real problem is.
My approach so far
I thought a good solution would be to first convert it into an IObservable<IObservable<int>>via the Window method and then use ToEnumerable, but somehow I don't get it to work.. I used Zip and Skip(1) to get a diff to last element, I used DistinctUntilChanged(), too. I spare you all the variantes I tried...
Probably the closest I came was this code:
int[] ints = new[] { 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1 };
var observable = Observable.Interval(TimeSpan.FromMilliseconds(1000)).Take(11).Select(i => ints[i]);
Subject<int> subject = new Subject<int>();
observable.Subscribe(subject);
var observableDiff = subject.Skip(1).Zip(subject, (n, p) => new { Previous = p, Next = n });
var windows = observable.Window(() => observableDiff.Where(x => x.Next == 1));
int index = 0;
windows.Subscribe(window =>
{
Console.WriteLine(string.Format("new window [{0}] ", index++));
window.Subscribe(number => Console.WriteLine(number));
});
That returns good results, but unfortunately it crashes at the end..
new window [0]
1
new window [1]
1
0
0
0
new window [2]
1
0
0
new window [3]
1
0
new window [4]
new window [5]
new window [6]
new window [7]
new window [8]
new window [9]
<-- it goes on here until window ~ [80] with a stackoverflow exception
If that bug in my code wouldn't exist, I would have achieved it...
Any help would be much appreciated. :)
Edit: I use Rx-Experimental, but it doesn't make a difference (checked with LinqPad). Also removed the Subject, it didn't influence anything. It seems with my new approach (Edit2), you need a subject, otherwise the start of the windows is totally weird.
Edit2: changed the problem slightly, to better highlight my problem, sorry. Also updated my solution.


This worked for me:
var ints = (new[] { 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1 }).ToObservable();
var result =
ints
.Publish(ns =>
ns
.Where(n => n == 1)
.Select(n =>
ns.TakeWhile(m => m == 0).StartWith(n).ToArray())
).Merge();
I've used Publish in to make sure that the ints observable is treated as "hot" rather than "cold".
My results look like this:


The built-in Buffer seems pretty close to what you need. An intermediate subscription between the source and the Buffer call will let you get the closings observables you need for Buffer.
IObservable<IList<T>> Buffer<T>(IObservable<T> source,
Func<T, bool> startNew)
{
return Observable.Create<IList<T>>(
obs =>
{
var starts = new Subject<Unit>();
return source.Do(v =>
{
if (startNew(v))
starts.OnNext(Unit.Default);
})
.Buffer(() => starts)
.Where(v => v != null && v.Count > 0)
.Subscribe(obs);
});
}


Ok, these are good answers, too, from the Rx forums:
James Miles suggestion:
var source = new[] { 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1 }.ToObservable();
var windows =
from window in source
.Buffer(2,1) // create an overlapping buffer with 2 items
.Publish(xs => xs.Window(() => xs.Where(x => x.Last() == 1))) // close the window if the 2nd item is == 1
from result in window
.Select(buffer => buffer.First()) // we are only interested in the first item (the 2nd item might be the 1!)
.ToArray() // aggregate the results of the window
where result.Any() // filter out final (empty) window
select result;
int index = 0;
windows.Subscribe(window =>
{
Console.WriteLine(string.Format("new window [{0}] ", index++));
foreach(var x in window)Console.WriteLine(x);
});
Dave Sexton suggested using the Parser class from Extensions for Reactive Extensions (Rxx), which seems to be a more semantic approach:
using Rxx.Parsers.Reactive.Linq;
public sealed class SplitLab : BaseConsoleLab
{
protected override void Main()
{
var xs = Observable.Generate(
new[] { 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1 }.GetEnumerator(),
e => e.MoveNext(),
e => e,
e => (int) e.Current,
e => TimeSpan.FromSeconds(.5));
var query = xs.Parse(parser =>
from next in parser
let one = next.Where(value => value == 1)
let other = next.Not(one)
let window = from start in one
from remainder in other.NoneOrMore()
select remainder.StartWith(start)
let windowAsString = window.Join()
select windowAsString);
using (query.Subscribe(TraceLine))
{
WaitForKey();
}
}
}
So many roads to rome..

Related

Select x subitems to new list with LINQ

I have a HUGE sorted list with integers - Hugelist.
I need to create a new list of items based on input:
Query Hugelist trying to find all series of integers matching my input (variable input like: 5,6,4,9 (ideal dynamic number of inputs).
Numbers has to be in sequence in Hugelist.
Need all matched data (5,6,4,9 AND the next 7 items in Hugelist) added to new list.
So I end up with a list of matches in FilteredList
How on earth do I do this with LINQ and don't foreach my way out of it?
hugelist.OrderBy (l => l.Date)
.GroupBy (l => new { l.ID, l.City })
.ToList()
.ForEach(g => g.Aggregate (0, (acc, m) => { m.DiffToPrev = m.Value - acc; return m.Value; }));
DifftoPrev is what I would like to query with multiple inputs (ints).

You can do it like this:
List<int> input = new List<int> { 0, 0, 0, 0, 5,6,4,9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
List<int> search = new List<int> { 5,6,4,9 };
var test = Enumerable.Range(0, input.Count - search.Count)
.Where(x => input.Skip(x)
.Take(search.Count)
.SequenceEqual(search)
)
.Select(x => input.Skip(x)
.Take(search.Count+7)
).SelectMany(x => x);
But actually, I would not do it with LINQ but with a loop, because you will iterate quite often over the list and it is not easy readable.
Online demo: https://dotnetfiddle.net/1ngzFF

How can I compare two lists of uints and calculate the difference in any values?

I have two lists of uint type called firstReadOfMachineTotals and secondReadOfMachineTotals
I'm completely new to C# programming.
I would like to compare both lists and, if any of the values in the second list are higher than the first list, calculate by how much.
For instance...
firstReadOfMachineTotals = 10, 20, 4000, 554
secondReadOfMachineTotals = 10, 40, 4000, 554
I want to return '20' (based on the second item being 20 more than the equivalent in the first list).
Thanks
PS. There will never be more than one number different in the second list.

You can use a combination of Zip, Where:
var firstReadOfMachineTotals = new[]{ 10, 20, 4000, 554 };
var secondReadOfMachineTotals = new[]{ 10, 40, 4000, 554};
var result = firstReadOfMachineTotals.Zip(secondReadOfMachineTotals, (a,b) => b > a ? b - a : 0)
.Where(x => x > 0)
.OrderByDescending(x => x)
.FirstOrDefault();
Console.WriteLine(result); // output = 20
This method will default to 0 when all values are the same. If instead you wanted control of this default you could also do:
var firstReadOfMachineTotals = new[]{ 10, 20, 4000, 554 };
var secondReadOfMachineTotals = new[]{ 10, 40, 4000, 554};
var result = firstReadOfMachineTotals.Zip(secondReadOfMachineTotals, (a,b) => b > a ? b - a : 0)
.Where(x => x>0)
.DefaultIfEmpty(int.MinValue) // or whatever default you desire
.Max();
Console.WriteLine(result); // output = 20

You can index into the lists and simply take the difference of each element at the specified index then sum the difference to retrieve the result.
int result = Enumerable.Range(0, Math.Min(list1.Count, list2.Count))
.Select(i => list2[i] - list1[i] <= 0 ? 0 : list2[i] - list1[i]).Sum();

Use Zip:
var result = firstReadOfMachineTotals.Zip(secondReadOfMachineTotals,
(f, s) => s > f ? s - f : 0).Where(f => f > 0).DefaultIfEmpty(-1).Max();

The simplest solution will be to sort both the arrays
array1.sort()
array2.sort()
and compare each indexes and take action
for(int i=0;i<array1.lenght;i++)
{
if(array1[i] < array2[i])
{
// Specify your action.
}
}

Another way to do this is the following:
int difference = arr1
.Zip(arr2, (a, b) => (int?)Math.Max(b - a, 0))
.SingleOrDefault(d => d != 0) ?? 0;
It returns the difference if there is an element in the second collection which is larger than its corresponding element from the first collection.
If there isn't any, it returns zero.
Information to read:
LINQ Zip
LINQ FirstOrDefault
?? Operator
Nullable Types (int?)

Linq : Checking how many times the same value consecutively

it's my first question so if it's not quite clear , you can ask for extra information. Keep in mind that english is not my native language :).
I was wondering if it's possible to have an elegant way for next specification.
I think linq could be a possibility but i haven't got enough experience with the technology to get it to work:).
Remark this is not a homework assignment it's just a way to get an new angle to solve these kind of problems.
I've tried with the aggegrate function, maybe an action could help.
I want to keep track of:
the max times a value occurs in an array consecutively.
Per value it should display the maximum times the value occured consecutively
for example:
we have an array of 6 elements with elements either 0 or 1
0 , 0 , 0 , 1 , 1 ,0 result : 3 times 0 , 2 times 1
0 , 0 , 1 , 1 , 1 ,0 result : 2 times 0 , 3 times 1
0 , 1 , 0 , 1 , 1 ,0 result : 1 time 0 , 2 times 1
0 , 0 , 1 , 1 , 0 ,0 result : 2 times 0 , 2 times 1
Thanks in advance

I don't think Linq being a good way out; but a simple method will do:
// Disclamer: Dictionary can't have null key; so source must not coтtain nulls
private static Dictionary<T, int> ConsequentCount<T>(IEnumerable<T> source) {
if (null == source)
throw new ArgumentNullException("source");
Dictionary<T, int> result = new Dictionary<T, int>();
int count = -1;
T last = default(T);
foreach (T item in source) {
count = count < 0 || !object.Equals(last, item) ? 1 : count + 1;
last = item;
int v;
if (!result.TryGetValue(last, out v))
result.Add(last, count);
else if (v < count)
result[item] = count;
}
return result;
}
Tests:
int[][] source = new int[][] {
new[] { 0, 0, 0, 1, 1, 0 },
new[] { 0, 0, 1, 1, 1, 0 },
new[] { 0, 1, 0, 1, 1, 0 },
new[] { 0, 0, 1, 1, 0, 0 }, };
string report = string.Join(Environment.NewLine, source
.Select(array => $"{string.Join(" , ", array)} result : " +
string.Join(", ",
ConsequentCount(array)
.OrderBy(pair => pair.Key)
.Select(pair => $"{pair.Value} times {pair.Key}"))));
Console.Write(report);
Outcome:
0 , 0 , 0 , 1 , 1 , 0 result : 3 times 0, 2 times 1
0 , 0 , 1 , 1 , 1 , 0 result : 2 times 0, 3 times 1
0 , 1 , 0 , 1 , 1 , 0 result : 1 times 0, 2 times 1
0 , 0 , 1 , 1 , 0 , 0 result : 2 times 0, 2 times 1

You can write your own method for groupping by consecutive
public static class Extension
{
public static IEnumerable<IEnumerable<int>> GroupConsecutive(this IEnumerable<int> list)
{
var group = new List<int>();
foreach (var i in list)
{
if (group.Count == 0 || i - group[group.Count - 1] == 0)
group.Add(i);
else
{
yield return group;
group = new List<int> {i};
}
}
yield return group;
}
}
And then use it like that:
var groups = new[] { 0, 0, 1, 1, 0, 0 }.GroupConsecutive();
var maxGroupped = groups.GroupBy(i => i.First()).Select(i => new
{
i.Key,
Count = i.Max(j => j.Count())
});
foreach (var g in maxGroupped)
Console.WriteLine(g.Count + " times " + g.Key);

Here is a lazy inefficient way (seems like linear complexity to me):
int[] arr = { 0, 0, 0, 1, 1, 0 };
string str = string.Concat(arr); // "000110"
int max0 = str.Split('1').Max(s => s.Length); // 3
int max1 = str.Split('0').Max(s => s.Length); // 2
and here is the efficient O(n) version:
int[] arr = { 0, 1, 1, 0, 0, 0 };
int i1 = 0, i = 1;
int[] max = { 0, 0 };
for (; i < arr.Length; i++)
{
if (arr[i] != arr[i1])
{
if (i - i1 > max[arr[i1]]) max[arr[i1]] = i - i1;
i1 = i;
}
}
if (i - i1 > max[arr[i1]]) max[arr[i1]] = i - i1;
Debug.Print(max[0] + "," + max[1]); // "3,2"

Linq will be a bit ugly, but it is possible though, your choice of Aggregate is the way to go, but it won't be a one liner in any case,
Something like this will work,
static void Main(string[] args)
{
var list = new List<int>() { 0, 0, 1, 1, 1, 0, 0 };
var result = list.Aggregate(new
{
Last = (int?)null,
Counts = new Dictionary<int, int>(),
Max = new Dictionary<int, int>()
}, (context, current) =>
{
int count;
if (!context.Counts.TryGetValue(current, out count))
count = 1;
if (context.Last == current)
count += 1;
int lastMax;
context.Max.TryGetValue(current, out lastMax);
context.Max[current] = Math.Max(lastMax, count);
if (context.Last != current)
count = 1;
context.Counts[current] = count;
return new { Last = (int?)current, context.Counts, context.Max };
});
Console.WriteLine(string.Join(",", list) + " Result: ");
var output = string.Join(", ", result.Max.Select(x => string.Format("{0} times {1}", x.Value, x.Key)));
Console.WriteLine(output);
Console.ReadKey();
}

Like others said, performance wise Linq might not be the right tool for the job.
My linq only version would be:
from array in arrayOfArrays
let result = new {
Zeroes = array.TakeWhile(x => x == 0).Count(),
Ones = array.SkipWhile(x => x == 0).TakeWhile(x => x == 1).Count()
}
select $"{String.Join(", ", array)} result : {result.Zeroes} times 0, {result.Ones} times 1"
I'm not sure if Linq2Objects will be smart here to optimize the query internally. We ARE within the query iterating mutliple times over the array. So like i said in advance there may be a performance hit if you execute this over a lot of arrays. If anyone would care to check performance of this in regards to other non linq solutions.

First of all thanks to everyone who put the time and effort in answering the question.
I've choosen Dmitry Bychenko as a valid answer , he was the first to provide an answer , and it was an elegant answer.
Matthew diserves also credit because he has shown me how the aggregate function works with conditionals.
last but not least the answer of victor was the simplest one. I did enhance it to work with generics .
void Main()
{
var groups = new[] { 0, 0, 1, 1, 0,30,1,1,1,1,1 , 22, 22, 15,15,0,0,0,0,0,0 }.GroupConsecutive();
groups.Dump();
var maxGroupped = groups.GroupBy(i => i.First()).Select(i => new
{
i.Key,
Count = i.Max(j => j.Count())
});
foreach (var g in maxGroupped)
Console.WriteLine(g.Count + " times " + g.Key);
}
public static class Extension
{
public static IEnumerable<IEnumerable<T>> GroupConsecutive<T>(this IEnumerable<T> list)
{
var group = new List<T>();
foreach (var value in list)
{
if (group.Count == 0 || value.Equals(group[group.Count-1]))
group.Add(value);
else
{
yield return group;
group = new List<T> {value};
}
}
yield return group;
}

LINQ Aggregate function up to current row

Assuming I have the following
var list = new []{
new { Price = 1000, IsFirst = true},
new { Price = 1100, IsFirst = false},
new { Price = 450, IsFirst = true},
new { Price = 300, IsFirst = false}
};
and I want to generate the following output:
Price IsFirst First Second Final
----------------------------------
1000 True 1000 0 1000
1100 False 0 1100 -100
450 True 450 0 350
300 False 0 300 50
Is it possible to have some sort of aggregate function processed up to current row? I like to have all the stuff in pure LINQ but as of now I have no other choice than manually iterating the list and sum the column conditionally.
var result = list.Select(x => new
{
Price = x.Price,
IsFirst = x.IsFirst,
First = x.IsFirst ? x.Price : 0,
Second = !x.IsFirst ? x.Price : 0,
Final = 0 // ???
}).ToList();
int sum = 0;
for(int i=0; i<result.Count(); i++)
{
sum += (result[i].IsFirst ? result[i].Price : - result[i].Price);
// updating Final value manually
}

The easiest way to do this is to use the Microsoft Reactive Extension Team's "Interactive Extension" method Scan. (Use NuGet and look for Ix-Main.)
var query =
list
.Scan(new
{
Price = 0,
IsFirst = true,
First = 0,
Second = 0,
Final = 0
}, (a, x) => new
{
Price = x.Price,
IsFirst = x.IsFirst,
First = x.IsFirst ? x.Price : 0,
Second = !x.IsFirst ? x.Price : 0,
Final = a.Final + (x.IsFirst ? x.Price : - x.Price)
});
This gives:
However, you can do it with the built-in Aggregate operator like this:
var query =
list
.Aggregate(new []
{
new
{
Price = 0,
IsFirst = true,
First = 0,
Second = 0,
Final = 0
}
}.ToList(), (a, x) =>
{
a.Add(new
{
Price = x.Price,
IsFirst = x.IsFirst,
First = x.IsFirst ? x.Price : 0,
Second = !x.IsFirst ? x.Price : 0,
Final = a.Last().Final + (x.IsFirst ? x.Price : - x.Price)
});
return a;
})
.Skip(1);
You get the same result.

What you want is called a running total.
As far as I know, there is no built-in method to do this in LINQ, but various people have written extension methods to do that. One that I quickly found online is this one:
Rollup Extension Method: Create Running Totals using LINQ to Objects
Based on this, it should be fairly easy to turn your code into an extension method that
resets the intermediate value when encountering an item with IsFirst = true,
otherwise decrements the value,
and yields it.

You can do something like this:
int final = 0;
var result = list.Select(x => new
{
Price = x.Price,
IsFirst = x.IsFirst,
First = x.IsFirst ? x.Price : 0,
Second = !x.IsFirst ? x.Price : 0,
Final = x.IsFirst ? final+=x.Price : final-=x.Price
}).ToList();
But you would need to define an integer (final) outside of the linq expression to keep track of the total sum.

Removing empty bytes from List<byte>

How do I delete empty bytes from a List<byte>?
For example I got a list with a size of [5].
[0] = 5
[1] = 3
[2] = 0
[3] = 0
[4] = 17
At this example I want to delete byte with index: 2 and 3.
The Items in the list change every second. So the next time the list could be filled with something like:
[0] = 0
[1] = 2
[2] = 3
[3] = 4
[4] = 0

It's something like
myList.RemoveAll(b => b == 0);

bytes.RemoveAll(x => x == 0)

How about using List.RemoveAll() method?
Removes all the elements that match the conditions defined by the
specified predicate.
YourList.RemoveAll(n => n == 0);
For example;
List<int> list = new List<int>(){5, 3, 0, 0, 17};
list.RemoveAll(n => n == 0);
foreach (var i in list)
{
Console.WriteLine(i);
}
Output will be;
5
3
17
Here is a DEMO.

Categories