We Keep Coding

How to implement a specialized overload of the List.RemoveAll method, with an index parameter in the predicate?

The List<T>.RemoveAll is a quite useful method, that allows to remove efficiently multiple items from a list. Unfortunately in some scenarios I needed some extra features that the method doesn't have, and some guarantees that the documentation doesn't provide. It also has a questionable behavior in case the match predicate fails, that causes me anxiety. So in this question I am asking for an implementation of the same method, in the form of an extension method, with these features and characteristics:
Instead of a Predicate<T> it accepts a Func<T, int, bool> delegate, where the int is the zero-based index of the T item.
It guarantees that the predicate will be invoked exactly once for each item, in a stricly ascending order.
In case the predicate returns true for some items and then fails for another item, the items that have been elected for removal are removed from the list before the propagation of the exception.
Here is the signature of the extension method that I am trying to implement:
public static int RemoveAll<T>(this List<T> list, Func<T, int, bool> predicate);
It returns the number of elements that were removed.
I attempted to implement it using as starting point the existing implementation, but it has some performance optimizations that make it quite complex, and injecting the desirable "exceptional" behavior is not obvious. I am interested for an implementation that is simple and reasonably efficient. Using LINQ in the implementation is not desirable, because it implies memory allocations that I would like to avoid.
Context: I should demonstrate the behavior of the built-in List<T>.RemoveAll method, and explain why I don't like it. In case the match predicate fails for an item in the middle of the list, the items that have already been elected for removal are either not removed, or they are replaced with duplicates of other elements. In all cases the list retains its original size. Here is a minimal demo:
List<int> list = new(Enumerable.Range(1, 15));
Console.WriteLine($"Before RemoveAll: [{String.Join(", ", list)}]");
try
{
list.RemoveAll(item =>
{
if (item == 10) throw new Exception();
bool removeIt = item % 2 == 1;
if (removeIt) Console.WriteLine($"Removing #{item}");
return removeIt;
});
}
catch { } // Ignore the error for demonstration purposes
finally
{
Console.WriteLine($"After RemoveAll: [{String.Join(", ", list)}]");
}
The list has 15 numbers, and the intention is to remove the odd numbers from the list. The predicate fails for the 10th number.
Output:
Before RemoveAll: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
Removing #1
Removing #3
Removing #5
Removing #7
Removing #9
After RemoveAll: [2, 4, 6, 8, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
Online demo.
As you can see the numbers 1 and 3 have been removed, the 5, 7 and 9 are still there, and the numbers 6 and 8 have been duplicated (there are two occurrences of each). On the contrary the output that I expected to see is:
After RemoveAll: [2, 4, 6, 8, 10, 11, 12, 13, 14, 15]
This would be a reasonable and predictable behavior I could count on. It keeps the levels of danger in a manageable level. I am not risking, for example, duplicating items in a virtual shopping cart, or printing twice some PDF documents from a selection. The existing behavior stretches a bit too much my comfort levels.
I have reported this behavior to Microsoft, and the feedback that I've got is that in case of failure the outcome is undefined. From their point of view there is no difference between the two above outputs (the actual and the expected). Both are equally corrupted, because both represent a state that is neither the original nor the final/correct state after a successful execution. So they don't think that there is any bug that needs to be fixed, and doing changes that could potentially affect negatively the performance of successful executions is not justified. They also believe that the existing behavior is not surprising or unexpected, so there is no reason to document it.

This solution is based on the idea to separate the selection of the items to be removed from the removal itself.
This has the following advantages:
If during the selection process, an exception occurs, the list will be left untouched
The removal process can only fail in catastrophic cases (OutOfMemoryException etc.)
But of course also some disadantages:
it requires extra memory to store the intermediate selection result
some optimizations might not be as effective
Because of the mentioned optimizations, I chose to base the selection result on ranges instead of individual indexes, so we can use List.RemoveRange which if more effective than individual RemoveAt calls (assumed that there are in fact ranges with more than one element).
public static List<(int start, int count)> GetIndexRanges<T>(this List<T> list,
Func<T, int, bool> predicate)
{
var result = new List<(int start, int count)>();
int start = -1;
for (var i = 0; i < list.Count; i++)
{
// see note 1 below
bool toBeRemoved = predicate(list[i], i);
if (toBeRemoved)
{
if (start < 0)
start = i; // new range starts
}
else if (start >= 0)
{
// range finished
result.Add((start, i - start));
start = -1;
}
}
if (start >= 0)
{
// orphan range at the end
result.Add((start, list.Count - start));
}
return result;
}
public static int RemoveIndexRanges<T>(this List<T> list,
List<(int start, int count)> ranges)
{
var removed = 0;
foreach (var range in ranges)
{
// the "- removed" is there to take into account
// that deletion moves the indexes.
list.RemoveRange(range.start - removed, range.count);
removed += range.count;
}
return removed;
}
Usage:
var ranges = list.GetIndexRanges((item, index) =>
{
//if (item == 10) throw new Exception();
return item % 2 == 1;
});
// See note 2 below
list.RemoveIndexRanges(ranges);
Note 1: As is, an exception in the predicate would just be propagated during the selection process, with no change to the ecollection. To give the caller more control over this, the following could be done: extend GetIndexRanges to still return everything collected so far, and in addition also return any exception as out parameter:
public static List<(int start, int count)> GetIndexRanges<T>(this List<T> list,
Func<T, int, bool> predicate, out Exception exception)
{
var result = new List<(int start, int count)>();
int start = -1;
for (var i = 0; i < list.Count; i++)
{
bool toBeRemoved = false;
try
{
toBeRemoved = predicate(list[i], i);
}
catch (Exception e)
{
exception = e;
break; // omit this line to continue with the selection process
}
if (toBeRemoved)
{
if (start < 0)
start = i; // new range starts
}
else if (start >= 0)
{
// range finished
result.Add((start, i - start));
start = -1;
}
}
if (start >= 0)
{
// orphan range at the end
result.Add((start, list.Count - start));
}
return result;
}
var ranges = list.GetIndexRanges((item, index) =>
{
if (item == 10) throw new Exception();
return item % 2 == 1;
}, out var exception);
// to fulfil requirement #3, we remove the ranges collected so far
// even in case of an exception
list.RemoveIndexRanges(ranges);
// and then throw the exception afterwards
if (exception != null)
ExceptionDispatchInfo.Capture(exception).Throw();
Note 2: As this is now a two-step process, it will fail if the list changes between the calls.

I think that I've managed to come up with an implementation that satisfies all three requirements:
/// <summary>
/// Removes all the elements that match the conditions defined by the specified
/// predicate. In case the predicate fails, the integrity of the list is preserved.
/// </summary>
public static int RemoveAll<T>(this List<T> list, Func<T, int, bool> predicate)
{
ArgumentNullException.ThrowIfNull(list);
ArgumentNullException.ThrowIfNull(predicate);
Span<T> span = CollectionsMarshal.AsSpan(list);
int i = 0, j = 0;
try
{
for (; i < span.Length; i++)
{
if (predicate(span[i], i)) continue;
if (j < i) span[j] = span[i];
j++;
}
}
finally
{
if (j < i)
{
for (; i < span.Length; i++, j++)
span[j] = span[i];
list.RemoveRange(j, span.Length - j);
}
}
return i - j;
}
For better performance it uses the CollectionsMarshal.AsSpan method (.NET 5) to get a Span<T> out of the list. The algorithm works just as well by using the indexer of the list instead of the span, and replacing the span.Length with list.Count.
Online demo.
I haven't benchmark this implementation, but I expect it to be only marginally slower than the native implementation.

So they don't think that there is any bug that needs to be fixed. They also believe that this behavior is not surprising or unexpected, so there is no need to document it.
They're correct. The method is documented as:
Removes all the elements that match the conditions defined by the specified predicate.
This supports two scenarios: the predicate returning true, removing an element, or false for leaving it as-is. A predicate throwing an exception is not a use case intended to be supported.
If you want to be able to pass a predicate that may throw, you could wrap it like this:
public static int RemoveAll<T>(this List<T> list, Func<T, int, bool> predicate)
{
Exception? caught = null;
int index = 0;
int removed = 0;
list.RemoveAll(item =>
{
// Ignore the rest of the list once thrown
if (caught != null) return false;
try
{
var remove = predicate(item, index);
if (remove)
{
removed++;
}
return remove;
}
catch (Exception e)
{
caught = e;
return false;
}
index++;
});
if (caught != null)
{
throw caught;
}
return removed;
}

I don't know microsoft is how to wrote this method.
I tried some code block. And i found case.
Actually problem is your throw new Exception(). If you dont this code that time yo code will run perfect. Exception trigger some another case. But i dont know what is that.
if (item >= 10) return false;
bool removeIt = item % 2 == 1;
if (removeIt) Console.WriteLine($"Removing #{item}");
return removeIt;
I found this. EDIT
Actually Func<T, int, bool> property is not deleted some item. It return boolean. As if return true he succesful deleted from list. If return false. it is not deleted from list.

Code Complexity Misunderstanding of Single Element in a Sorted Array

The problem which is on LeetCode says that
You are given a sorted array consisting of only integers where every
element appears exactly twice, except for one element which appears
exactly once.
Return the single element that appears only once.
Your solution must run in O(log n) time and O(1) space.
Example 1:
Input: nums = [1,1,2,3,3,4,4,8,8]
Output: 2
Example 2:
Input: nums = [3,3,7,7,10,11,11]
Output: 10
My friend said that leetcode has accepted it as one of its right solution as you can see below image. However, I can't understand how the code is O(logn). Could you explain me? I assert that the code is of O(n) because it iterates through one by on up to its size.
public class Solution {
public int SingleNonDuplicate(int[] nums) {
int result = nums[0];
for (int i = 0; i < nums.Length; i++)
{
if (nums[i] != result && i % 2 == 1)
{
result = nums[i - 1];
return result;
}
else
{
result = nums[i];
}
}
return result;
}
}

Interview function with time complexity

I had an interview question to write a program in C# that Outputs odd number of occurrences in an array.
Example: [2, 2, 3, 3, 3] => [3] (Considering the array is sorted)
My solution was:
public list<int> OddOccurance(list<int> InputList)
{
list<int> output = new list<int>();
for(int i=0; i<InputList.length; i++)
{
int Count = 0;
for(int j=1; j<(InputList.length-1); j++)
{
if(InputList[i] == InputList[j])
{
Count++;
}
}
if(Count % 2 != 0)
{
output.add(InputList[i]);
}
}
return output.distinct();
}
I am thinking the answer is correct only but the interviewer had asked me like different ways of how I can make the solution much faster.
Can anyone please tell me the time complexity of the above solution please.
If there is a way to make the above solution much faster then what can be the time complexity of that solution.

Your solution is O(n^2) - if you don't know why - evaluate sum:
This is an equation which describes the running time of your algorithm. You can solve it in linear time easily - just increment i instead of inner loop over all values in array.
for (int i=0; i<InputList.Length; ++i)
{
int currentValue = InputList[i];
int j=i+1;
int count = 1;
while (InputList[j] == currentValue && j<InputList.Length)
{
count++;
i++;
j++;
}
if (count % 2 == 0)
..
}
If array is not sorted - use dictionary (hash table - Dictionary in C#) - value is a dictionary key, count is a dictionary value. (that will give you Contains key check in O(1)) Another way to get linear time if implemented properly.

The root problem of your solution is seen on this line:
return output.Distinct();
The very fact that you are doing a Distinct means that you may be adding more entries than you should.
So how can you optimize it? Observe that since the array is sorted, the only place where you can find a number that's the same as the one you're looking at is next to it, or next to another number that's equal to your current number. In other words, your numbers go in "runs".
This observation lets you go from two nested loops and an O(N2) solution to a single loop and an O(N) solution. Simply walk the array, and check lengths of each "run": when you see a new number, store its index. If you come across a new number, see if the length of the "run" is odd, and start a new run:
int start = 0;
int pos = 1;
while (pos < InputList.Length) {
if (InputList[pos] != InputList[start]) {
if ((pos-start) % 2 == 1) {
output.Add(InputList[start]);
}
start = pos;
}
pos++;
}
// Process the last run
if ((InputList.Length-start) % 2 == 1) {
output.Add(InputList[start]);
}
Demo.

Removing masked entries from an array

The task is to keep an array of objects untouched if input is null and, otherwise, remove the elements that are on positions specified by the input. I've got it working but I'm vastly dissatisfied with the code quality.
List<Stuff> stuff = new List<Stuff>{ new Stuff(1), new Stuff(2), new Stuff(3) };
String input = "5";
if(input == null)
return stuff;
int mask = Int32.Parse(input);
for (int i = stuff.Count - 1; i >= 0; i--)
if ((mask & (int)Math.Pow(2, i)) == 0)
stuff.RemoveAt(i);
return stuff;
The actual obtaining input and the fact that e.g. String.Empty will cause problems need not to be regarded. Let's assume that those are handled.
How can I make the code more efficient?
How can I make the syntax more compact and graspable?

Instead of the backwards running loop, you could use Linq with the following statement.
stuff = stuff.Where( (iStuff, idx) => (mask & (int)Math.Pow(2, idx)) != 0 );
Or even cooler using bitwise shit.
stuff = stuff.Where((_, index) => (mask >> index & 1) == 1);
It uses an overload of Where which can access the position in the sequence, as documented here. For a similar task, there is also an overload of Select which gives access to the index, as documented here.

Untested, but you could make an extension method that iterates the collection and filters, returning matching elements as it goes. Repeatedly bit-shifting the mask and checking the 0th bit seems the easiest to follow - for me at least.
static IEnumerable<T> TakeMaskedItemsByIndex(this IEnumerable<T> collection, ulong mask)
{
foreach (T item in collection)
{
if((mask & 1) == 1)
yield return item;
mask = mask >> 1;
}
}

Function to look through list and determine trend

So I have a list of items. Each item on the list has a property called notional. Now, the list is already sorted. What I need to do is, develop a function that sets the type of list to one of the following:
Bullet - notional is the same for every item
Amortizing - notional decreases over the course of the schedule (might stay the same from element to element but it should never go up, and should end lower)
Accreting - notional increases over the course of the schedule (might stay the same from element to element but it should never go down, and should end higher)
Rollercoaster - notional goes up and down (could end the same, higher, or lower, but shouldn't be the same for each element and shouldn't be classfied as the other types)
What would this method look like and what would be the most efficient way to go through the list and figure this out?
Thanks!

This would be a straightforward way to do it:
bool hasGoneUp = false;
bool hasGoneDown = false;
T previous = null; // T is the type of objects in the list; assuming ref type
foreach(var item in list)
{
if (previous == null) {
previous = item;
continue;
}
hasGoneUp = hasGoneUp || item.notional > previous.notional;
hasGoneDown = hasGoneDown || item.notional < previous.notional;
if(hasGoneUp && hasGoneDown) {
return Trend.Rollercoaster;
}
previous = item;
}
if (!hasGoneUp && !hasGoneDown) {
return Trend.Bullet;
}
// Exactly one of hasGoneUp and hasGoneDown is true by this point
return hasGoneUp ? Trend.Accreting : Trend.Amortizing;

Let trendOut = Bullet
Loop from First Item to Last item
2.1. If previous notional < next notional
2.1.a. If trendOut = Amortizing return RollerCoaster
2.1.b. Else set trendOut = Accreting
2.2. if Previous Notional > next notional
2.2.a. If trendOut = Accreting return RollerCoaster
2.2.b. Else set trendOut = Amortizing
return trendOut.

You could probably do something as simple as this
var changeList = new List<Integer>
for(i = 0; i < yourList.Count() - 1; i++)
{
changeList.Add(yourList.Item(i + 1) - yourList.Item(i));
}
//Determine if the nature of the list
var positiveChangeCount = changeList.Where(x => x < 0);
var negativeChangeCount = changeList.Where(x => X > 0);
if (positiveChangeCount = yourList.Count)
{
Accreting;
}
elseif (negativeChangeCount = yourList.Count)
{
Amortizing;
}
elseif (negativeChangeCount + PositiveChangeCount = 0)
{
Bullet;
}
else
{
Rollercoaster;
}

I usually start of by optimizing for simplicity first and then performance. Hence, I would start by making a second list of N-1 elements, whose {elements} are differences between the {notionals} of the first list.
Hence, for the second list, I would expect the following for the list of your needs
Bullet - ALL elements are 0
Amortising - ALL elements stay 0 or negative
Accreting - ALL elements stay 0 or positive
Rollercoaster - Elements oscillate between negative & positive
You can probably optimize it an do it in one pass. Basically, this is a discrete differentiation over your data.

bool OnlyGreaterOrEqual=true;
bool OnlyLessOrEqual=true;
foreach(int i=1;i<itemList.Count;i++){
if(itemList[i].notional>itemList[i-1].notional){
OnlyLessOrEqual=false;
}else if(itemList[i].notional<itemList[i-1].notional){
OnlyGreaterOrEqual=false;
}
}
if(OnlyGreaterOrEqual && OnlyLessOrEqual){
return "Bullet";
}else if(OnlyGreaterOrEqual){
return "Accreting":
}else if(OnlyLessOrEqual){
return "Amortizing";
}else{
return "RollerCoast";
}

This is basically a Linq implementation of Danish's answer. It'll require (worst case) 3 passes through the list, but because they are so small it won't really matter from a performance point of view. (I wrote it to work on a list of ints so you'll have to modify it easily to work with your types).
var tmp = values
.Skip(1)
.Zip( values, (first, second) => first - second )
.ToList();
var up = tmp.Any( t => t > 0 );
var down = tmp.Any( t => t < 0 );
if( up && down )
// Rollercoaster
else if( up )
// Accreting
else if( down )
// Amortizing
else
// Bullet
You could also (ab)use the Aggregate operator and Tuple to do it as one query. However, this will fail if the collection is empty and is a bit weird to use in production code.
var result = values.Skip(1).Aggregate(
Tuple.Create<int, bool, bool>( values.First(), false, false ),
( last, current ) => {
return Tuple.Create(
current,
last.Item2 || (current - last.Item1) > 0,
last.Item3 || (current - last.Item1) < 0 );
});
result will be a tuple that contains:
the last element of the collection (which is of no use)
Item2 will contain a boolean indicating whether any element was bigger than the previous element
Item3 will contain a boolean indicating whether any element was smaller than the previous element
The same switch statement as above can be used to decide which pattern your data follows.