Make using statement usable for multiple disposable objects - c#

I have a bunch of text files in a folder, and all of them should have identical headers. In other words the first 100 lines of all files should be identical. So I wrote a function to check this condition:
private static bool CheckHeaders(string folderPath, int headersCount)
{
var enumerators = Directory.EnumerateFiles(folderPath)
.Select(f => File.ReadLines(f).GetEnumerator())
.ToArray();
//using (enumerators)
//{
for (int i = 0; i < headersCount; i++)
{
foreach (var e in enumerators)
{
if (!e.MoveNext()) return false;
}
var values = enumerators.Select(e => e.Current);
if (values.Distinct().Count() > 1) return false;
}
return true;
//}
}
The reason I am using enumerators is memory efficiency. Instead of loading all file contents in memory I enumerate the files concurrently line-by-line until a mismatch is found, or all headers have been examined.
My problem is evident by the commented lines of code. I would like to utilize a using block to safely dispose all the enumerators, but unfortunately using (enumerators) doesn't compile. Apparently using can handle only a single disposable object. I know that I can dispose the enumerators manually, by wrapping the whole thing in a try-finally block, and running the disposing logic in a loop inside finally, but is seems awkward. Is there any mechanism I could employ to make the using statement a viable option in this case?
Update
I just realized that my function has a serious flaw. The construction of the enumerators is not robust. A locked file can cause an exception, while some enumerators have already been created. These enumerators will not be disposed. This is something I want to fix. I am thinking about something like this:
var enumerators = Directory.EnumerateFiles(folderPath)
.ToDisposables(f => File.ReadLines(f).GetEnumerator());
The extension method ToDisposables should ensure that in case of an exception no disposables are left undisposed.

You can create a disposable-wrapper over your enumerators:
class DisposableEnumerable : IDisposable
{
private IEnumerable<IDisposable> items;
public event UnhandledExceptionEventHandler DisposalFailed;
public DisposableEnumerable(IEnumerable<IDisposable> items) => this.items = items;
public void Dispose()
{
foreach (var item in items)
{
try
{
item.Dispose();
}
catch (Exception e)
{
var tmp = DisposalFailed;
tmp?.Invoke(this, new UnhandledExceptionEventArgs(e, false));
}
}
}
}
and use it with the lowest impact to your code:
private static bool CheckHeaders(string folderPath, int headersCount)
{
var enumerators = Directory.EnumerateFiles(folderPath)
.Select(f => File.ReadLines(f).GetEnumerator())
.ToArray();
using (var disposable = new DisposableEnumerable(enumerators))
{
for (int i = 0; i < headersCount; i++)
{
foreach (var e in enumerators)
{
if (!e.MoveNext()) return false;
}
var values = enumerators.Select(e => e.Current);
if (values.Distinct().Count() > 1) return false;
}
return true;
}
}
The thing is you have to dispose those objects separately one by one anyway. But it's up to you where to encapsulate that logic. And the code I've suggested has no manual try-finally,)

To the second part of the question. If I get you right this should be sufficient:
static class DisposableHelper
{
public static IEnumerable<TResult> ToDisposable<TSource, TResult>(this IEnumerable<TSource> source,
Func<TSource, TResult> selector) where TResult : IDisposable
{
var exceptions = new List<Exception>();
var result = new List<TResult>();
foreach (var i in source)
{
try { result.Add(selector(i)); }
catch (Exception e) { exceptions.Add(e); }
}
if (exceptions.Count == 0)
return result;
foreach (var i in result)
{
try { i.Dispose(); }
catch (Exception e) { exceptions.Add(e); }
}
throw new AggregateException(exceptions);
}
}
Usage:
private static bool CheckHeaders(string folderPath, int headersCount)
{
var enumerators = Directory.EnumerateFiles(folderPath)
.ToDisposable(f => File.ReadLines(f).GetEnumerator())
.ToArray();
using (new DisposableEnumerable(enumerators))
{
for (int i = 0; i < headersCount; i++)
{
foreach (var e in enumerators)
{
if (!e.MoveNext()) return false;
}
var values = enumerators.Select(e => e.Current);
if (values.Distinct().Count() > 1) return false;
}
return true;
}
}
and
try
{
CheckHeaders(folderPath, headersCount);
}
catch(AggregateException e)
{
// Prompt to fix errors and try again
}

I'm going to suggest an approach that uses recursive calls to Zip to allow parallel enumeration of a normal IEnumerable<string> without the need to resort to using IEnumerator<string>.
bool Zipper(IEnumerable<IEnumerable<string>> sources, int take)
{
IEnumerable<string> ZipperImpl(IEnumerable<IEnumerable<string>> ss)
=> (!ss.Skip(1).Any())
? ss.First().Take(take)
: ss.First().Take(take).Zip(
ZipperImpl(ss.Skip(1)),
(x, y) => (x == null || y == null || x != y) ? null : x);
var matching_lines = ZipperImpl(sources).TakeWhile(x => x != null).ToArray();
return matching_lines.Length == take;
}
Now build up your enumerables:
IEnumerable<string>[] enumerables =
Directory
.EnumerateFiles(folderPath)
.Select(f => File.ReadLines(f))
.ToArray();
Now it's simple to call:
bool headers_match = Zipper(enumerables, 100);
Here's a trace of running this code against three files with more than 4 lines:
Ben Petering at 5:28 PM ACST
Ben Petering at 5:28 PM ACST
Ben Petering at 5:28 PM ACST
From a call 2019-05-23, James mentioned he’d like the ability to edit the current shipping price rules (eg in shipping_rules.xml) via the admin.
From a call 2019-05-23, James mentioned he’d like the ability to edit the current shipping price rules (eg in shipping_rules.xml) via the admin.
From a call 2019-05-23, James mentioned he’d like the ability to edit the current shipping price rules (eg in shipping_rules.xml) via the admin.
He also mentioned he’d like to be able to set different shipping price rules for a given time window, e.g. Jan 1 to Jan 30.
He also mentioned he’d like to be able to set different shipping price rules for a given time window, e.g. Jan 1 to Jan 30.
He also mentioned he’d like to be able to set different shipping price rules for a given time window, e.g. Jan 1 to Jan 30.
These storyishes should be considered when choosing the appropriate module to use.
These storyishes should be considered when choosing the appropriate module to use.X
These storyishes should be considered when choosing the appropriate module to use.
Note that the enumerations stop when they encountered a mismatch header in the 4th line on the second file. All enumerations then stopped.

Creating an IDisposable wrapper as #Alex suggested is correct. It needs just a logic to dispose already opened files if some of them is locked and probably some logic for error states. Maybe something like this (error state logic is very simple):
public class HeaderChecker : IDisposable
{
private readonly string _folderPath;
private readonly int _headersCount;
private string _lockedFile;
private readonly List<IEnumerator<string>> _files = new List<IEnumerator<string>>();
public HeaderChecker(string folderPath, int headersCount)
{
_folderPath = folderPath;
_headersCount = headersCount;
}
public string LockedFile => _lockedFile;
public bool CheckFiles()
{
_lockedFile = null;
if (!TryOpenFiles())
{
return false;
}
if (_files.Count == 0)
{
return true; // Not sure what to return here.
}
for (int i = 0; i < _headersCount; i++)
{
if (!_files[0].MoveNext()) return false;
string currentLine = _files[0].Current;
for (int fileIndex = 1; fileIndex < _files.Count; fileIndex++)
{
if (!_files[fileIndex].MoveNext()) return false;
if (_files[fileIndex].Current != currentLine) return false;
}
}
return true;
}
private bool TryOpenFiles()
{
bool result = true;
foreach (string file in Directory.EnumerateFiles(_folderPath))
{
try
{
_files.Add(File.ReadLines(file).GetEnumerator());
}
catch
{
_lockedFile = file;
result = false;
break;
}
}
if (!result)
{
DisposeCore(); // Close already opened files.
}
return result;
}
private void DisposeCore()
{
foreach (var item in _files)
{
try
{
item.Dispose();
}
catch
{
}
}
_files.Clear();
}
public void Dispose()
{
DisposeCore();
}
}
// Usage
using (var checker = new HeaderChecker(folderPath, headersCount))
{
if (!checker.CheckFiles())
{
if (checker.LockedFile is null)
{
// Error while opening files.
}
else
{
// Headers do not match.
}
}
}
I also removed .Select() and .Distinct() when checking the lines. The first just iterates over the enumerators array - the same as foreach above it, so you are enumerating this array twice. Then creates a new list of lines and .Distinct() enumerates over it.

Related

Find exception hiding/swallowing in C# code in VS2013

is there some way built in function/extension/tool to find all exception hidings/exception swallowing in C# solution(ASP.NET WebForms)n in VS2013.
Thanks
EDIT:
I have existing solution in which some programmers use hide/swallow exceptions(empty catch, catch only with some useless code). And I am looking for some way to find all these places in code, analyze them, and then fix them.
You can write some code using Roslyn to handle this pretty easily.
I actually wrote some code to do exactly that for a friend. It was my first attempt at using the Roslyn SDK, so my code is probably a terrible mess, but it was definitely functional.
static void Main(string[] args)
{
var result = Microsoft.CodeAnalysis.CSharp.CSharpSyntaxTree.ParseFile(#"..\..\Test.cs");
var root = result.GetRoot();
var exceptionNodes = FindCatchNodes(root);
foreach (var node in exceptionNodes)
{
var line = node.GetLocation().GetLineSpan().StartLinePosition.Line + 1;
if (IsTotallyEmptyCatch(node))
{
Console.WriteLine("Totally empty catch: line {0}", line);
}
if (JustRethrows(node))
{
Console.WriteLine("Pointless rethrow: line {0}", line);
}
}
}
static List<SyntaxNodeOrToken> FindCatchNodes(SyntaxNodeOrToken node)
{
var exceptions = new List<SyntaxNodeOrToken>();
var isCatchBlock = node.IsKind(SyntaxKind.CatchClause);
if (isCatchBlock)
{
exceptions.Add(node);
}
foreach (var result in node.ChildNodesAndTokens().Select(FindCatchNodes).Where(result => result != null))
{
exceptions.AddRange(result);
}
return exceptions;
}
static bool IsTotallyEmptyCatch(SyntaxNodeOrToken catchBlock)
{
var block = catchBlock.ChildNodesAndTokens().First(t => t.CSharpKind() == SyntaxKind.Block);
var children = block.ChildNodesAndTokens();
return (children.Count == 2 && children.Any(c => c.CSharpKind() == SyntaxKind.OpenBraceToken) &&
children.Any(c => c.CSharpKind() == SyntaxKind.CloseBraceToken));
}
static bool JustRethrows(SyntaxNodeOrToken catchBlock)
{
var block = catchBlock.ChildNodesAndTokens().First(t => t.CSharpKind() == SyntaxKind.Block);
var children = block.ChildNodesAndTokens();
return (children.Count == 3 && children.Any(c => c.CSharpKind() == SyntaxKind.OpenBraceToken) &&
children.Any(c => c.CSharpKind() == SyntaxKind.CloseBraceToken) && children.Any(c=>c.CSharpKind() == SyntaxKind.ThrowStatement));
}
Given this test file:
using System;
namespace RoslynTest
{
public class Test
{
public void Foo()
{
try
{
var x = 0;
}
catch
{
}
}
public void Bar()
{
try
{
var x = 0;
}
catch (Exception ex)
{
throw;
}
}
public void Baz()
{
try
{
var x = 0;
}
catch (Exception ex)
{
throw ex;
}
}
}
}
The output is:
Totally empty catch: ....\Test.cs: line 12
Pointless rethrow: ....\Test.cs: line 24
Pointless rethrow: ....\Test.cs: line 37
I don't know about the built-in methods. But you can write your own tool to find such places. Just regex all your files in solution and count catch and throw. There should be the same amount for each file :)

Get used classes inside a dll

I'm inspecting a dll file and I need to know which classes of other dll files are used in this dll file. With the first loop it's possible to iterate through all classes of the dll- but now I want to get a list of all classes, which are used in some way in this class (those classes have a specific naming convention "Some.Test.Class*").
Could the solution be to parse every instruction of every method of the class and than searching in the instruction for the name of the class?
Does anybody have a better idea?
foreach (TypeDefinition type in this.currentAssembly.MainModule.Types)
{
foreach (MethodDefinition method in type.Methods)
{
if (method.HasBody)
{
for (int cnt = 0; cnt < method.Body.Instructions.Count - 1; cnt++)
{
Instruction instruction = method.Body.Instructions[cnt];
/*
????????
*/
}
}
}
}
P.S.: I must not load some referenced dll files
I am using Mono.Cecil:
private static List<MethodDefinition> GetAllInnerMethods(MethodDefinition method)
{
var queueMethodDefinition = new Queue<MethodDefinition>();
var hsMethodDefinition = new HashSet<MethodDefinition>();
queueMethodDefinition.Enqueue(method);
while (queueMethodDefinition.Count > 0)
{
MethodDefinition methodCurr = queueMethodDefinition.Dequeue();
if (!hsMethodDefinition.Contains(methodCurr))
{
hsMethodDefinition.Add(methodCurr);
IEnumerable<MethodDefinition> innerMethodsByCaller = GetInnerMethodsByCaller(methodCurr);
foreach (var currInnerMethod in innerMethodsByCaller)
{
if (!hsMethodDefinition.Contains(currInnerMethod))
{
queueMethodDefinition.Enqueue(currInnerMethod);
}
}
}
}
return hsMethodDefinition.ToList();
}
private static IEnumerable<MethodDefinition> GetInnerMethodsByCaller(MethodDefinition caller)
{
return caller.Body.Instructions
.Where(x => (x.OpCode == OpCodes.Call || x.OpCode == OpCodes.Calli || x.OpCode == OpCodes.Callvirt) && x.Operand is MethodDefinition)
.Select(x => (MethodDefinition)x.Operand);
}

compare List<string> and List<T>

I'm using C# and framework 4.0.
I have a list of type string and another list of type class T;
How can I compare List with a List and save the difference?
private void simpleButton_Compare_Click(object sender, EventArgs e)
{
try
{
bool Is_Egal = true;
int i = 0;
foreach (string Od_Scan in Ordre_Scan)
{
if (!Outils.Get_Ordre_Donne()[i].NoOrdre.Contains(Od_Scan) && !String.IsNullOrWhiteSpace(Od_Scan))
{
Is_Egal = false;
Temp_Od_Scan.Add(Od_Scan);
}
i++;
}
foreach (Pers_Compare Od_Done in Outils.Get_Ordre_Donne())
{
if (!Ordre_Scan.Contains(Od_Done.NoOrdre) && !String.IsNullOrWhiteSpace(Od_Done.NoOrdre))
{
Is_Egal = false;
Temp_Od_Donne.Add(Od_Done);
}
else
{
Temp_Od_Donne_Egal.Add(Od_Done);
}
}
if (Is_Egal)
{
MessageBox.Show("égalité");
}
else
{
MessageBox.Show("PAS égalité");
}
}
catch (Exception excThrown)
{
MessageBox.Show(excThrown.Message);
}
}
and the data :
List<string> Ordre_Scan= new List<string> { "azer","qsdf"};
Pers_Compare obj = new Pers_Compare();
obj.Nolv = 1;
obj.Noordre = "qsdf"
Pers_Compare obj2 = new Pers_Compare();
obj2.Nolv = 1;
obj2.Noordre = "wxcv"
List<Pers_Compare> Ordre_Donne = new List<Pers_Compare>();
Ordre_Donne.add(obj);
Ordre_Donne.add(obj2);
And I want to save the data in Ordre_Donne but not in Od_Scan and vice versa.
foreach (string Od_Scan in Temp_Od_Scan)
{
all item that not found in List A
--> wxcv
}
foreach (var Od_Done in Temp_Od_Donne)
{
all item that not found in List B
--> azer
}
The answer given for a slightly different question (comparing a List with another List) seems to me to be a good solution for your issue, they address multiple issues to do with comparisons of lists.
EDIT: However you should be more specific with your requirements i.e. what exactly is a 'difference', e.g. is {1,1,2} and {1,2} the same?
Here is the answer given the most votes... (included here just encase it gets removed for some reason (as per Bob' suggestion))
"
DESCRIPTION:
I need to check that they both have the same elements, regardless of their position within the list. Each MyType object may appear multiple times on a list. Is there a built-in function that checks this? What if I guarantee that each element appears only once in a list?
EDIT: Guys thanks for the answers but I forgot to add something, the number of occurrences of each element should be the same on both lists.
ANSWER:
If you want them to be really equal (i.e. the same items and the same number of each item), I think that the simplest solution is to sort before comparing:
Enumerable.SequenceEqual(list1.OrderBy(t => t), list2.OrderBy(t => t))
Edit:
Here is a solution that performs a bit better (about ten times faster), and only requires IEquatable, not IComparable:
public static bool ScrambledEquals<T>(IEnumerable<T> list1, IEnumerable<T> list2) {
var cnt = new Dictionary<T, int>();
foreach (T s in list1) {
if (cnt.ContainsKey(s)) {
cnt[s]++;
} else {
cnt.Add(s, 1);
}
}
foreach (T s in list2) {
if (cnt.ContainsKey(s)) {
cnt[s]--;
} else {
return false;
}
}
return cnt.Values.All(c => c == 0);
}
Edit 2:
To handle any data type as key (for example nullable types as Frank Tzanabetis pointed out), you can make a version that takes a comparer for the dictionary:
public static bool ScrambledEquals<T>(IEnumerable<T> list1, IEnumerable<T> list2, IEqualityComparer<T> comparer) {
var cnt = new Dictionary<T, int>(comparer);
...
"
var list1 = Ordre_Donne.Where(o => !Ordre_Scan.Any(s => s == o.Noordre));
var list2 = Ordre_Scan.Where(s => !Ordre_Donne.Any(o => o.Noordre == s));
You can either implement IComparable on your Pers_Compare class, which will look something like:
public int CompareTo(string other)
{
return this.Noordre.CompareTo(other);
}
Or, if you don't have control of the data structure, you could do something like
var Temp_Od_Donne = from od in Ordre_Donne
where !Ordre_Scan.Contains(od.Noordre)
select od;
var Temp_Od_Scan = from os in Ordre_Scan
where !Ordre_Donne.Select(od => od.Noordre).Contains(os)
select os;

List<T>Get Chunk Number being executed

I am breaking a list into chunks and processing it as below:
foreach (var partialist in breaklistinchunks(chunksize))
{
try
{
do something
}
catch
{
print error
}
}
public static class IEnumerableExtensions
{
public static IEnumerable<List<T>> BreakListinChunks<T>(this IEnumerable<T> sourceList, int chunkSize)
{
List<T> chunkReturn = new List<T>(chunkSize);
foreach (var item in sourceList)
{
chunkReturn.Add(item);
if (chunkReturn.Count == chunkSize)
{
yield return chunkReturn;
chunkReturn = new List<T>(chunkSize);
}
}
if (chunkReturn.Any())
{
yield return chunkReturn;
}
}
}
If there is an error, I wish to run the chunk again. Is it possible to find the particular chunk number where we received the error and run that again ?
The batches have to be executed in sequential order .So if batch#2 generates an error, then I need to be able to run 2 again, if it fails again. I just need to get out of the loop for good .
List<Chunk> failedChunks = new List<Chunk>();
foreach (var partialist in breaklistinchunks(chunksize))
{
try
{
//do something
}
catch
{
//print error
failedChunks.Add(partiallist);
}
}
// attempt to re-process failed chunks here
I propose this answer based on your comment to Aaron's answer.
The batches have to be executed in sequential order .So if 2 is a problem , then I need to be able to run 2 again, if it fails again. I just need to get out of the loop for good.
foreach (var partialist in breaklistinchunks(chunksize))
{
int fails = 0;
bool success = false;
do
{
try
{
// do your action
success = true; // should be on the last line before the 'catch'
}
catch
{
fails += 1;
// do something about error before running again
}
}while (!success && fails < 2);
// exit the iteration if not successful and fails is 2
if (!success && fails >= 2)
break;
}
I made a possible solution for you if you don't mind switching from Enumerable to Queue, which kind of fits given the requirements...
void Main()
{
var list = new Queue<int>();
list.Enqueue(1);
list.Enqueue(2);
list.Enqueue(3);
list.Enqueue(4);
list.Enqueue(5);
var random = new Random();
int chunksize = 2;
foreach (var chunk in list.BreakListinChunks(chunksize))
{
foreach (var item in chunk)
{
try
{
if(random.Next(0, 3) == 0) // 1 in 3 chance of error
throw new Exception(item + " is a problem");
else
Console.WriteLine (item + " is OK");
}
catch (Exception ex)
{
Console.WriteLine (ex.Message);
list.Enqueue(item);
}
}
}
}
public static class IEnumerableExtensions
{
public static IEnumerable<List<T>> BreakListinChunks<T>(this Queue<T> sourceList, int chunkSize)
{
List<T> chunkReturn = new List<T>(chunkSize);
while(sourceList.Count > 0)
{
chunkReturn.Add(sourceList.Dequeue());
if (chunkReturn.Count == chunkSize || sourceList.Count == 0)
{
yield return chunkReturn;
chunkReturn = new List<T>(chunkSize);
}
}
}
}
Outputs
1 is a problem
2 is OK
3 is a problem
4 is a problem
5 is a problem
1 is a problem
3 is OK
4 is OK
5 is OK
1 is a problem
1 is OK
One possibility would be to use a for loop instead of a foreach loop and use the counter as a means to determine where an error occurred. Then you could continue from where you left off.
You can use break to exit out of the loop as soon as a chunk fails twice:
foreach (var partialList in breaklistinchunks(chunksize))
{
if(!TryOperation(partialList) && !TryOperation(partialList))
{
break;
}
}
private bool TryOperation<T>(List<T> list)
{
try
{
// do something
}
catch
{
// print error
return false;
}
return true;
}
You could even make the loop into a one-liner with LINQ, but it is generally bad practice to combine LINQ with side-effects, and it's not very readable:
breaklistinchunks(chunksize).TakeWhile(x => TryOperation(x) || TryOperation(x));

Parallel.Foreach + yield return?

I want to process something using parallel loop like this :
public void FillLogs(IEnumerable<IComputer> computers)
{
Parallel.ForEach(computers, cpt=>
{
cpt.Logs = cpt.GetRawLogs().ToList();
});
}
Ok, it works fine. But How to do if I want the FillLogs method return an IEnumerable ?
public IEnumerable<IComputer> FillLogs(IEnumerable<IComputer> computers)
{
Parallel.ForEach(computers, cpt=>
{
cpt.Logs = cpt.GetRawLogs().ToList();
yield return cpt // KO, don't work
});
}
EDIT
It seems not to be possible... but I use something like this :
public IEnumerable<IComputer> FillLogs(IEnumerable<IComputer> computers)
{
return computers.AsParallel().Select(cpt => cpt);
}
But where I put the cpt.Logs = cpt.GetRawLogs().ToList(); instruction
Short version - no, that isn't possible via an iterator block; the longer version probably involves synchronized queue/dequeue between the caller's iterator thread (doing the dequeue) and the parallel workers (doing the enqueue); but as a side note - logs are usually IO-bound, and parallelising things that are IO-bound often doesn't work very well.
If the caller is going to take some time to consume each, then there may be some merit to an approach that only processes one log at a time, but can do that while the caller is consuming the previous log; i.e. it begins a Task for the next item before the yield, and waits for completion after the yield... but that is again, pretty complex. As a simplified example:
static void Main()
{
foreach(string s in Get())
{
Console.WriteLine(s);
}
}
static IEnumerable<string> Get() {
var source = new[] {1, 2, 3, 4, 5};
Task<string> outstandingItem = null;
Func<object, string> transform = x => ProcessItem((int) x);
foreach(var item in source)
{
var tmp = outstandingItem;
// note: passed in as "state", not captured, so not a foreach/capture bug
outstandingItem = new Task<string>(transform, item);
outstandingItem.Start();
if (tmp != null) yield return tmp.Result;
}
if (outstandingItem != null) yield return outstandingItem.Result;
}
static string ProcessItem(int i)
{
return i.ToString();
}
I don't want to be offensive, but maybe there is a lack of understanding. Parallel.ForEach means that the TPL will run the foreach according to the available hardware in several threads. But that means, that ii is possible to do that work in parallel! yield return gives you the opportunity to get some values out of a list (or what-so-ever) and give them back one-by-one as they are needed. It prevents of the need to first find all items matching the condition and then iterate over them. That is indeed a performance advantage, but can't be done in parallel.
Although the question is old I've managed to do something just for fun.
class Program
{
static void Main(string[] args)
{
foreach (var message in GetMessages())
{
Console.WriteLine(message);
}
}
// Parallel yield
private static IEnumerable<string> GetMessages()
{
int total = 0;
bool completed = false;
var batches = Enumerable.Range(1, 100).Select(i => new Computer() { Id = i });
var qu = new ConcurrentQueue<Computer>();
Task.Run(() =>
{
try
{
Parallel.ForEach(batches,
() => 0,
(item, loop, subtotal) =>
{
Thread.Sleep(1000);
qu.Enqueue(item);
return subtotal + 1;
},
result => Interlocked.Add(ref total, result));
}
finally
{
completed = true;
}
});
int current = 0;
while (current < total || !completed)
{
SpinWait.SpinUntil(() => current < total || completed);
if (current == total) yield break;
current++;
qu.TryDequeue(out Computer computer);
yield return $"Completed {computer.Id}";
}
}
}
public class Computer
{
public int Id { get; set; }
}
Compared to Koray's answer this one really uses all the CPU cores.
You can use the following extension method
public static class ParallelExtensions
{
public static IEnumerable<T1> OrderedParallel<T, T1>(this IEnumerable<T> list, Func<T, T1> action)
{
var unorderedResult = new ConcurrentBag<(long, T1)>();
Parallel.ForEach(list, (o, state, i) =>
{
unorderedResult.Add((i, action.Invoke(o)));
});
var ordered = unorderedResult.OrderBy(o => o.Item1);
return ordered.Select(o => o.Item2);
}
}
use like:
public void FillLogs(IEnumerable<IComputer> computers)
{
cpt.Logs = computers.OrderedParallel(o => o.GetRawLogs()).ToList();
}
Hope this will save you some time.
How about
Queue<string> qu = new Queue<string>();
bool finished = false;
Task.Factory.StartNew(() =>
{
Parallel.ForEach(get_list(), (item) =>
{
string itemToReturn = heavyWorkOnItem(item);
lock (qu)
qu.Enqueue(itemToReturn );
});
finished = true;
});
while (!finished)
{
lock (qu)
while (qu.Count > 0)
yield return qu.Dequeue();
//maybe a thread sleep here?
}
Edit:
I think this is better:
public static IEnumerable<TOutput> ParallelYieldReturn<TSource, TOutput>(this IEnumerable<TSource> source, Func<TSource, TOutput> func)
{
ConcurrentQueue<TOutput> qu = new ConcurrentQueue<TOutput>();
bool finished = false;
AutoResetEvent re = new AutoResetEvent(false);
Task.Factory.StartNew(() =>
{
Parallel.ForEach(source, (item) =>
{
qu.Enqueue(func(item));
re.Set();
});
finished = true;
re.Set();
});
while (!finished)
{
re.WaitOne();
while (qu.Count > 0)
{
TOutput res;
if (qu.TryDequeue(out res))
yield return res;
}
}
}
Edit2: I agree with the short No answer. This code is useless; you cannot break the yield loop.

Categories