We Keep Coding

One longer call is causing all parallel loops to pause or blocked

I am calling a VB 6.0 dll in Parallel.ForEach and expecting all calls to be started simultaneously or at least 2 of them based on my PC's cores or threads availability in thread pool
VB6 dll
Public Function DoJunk(ByVal counter As Long, ByVal data As String) As Integer
Dim i As Long
Dim j As Long
Dim s As String
Dim fno As Integer
fno = FreeFile
Open "E:\JunkVB6Dll\" & data & ".txt" For Output Access Write As #fno
Print #fno, "Starting loop with counter = " & counter
For i = 0 To counter
Print #fno, "counting " & i
Next
Close #fno
DoJunk = 1
End Function
counter is being passed from the caller to control execution time of the call and file is being written to make it an IO based process.
C# caller
private void ReportProgress(int value)
{
progressBar.Value = value;
//progressBar.Value++;
}
private void button1_Click(object sender, EventArgs e)
{
progressBar.Value = 0;
counter = 0;
Stopwatch watch = new Stopwatch();
watch.Start();
//var range = Enumerable.Range(0, 100);
var range = Enumerable.Range(0, 20);
bool finished = false;
Task.Factory.StartNew(() =>
{
Parallel.ForEach(range, i =>
{
#region COM CALL
JunkProject.JunkClass junk = new JunkProject.JunkClass();
try
{
Random rnd = new Random();
int dice = rnd.Next(10, 40);
int val = 0;
if (i == 2)
val = junk.DoJunk(9000000, i.ToString());
else
val = junk.DoJunk(dice * 10000, i.ToString());
System.Diagnostics.Debug.Print(junk.GetHashCode().ToString());
if (val == 1)
{
Interlocked.Increment(ref counter);
progressBar.Invoke((Action)delegate { ReportProgress(counter); });
}
junk = null;
}
catch (Exception excep)
{
i = i;
}
finally { junk = null; }
#endregion
});
}).ContinueWith(t =>
{
watch.Stop();
MessageBox.Show(watch.ElapsedMilliseconds.ToString());
});
}
This line is making a specific call longer than the others.
val = junk.DoJunk(9000000, i.ToString());
Here this second process is causing all calls inside the Parallel.ForEach to stop i.e. no other file is created unless this 2nd call gets completed.
Is it an expected behavior or i am doing something wrong?

As #John Wu suggested that you can create AppDomain to allow COM to run on different App Domain, I believe you could run your parallel like this.
Parallel.ForEach(range, i =>
{
AppDomain otherDomain = AppDomain.CreateDomain(i.ToString());
otherDomain.DoCallBack(delegate
{
//Your COM call
});
});
EDIT
Right.. I am not sure how can you set serializable on VB6.0 class. You can try the other way (Marshaling objects by reference). Noted: I haven't actually tested this, but I would like to know if that will work.
Parallel.ForEach(range, i =>
{
AppDomain otherDomain = AppDomain.CreateDomain(i.ToString());
var comCall = (ComCall) otherDomain.CreateInstanceFromAndUnwrap(Assembly.GetExecutingAssembly().Location, typeof(ComCall).ToString());
comCall.Run();
AppDomain.Unload(otherDomain);
});
and the class
public class ComCall : MarshalByRefObject
{
public void Run()
{
//Your COM Call
}
}
Here is also additional reference regarding the topic.
https://www.codeproject.com/Articles/14791/NET-Remoting-with-an-easy-example

Simulating message latency accurately

I have written a simple "latency simulator" which works, but at times, messages are delayed for longer than the time specified. I need help to ensure that messages are delayed for the correct amount of time.
The main problem, I believe, is that I am using Thread.Sleep(x), which is depended on various factors but mainly on the clock interrupt rate, which causes Thread.Sleep() to have a resolution of roughly 15ms. Further, intensive tasks will demand more CPU time and will occasionally result in a delay greater than the one requested. If you are not familiar with the resolution issues of Thread.Sleep, you can read these SO posts: here, here and here.
This is my LatencySimulator:
public class LatencySimulatorResult: EventArgs
{
public int messageNumber { get; set; }
public byte[] message { get; set; }
}
public class LatencySimulator
{
private int messageNumber;
private int latency = 0;
private int processedMessageCount = 0;
public event EventHandler messageReady;
public void Delay(byte[] message, int delay)
{
latency = delay;
var result = new LatencySimulatorResult();
result.message = message;
result.messageNumber = messageNumber;
if (latency == 0)
{
if (messageReady != null)
messageReady(this, result);
}
else
{
ThreadPool.QueueUserWorkItem(ThreadPoolCallback, result);
}
Interlocked.Increment(ref messageNumber);
}
private void ThreadPoolCallback(object threadContext)
{
Thread.Sleep(latency);
var next = (LatencySimulatorResult)threadContext;
var ready = next.messageNumber == processedMessageCount + 1;
while (ready == false)
{
ready = next.messageNumber == processedMessageCount + 1;
}
if (messageReady != null)
messageReady(this, next);
Interlocked.Increment(ref processedMessageCount);
}
}
To use it, you create a new instance and bind to the event handler:
var latencySimulator = new LatencySimulator();
latencySimulator.messageReady += MessageReady;
You then call latencySimulator.Delay(someBytes, someDelay);
When a message has finished being delayed, the event is fired and you can then process the delayed message.
It is important that the order in which messages are added is maintained. I cannot have them coming out the other end of the latency simulator in some random order.
Here is a test program to use the latency simulator and to see how long messages have been delayed for:
private static LatencySimulator latencySimulator;
private static ConcurrentDictionary<int, PendingMessage> pendingMessages;
private static List<long> measurements;
static void Main(string[] args)
{
var results = TestLatencySimulator();
var anomalies = results.Result.Where(x=>x > 32).ToList();
foreach (var result in anomalies)
{
Console.WriteLine(result);
}
Console.ReadLine();
}
static async Task<List<long>> TestLatencySimulator()
{
latencySimulator = new LatencySimulator();
latencySimulator.messageReady += MessageReady;
var numberOfMeasurementsMax = 1000;
pendingMessages = new ConcurrentDictionary<int, PendingMessage>();
measurements = new List<long>();
var sendTask = Task.Factory.StartNew(() =>
{
for (var i = 0; i < numberOfMeasurementsMax; i++)
{
var message = new Message { Id = i };
pendingMessages.TryAdd(i, new PendingMessage() { Id = i });
latencySimulator.Delay(Serialize(message), 30);
Thread.Sleep(50);
}
});
//Spin some tasks up to simulate high CPU usage
Task.Factory.StartNew(() => { FindPrimeNumber(100000); });
Task.Factory.StartNew(() => { FindPrimeNumber(100000); });
Task.Factory.StartNew(() => { FindPrimeNumber(100000); });
sendTask.Wait();
return measurements;
}
static long FindPrimeNumber(int n)
{
int count = 0;
long a = 2;
while (count < n)
{
long b = 2;
int prime = 1;// to check if found a prime
while (b * b <= a)
{
if (a % b == 0)
{
prime = 0;
break;
}
b++;
}
if (prime > 0)
{
count++;
}
a++;
}
return (--a);
}
private static void MessageReady(object sender, EventArgs e)
{
LatencySimulatorResult result = (LatencySimulatorResult)e;
var message = (Message)Deserialize(result.message);
if (pendingMessages.ContainsKey(message.Id) != true) return;
pendingMessages[message.Id].stopwatch.Stop();
measurements.Add(pendingMessages[message.Id].stopwatch.ElapsedMilliseconds);
}
static object Deserialize(byte[] arrBytes)
{
using (var memStream = new MemoryStream())
{
var binForm = new BinaryFormatter();
memStream.Write(arrBytes, 0, arrBytes.Length);
memStream.Seek(0, SeekOrigin.Begin);
var obj = binForm.Deserialize(memStream);
return obj;
}
}
static byte[] Serialize<T>(T obj)
{
BinaryFormatter bf = new BinaryFormatter();
using (var ms = new MemoryStream())
{
bf.Serialize(ms, obj);
return ms.ToArray();
}
}
If you run this code, you will see that about 5% of the messages are delayed for more than the expected 30ms. In fact, some are as high as 60ms. Without any background tasks or high CPU usage, the simulator behaves as expected.
I need them all to be 30ms (or as close to as possible) - I do not want some arbitrary 50-60ms delays.
Can anyone suggest how I can refactor this code so that I can achieve the desired result, but without the use of Thread.Sleep() and with as little CPU overhead as possible?

Playing 30 sound effects at the same time repeatedly

I'm trying to play about 30 piano notes at the same time in my XNA application on Windows Phone 7.
I have imported and loaded the wave files like below
protected override void LoadContent()
{
spriteBatch = new SpriteBatch(GraphicsDevice);
sound3 = Content.Load<SoundEffect>("1");
sound5 = Content.Load<SoundEffect>("3");
sound6 = Content.Load<SoundEffect>("4");
sound7 = Content.Load<SoundEffect>("5");
sound8 = Content.Load<SoundEffect>("6");
}
Every sound effect file is less than a second, so I'm trying to play all of them at the same time. I play them using a for loop which runs every second.(So at every loop 30 sounds will be played then it goes on and plays the same 30 sounds each second)
It works fine for a few seconds but suddenly it stops playing any sound (The loop is still working) then again starts working for once or twice and again stops working . It also sometimes makes some bad noises as if the audio system cant support too many sounds to be play at a time.
I'm not sure how I can solve the problem , if its a buffer problem or threads.

Nico Schertler is right! I've faced the same problem and fixed it by managing instances. Normally when you play soundeffect you always get the instance! By default you have to hope XNA/Monogame will care about it, but it doesn't. When you have to many live instances (even stopped, you get breaks, noise, even silence.
I hold rectangular array for all my sound effects with up to 4 instance for each, and if I have instances of a sound I want to play instead of creating I stop the oldest (by timestamp), play it and remember the current timestamp.
See the following code:
private const int MAX_INST_OF_ONE_SOUND = 4;
private SoundEffectInstance[][] sndInstArray = null;
private float[][] sndInstTimes = null;
public init()
{
sndArray = new SoundEffect[SoundsSchema.sounds.Length];
sndInstArray = new SoundEffectInstance[SoundsSchema.sounds.Length][];
sndInstTimes = new float[SoundsSchema.sounds.Length][];
for (int i = 0; i < SoundsSchema.sounds.Length; i++)
{
try
{
sndArray[i] = content.Load<SoundEffect>(SoundsSchema.sounds[i]);//SoundsSchema is string list holder class
}
catch (System.Exception)
{
}
sndInstArray[i] = new SoundEffectInstance[MAX_INST_OF_ONE_SOUND];
sndInstTimes[i] = new float[MAX_INST_OF_ONE_SOUND];
}
}
private SoundEffectInstance getValidInstance(int sound)
{
if (sound < 0 || sound > sndInstArray.Length)
return null;
SoundEffectInstance inst = null;
for (int i = 0; i < MAX_INST_OF_ONE_SOUND; i++)
{
if (sndInstArray[sound][i] == null || (sndInstArray[sound][i] != null && sndInstArray[sound][i].IsDisposed))
{
sndInstArray[sound][i] = sndArray[sound].CreateInstance();
sndInstTimes[sound][i] = MyEngine.CurTime;
inst = sndInstArray[sound][i];
break;
}
}
if (inst == null)
{
float min_time = float.MaxValue;
int ind = -1;
for (int i = 0; i < MAX_INST_OF_ONE_SOUND; i++)
{
if (sndInstArray[sound][i] != null && sndInstTimes[sound][i] < min_time)
{
min_time = sndInstTimes[sound][i];
ind = i;
}
}
if (ind == -1)
ind = 0;
if (sndInstArray[sound][ind].IsDisposed)
sndInstArray[sound][ind] = sndArray[sound].CreateInstance();
else
{
try
{
sndInstArray[sound][ind].Stop();
}
catch
{}
}
sndInstTimes[sound][ind] = MyEngine.CurTime;
inst = sndInstArray[sound][ind];
}
return inst;
}
public virtual void playSound(int sound, float volume, float panoram, bool loop)
{
if (sound < 0 || sound > sndArray.Length)
return null;
if (!mMuted && mVolume > 0)
{
SoundEffectInstance sndinst = getValidInstance(sound);
if (sndinst == null)
return null;
try
{
sndinst.IsLooped = loop;
sndinst.Pan = panoram;
sndinst.Volume = mVolume * vol;
sndinst.Play();
}
catch
{
}
}
}

Is using IEnumerator is the best way to implement asynchronous while multithreading is seems to be slow?

I am trying to create some kind of framework that simplifies process of writing object interaction algorithm. (One object -- many clients(algorithms))
For example I want to implement algorithm that do some very simple job and waits some condition meet in a loop:
public void MakeVerySimpleJob() { }
public void AsyncLoop()
{ // real algorithm can be more complex, but job is always very simple
while (true)
{
MakeVerySimpleJob();
WakeUpCondition = "As fast as u can!";
JobComplete.Set();
WakeUp.WaitOne();
}
}
void main()
{
Thread MyThread = new Thread(AsyncLoop);
MyThread.Start();
var w = new System.Diagnostics.Stopwatch(); w.Start();
for (int i = 0; i < 100000; i++)
{
// waitin for thread
JobComplete.WaitOne();
// ok we did it
WakeUpCondition = null;
WakeUp.Set();
}
w.Stop();
}
AutoResetEvent JobComplete = new AutoResetEvent(false);
AutoResetEvent WakeUp = new AutoResetEvent(false);
Unfortunately it consumes about 500ms to do 100000 simple jobs.
Ok multithreading is not acceptable in my case, but I dont want to force users to write algorithms in this manner:
// invoke it again and again
public void PseudoAsyncLoop()
{
if (CurrentState == 1)
{
MakeVerySimpleJob();
CurrentState = 2;
return;
}
else is (CurrentState == some_sate)
{
}
}
int CurrentState = 0;
So i look at Enumerators. With Enumerators user can implement their own algorithm in traditional style:
public IEnumerable<bool> PseudoAsyncLoop()
{
while (true)
{
MakeVerySimpleJob();
WakeUpCondition = "As fast as u can!";
yield return true;
}
}
public string WakeUpCondition { get; private set; }
void main()
{
var MyLoop = PseudoAsyncLoop();
var LoopEnumerator = MyLoop.GetEnumerator();
var w = new System.Diagnostics.Stopwatch(); w.Start();
for(int i = 0; i < 100000; i ++)
{
LoopEnumerator.MoveNext();
// ok we did it
WakeUpCondition = null;
}
w.Stop();
}
Now it takes about 3ms, great. But i think its something wrong with that all...
My questions is:
Am I in right direction?
How does professional programmers solves that types of problems?
May be there is some ways to optimize multithreaded version?

I don't completely understand what are you doing or why, but if this is actually representative of your code, then you can speed it up by using one of the -Slim synchronization primitives. There is no AutoResetEventSlim, but you can use SemaphoreSlim instead:
private readonly SemaphoreSlim JobComplete = new SemaphoreSlim(0, 1);
private readonly SemaphoreSlim WakeUp = new SemaphoreSlim(0, 1);
private void AsyncLoop()
{
while (true)
{
MakeVerySimpleJob();
WakeUpCondition = "As fast as u can!";
JobComplete.Release();
WakeUp.Wait();
}
}
private void main()
{
Thread MyThread = new Thread(AsyncLoop);
MyThread.Start();
for (int i = 0; i < 100000; i++)
{
JobComplete.Wait();
WakeUpCondition = null;
WakeUp.Release();
}
}
This results in about 5.5× faster execution on my machine.

Hello, trying to create an illustration of memory-barriers, in c#

I took upon myself to present my team with a situation where a bug would be introduced by the rearrangement of instructions, however my understanding of CPUs, CLR, and JIT is quite amateurish and I did not manage to pull off a good example.
Below I show what is the best I came up with, so please look at the code snippet to understand what I am talking about from here on.
The main point is in thread2's if statement, if it ever happens - it means that the instructions were rearranged. if i manually rearrange the instructions in thread 1 or in thread 2 -> the printing will happened(even if you you swap c.x and c.y reads in thread 2, it will print due to a race condition).
My idea was to force a rearrangement of writes of x and z by making the variables which are placed farther apart integers thinking it could write them both withing one cpu cycle due to the 8 byte word size, instead of it being 3 cycles of writing 4 -> 8 -> 4 bytes. (I know it is not actually 3 cpu cycles, unfortunately, I don't know anything about assembly.) I even tried as a last resort to put it in a struct, thinking that would force some kind of an optimization from JIT.
Any help would be appreciated, because I am very eager to make it work. (I have also tried to follow the examples shown in the ebook by Joseph Albahari, but those did not work, this is why i tried to make a more sophisticated example.) I also did not forget compiling in Release for x64 instruction set.
Code:
public class Program
{
public static void Main()
{
var stopWatch = new Stopwatch();
for (var i = 0; i < 100000000; i++)
{
var delegates = new MultiTreadingDelegates(i);
Task.Run(delegates.Thread1);
Task.Run(delegates.Thread2);
}
Console.WriteLine("finished");
Console.ReadKey();
}
}
public class MultiTreadingDelegates
{
private int i = 0;
private Container container = new Container();
public MultiTreadingDelegates(int i)
{
this.i = i;
}
public void Thread1()
{
container.X = 10000000;
container.Z = 6000000000;
container.Y = 20000000;
}
public void Thread2()
{
int y = container.Y;
long z = container.Z;
int x = container.X;
if (x != 0 && z == 0 && y != 0)
{
System.Console.WriteLine($"i = {i}{Environment.NewLine}"
+ $"x = {x}{Environment.NewLine}"
+ $"z = {z}{Environment.NewLine}"
+ $"y = {y}{Environment.NewLine}"
);
}
}
}
public struct Container
{
public int X;
public long Z;
public int Y;
}

Inspired by the lecture of Sasha Goldshtein - a video i was given as a comment to my question - I have managed to pull off and example of reordering on an Intel machine(code below)! I thank everyone once again for their help.
class Program
{
static void Main(string[] args)
{
Task.Run(DelegatesUsingPetersons.Thread1);
Task.Run(DelegatesUsingPetersons.Thread2).GetAwaiter().GetResult();
}
}
static class DelegatesUsingPetersons
{
private static long x = 0;
private static long y = 0;
private static bool flag1 = false;
private static bool flag2 = false;
public static void Thread1()
{
while (true)
{
flag1 = true;
/*Thread.MemoryBarrier();*/ //Uncomment to fix locking mechanism
if (flag2 == false)
{
x++;
y++;
}
flag1 = false;
}
}
public static void Thread2()
{
long lx = 0;
long ly = 0;
while (true)
{
flag2 = true;
/*Thread.MemoryBarrier();*/ //Uncomment to fix locking mechanism
if (flag1 == false)
{
lx = x;
ly = y;
}
flag2 = false;
if (lx != ly)
{
Console.WriteLine($"lx={lx}, ly={ly} - OMG this cannot happen!");
}
}
}
}
If you want to juxtapose it with a working "more traditional" code, here is the same kind of code just without Mr.Peterson doing all the fancy algorithmic witchcraft
static class DelegatesUsingLock
{
private static long x = 0;
private static long y = 0;
private static object loq = new object();
public static void Thread1()
{
while (true)
{
if (Monitor.TryEnter(loq))
{
x++;
y++;
Monitor.Exit(loq);
}
}
}
public static void Thread2()
{
long lx = 0;
long ly = 0;
while (true)
{
if (Monitor.TryEnter(loq))
{
lx = x;
ly = y;
Monitor.Exit(loq);
}
if (lx != ly)
{
Console.WriteLine($"lx={lx}, ly={ly} - This Never Happens");
}
}
}
}