I know there are MANY similiar questions, but I can't seem to get to the bottom of this.
In my program I execute a verification method which should compare two ascii HEX files with eachother (one is local, the other is read from a USB device). Some code:
private void buttonVerify_Click(object sender, EventArgs e)
{
onlyVerifying = true;
Thread t = new Thread(verifyProgram);
}
private void verifyProgram()
{
verifying = true;
externalFlashFile.Clear();
// After this method is finished, the returned data will end up in
// this.externalFlashFile since this listen to the usb's returned data
hexFile.readExternalFlashForVerify(usbDongle, autoEvent);
externalFlashFile.RemoveAt(0);
//externalFlashFile.RemoveAt(externalFlashFile.Count - 1);
hexFile.verifyProgram(externalFlashFile);
}
public void verifyProgram(List<string> externalProgram)
{
byte[] originalFile = null; // Will be modified later with given size
byte[] externalFile = new byte[4096];
int k = 0, errors = 0;
// Remove last line which contains USB command data
externalProgram.RemoveAt(externalProgram.Count - 1);
foreach (String currentLine in externalProgram)
{
for (int i = 0; i < 64; i += 2)
{
string currentDataByte = currentLine.Substring(i, 2);
externalFile[k] = Convert.ToByte(currentDataByte, 16);
k++;
}
progress += steps;
}
//... compare externalFile and originalFile
When executing the readExternalFlashForVerify the USB is responding with requested data. This data is parsed and calls an eventhandler:
public void usbDongle_OnDataParsed(object sender, EventArgs e)
{
if (verifying)
{
usbDongle.receivedBytesString.Trim();
externalFlashFile.Add(usbDongle.receivedBytesString.Substring(2, 32 * 2));
// Allow hexFile continue its thread processing
autoEvent.Set();
}
}
The first run is always completes correctly. The following executions, at the third or fourth iteration of the foreach, I get an extra element in externalProgram. This is not a global variable (argument in function call) and the function is not called anywhere else. This ofcourse throws an exception.
I tried adding .ToList() to externalProgram in the foreach but that didn't do any difference. How can my externalProgram be modified during this execution?
EDIT: I never found the cause of this, but replacing the foreach with a hard-coded for-loop solved the issue at hand. Not an optimal solution, but don't have much time on this.
// The list should never be larger than 128 items
for (int j = 0; j < 0x7f ; j++)
{
string currentLine = externalProgram[j];
// ...
Usually when you receive an exception with a message like that it is caused by multiple accesses from different threads to a list.
What I suggest you is to use a lock when you add and remove items from that list, so you're sure the indexes to that collection are not changing. You have to think what would happen if you try to remove the last element (of index 3, for example) of a collection when someone else removes a previous item (changing the lenght of the collection to 3...).
This example: Properly locking a List<T> in MultiThreaded Scenarios? describes better what I mean.
Probably this line is a problem:
externalProgram.RemoveAt(externalProgram.Count - 1);
If verifyProgram is called multiple times, it will remove more and more lines from externalProgram list passed by reference
Related
I'm writing a C# library that needs to treat a List at high speed via multiple Timers.
I ran into very erratic error, where I try to remove an element that I know for sure is contained into the List but the program returns the following error :
System.IndexOutOfRangeException : 'index was outside the bounds of the array.'
I've made a simple example to reproduce this behaviour. Because of that issue's randomness, I've pushed hard on List operations so it throws the error right away. So this example is necessary "weird".
I've made a public repo on here : Issue Example Repo
Basically, here's what I'm dealing with:
list = new List<DummyElement>();
for (int i = 0; i < 1000; i++)
{
Timer addTimer = new Timer(0.01f);
addTimer.Start();
addTimer.Elapsed += AddItem;
Timer removeTimer = new Timer(0.01f);
removeTimer.Start();
removeTimer.Elapsed += RemoveItem;
}
void AddItem(object source, ElapsedEventArgs e)
{
list.Add(new DummyElement());
}
void RemoveItem(object source, ElapsedEventArgs e)
{
int listCount = list.Count;
if (listCount > 0) // This condition is successfully passed, so there is at least one element on the list
{
list.RemoveAt(0); // This line throw an IndexOutOfRangeException error
}
}
I believe it is a thread related issue, as if the list count was changing AFTER the condition was successfully passed.
I know nothing about thread, how can I deal with this issue?
In the For loop that goes upto 1000 - you are creating about 1000 Timers that add an item into list and 1000 timers that remove first item.
Since you haven't used any synchronization here's what happens:- Say there is 1 item in List and 2 RemoveItems are executing. both see listCount > 0 as True then one of them goes ahead and removes the Item at 0th Index while the other gets an Exception cause there is no item to remove now.
Now I cant suggest a solution to this by just looking at the code. I also need to understand the intent.
This is a Text Book Producer Consumer problem so the Text book advice here is using Lock construct:
Assume you have a class member like:
private object _lockMe = new object();
void RemoveItem(object source, ElapsedEventArgs e)
{
lock(_lockMe)
{
int listCount = list.Count;
if (listCount > 0) // This condition is successfully passed, so there is at least one element on the list
{
list.RemoveAt(0); // This line throw an IndexOutOfRangeException error
}
}
}
I have a CustomControl called PlaylistView. It displays elements in a playlist with name and thumbnail. The method DisplayPlaylist ensures that a thread is started, in which the individual elements are added one by one and the thumbnails (30th frame) are read out:
public void DisplayPlaylist(Playlist playlist)
{
Thread thread = new Thread(() => DisplayElements(playlist));
thread.Start();
}
private void DisplayElements(Playlist playlist)
{
for (int i = 0; i < playlist.elements.Count; i++)
DisplayElement(playlist.elements[i], i);
}
private void DisplayElement(IPlayable element, int index)
{
VideoSelect videoSelect = null;
if (element is Audio)
//
else if (element is Video)
videoSelect = new VideoSelect(index, element.name, GetThumbnail(element.path, SystemData.thumbnailFrame));
videoSelect.Location = GetElementsPosition(index);
panel_List.BeginInvoke(new Action(() =>
{
panel_List.Controls.Add(videoSelect);
}));
}
private Bitmap GetThumbnail(string path, int frame)
{
VideoFileReader reader = new VideoFileReader();
try
{
reader.Open(path);
for (int i = 1; i < frame; i++)
reader.ReadVideoFrame();
return reader.ReadVideoFrame();
}
catch
{
return null;
}
}
But there is a problem.
It is much too slow (about 10 elements/sec). With a playlist length of 614, you would have to wait more than a minute until all are displayed. Each time you change the playlist, such as adding or deleting an item, the procedure starts with the new item. Adding 2 or more will make it even more complicated.
I now had the approach to use multiple threads and the number of threads used for this to be specified by the user (1 to max 10). The implementation in the code currently looks like this (only changed parts compared to the previously posted code)
public void DisplayPlaylist(Playlist playlist)
{
for (int i = 0; i < SystemData.usedDisplayingThreads; i++)
{
Thread thread = new Thread(() => DisplayElements(playlist, i));
thread.Start();
}
}
private void DisplayElements(Playlist playlist, int startIndex)
{
for (int i = startIndex; i < playlist.elements.Count; i += SystemData.usedDisplayingThreads)
DisplayElement(playlist.elements[i], i);
}
The problem is that now very often null is returned by the GetThumbnail function, so an error occurs. In addition, a System.AccessViolationException is often thrown out.
In my opinion, the reason for this is the presence of multiple, simultaneously active VideoFileReaders. However, I do not know what exactly triggers the problem so I cannot present any solution. Maybe you know what the actual trigger is and how to fix the problem or maybe you also know other methods for speed improvement, which maybe even more elegant.
I would start with logging what exception is raised in GetThumbnail method. Your code hides it and returns null. Change to catch (Exception exc), write exception details in log or at least evaluate in debugger. That can give a hint.
Also I'm pretty sure your VideoFileReader instances are IDisposable, so you have to dispose them by invoking reader.Close(). Maybe previous instances were not disposed and you are trying to open same file multiple times.
Update: video frame has to be disposed as well. Probably you will need to do a copy of bitmap if it is referenced with reader and prevents disposion.
I record sound in c#(wpf) and when data from sound card is available it calls this event:
void myWaveIn_DataAvailable(object sender, WaveInEventArgs e)
{
for (int index = 0; index < e.BytesRecorded; index += 2)//Here I convert in a loop the stream into floating number samples
{
short sample = (short)((e.Buffer[index + 1] << 8) |
e.Buffer[index + 0]);
samples32Queue.Enqueue(sample/32768f);
}
//***Do some Processing with data inside Queue
}
As you can see I push every sample from the recorded buffer to a queue that is declared like that:
Queue<float> samples32Queue = new Queue<float>();
As you can see inside the event after the for loop I want to do some processing on the Queue. I worry that while processing the data, a new samples will come from the sound card and my processing will got lost.
What is the right approach to make that?
Is the processing that I call from the event is a static method/non-static?
Given the fact you can buffer the samples and process them later, consider using BlockingCollection. BlockingCollection is an excellent solution for producer-consumer pattern, which to my understanding, is your case. On one side you have myWaveIn_DataAvailable() method as a producer adding samples to the collection, and on the other end you have another consuming thread (it doesn't have to be another thread, though) collecting the samples and processing them. There are various ways the consumer can be implemented, and they are well documented in MSDN.
EDIT:
See the following example, I did not test it but it should give you a start point:
class ProducerConsumerExample
{
BlockingCollection<float> samples32Collection;
Thread consumer;
public ProducerConsumerExample()
{
samples32Collection = new BlockingCollection<float>();
consumer = new Thread(() => LaunchConsumer());
consumer.Start(); //you don't have to luanch the consumer here...
}
void Terminate() //Call this to terminate the consumer
{
consumer.Abort();
}
void myWaveIn_DataAvailable(object sender, WaveInEventArgs e)
{
for (int index = 0; index < e.BytesRecorded; index += 2)//Here I convert in a loop the stream into floating number samples
{
short sample = (short)((e.Buffer[index + 1] << 8) |
e.Buffer[index + 0]);
samples32Collection.Add(sample / 32768f);
}
}
void LaunchConsumer()
{
while (true /* insert your abort condition here*/)
{
try
{
var sample = samples32Collection.Take(); //this thread will wait here until the producer add new item(s) to the collection
Process(sample); //in the meanwhile, more samples could be added to the collection
//but they will not be processed until this thread is done with Process(sample)
}
catch (InvalidOperationException) { }
}
}
}
Ok, so I have a program that checks a twitch url for whenever someone new follows the channel by comparing a certain string is different from a "temp" string that I use for reference. But instead of only outputting a message every time the string is different it gets stuck in a loop of outputting the latest follower and then second latest follower then latest follower again etc.
What am I missing? Also, is there a better way of checking if a certain string is updated?
private void DonationListen()
{
try
{
followers = this.donationClient.DownloadString("https://api.twitch.tv/kraken/channels/" + channel.Trim() + "/follows");
donationTimer.Interval = 10000;
donationTimer.Elapsed += new ElapsedEventHandler(CheckUpdates);
donationTimer.Start();
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
}
private void CheckUpdates(object source, ElapsedEventArgs e)
{
donationTimer.Stop();
int startIndex = followers.IndexOf("display_name\":") + 15;
int endIndex = followers.IndexOf(",\"logo", startIndex);
prevFollower = followers.Substring(startIndex, (endIndex - 1) - startIndex);
if (firstRun == true)
{
temp = prevFollower;
}
else if (prevFollower != temp)
{
//New follower detected
temp = prevFollower;
if (updateFollower != null)
{
updateFollower(prevFollower);
}
}
else
{
//Follower is the same as before
}
firstRun = false;
DonationListen();
}
I'm thinking it might have something to do with the downloadstring trying to get a new string from the url but failing since it's currently being updated and therefore the CheckUpdates doesn't have correct information or something?
Without a good code example, it is difficult to know for sure what the problem is. So we are left inspecting the code you did show us.
Based on that, it appears to me as though your "loop" is being caused by repeatedly subscribing to the same event.
In your DonationListen() method, you have this statement:
donationTimer.Elapsed += new ElapsedEventHandler(CheckUpdates);
In the CheckUpdates() method (i.e. the handler you are subscribing), you have this statement (as the very last statement):
DonationListen();
In other words, every time the timer's Elapsed event is raised, you add another event handler instance to the event. For every handler you add, the CheckUpdates() method will be called.
Again, without a good code example, it is difficult to know for sure what the best fix would be. But given the code that is here, it appears to me that you could just remove that last statement from the CheckUpdates() method, as the DonationListen() method does not appear to do anything that needs doing again.
I have an WinRT application that fires notifications everytime it recieves data from a device. I also have a UI control that is databound to an observable collection which I wish to add the new data to
While I have made it capable of updating the observable collection, it causes the UI to become very laggy, as the amount of data generated it fast. It would therefore be better to batch the update, maybe every few hundred milliseconds.
Below shows a snippet of the code. First I create the periodic timer
TimerElapsedHandler f = new TimerElapsedHandler(batchUpdate);
CreatePeriodicTimer(f, new TimeSpan(0, 0, 3));
Below is my event handler for when new data comes in, along with the temporary list that stores the information
List<FinancialStuff> lst = new List<FinancialStuff>();
async void myData_ValueChanged(GattCharacteristic sender, GattValueChangedEventArgs args)
{
var data = new byte[args.CharacteristicValue.Length];
DataReader.FromBuffer(args.CharacteristicValue).ReadBytes(data);
lst.Add(new FinancialStuff() { Time = "DateTime.UtcNow.ToString("mm:ss.ffffff")", Amount = data[0] });
}
Then my batch update, which is called peroidically
private void batchUpdate(ThreadPoolTimer source)
{
AddItem<FinancialStuff>(financialStuffList, lst);
}
Then finally, for testing I want to clear the observable collection and items.
public async void AddItem<T>(ObservableCollection<T> oc, List<T> items)
{
lock (items)
{
if (Dispatcher.HasThreadAccess)
{
foreach (T item in items)
oc.Add(item);
}
else
{
Dispatcher.RunAsync(CoreDispatcherPriority.Low, () =>
{
oc.Clear();
for (int i = 0; i < items.Count; i++)
{
items.Count());
oc.Add(items[i]);
}
lst.Clear();
});
}
}
}
While this seems to work, after a few updates the UI locks up and it updates very slowly/if not at all. For testing, it's only getting a few hundred items in the list by the time the timer is fired.
Can anybody enlighten me as to why as to why this is happening - I'm presuming my design is very poor.
Thanks
You're not locking your list in the event handler
// "lst" is never locked in your event handler
List<FinancialStuff> lst = new List<FinancialStuff>();
lst.Add(new FinancialStuff() { Time = "DateTime.UtcNow.ToString("mm:ss.ffffff")", Amount = data[0] });
Passing "lst" above to your async method
AddItem<FinancialStuff>(financialStuffList, lst);
You're locking "items" below, which is really "lst" above. However, you're adding to the list while your processing it. I assume the event handler has a higher priority so your processing is slower than your add. This can lead to "i < items.Count" being true forever.
public async void AddItem<T>(ObservableCollection<T> oc, List<T> items)
{
// "lst" reference is locked here, but it wasn't locked in the event handler
lock (items)
{
if (Dispatcher.HasThreadAccess)
{
foreach (T item in items)
oc.Add(item);
}
else
{
Dispatcher.RunAsync(CoreDispatcherPriority.Low, () =>
{
oc.Clear();
// This may never exit the for loop
for (int i = 0; i < items.Count; i++)
{
items.Count());
oc.Add(items[i]);
}
lst.Clear();
});
}
}
}
EDIT:
Do you need to view every piece of data? There is going to be some overhead when using a lock. If you're getting data quicker than the speed of how fast you can render it, you'll eventually be backed up and/or have a very large collection to render, which might also cause some problems. I suggest you do some filtering to only draw the last x number of items (say 100). Also, I'm not sure why you need the if (Dispatcher.HasThreadAccess) condition either.
Try the following:
public async void AddItem<T>(ObservableCollection<T> oc, List<T> items)
{
// "lst" reference is locked here, but it wasn't locked in the event handler
lock (items)
{
// Change this to what you want
const int maxSize = 100;
// Make sure it doesn't index out of bounds
int startIndex = Math.Max(0, items.Count - maxSize);
int length = items.Count - startIndex;
List<T> itemsToRender = items.GetRange(startIndex, length);
// You can clear it here in your background thread. The references to the objects
// are now in the itemsToRender list.
lst.Clear();
// Dispatcher.RunAsync(CoreDispatcherPriority.Low, () =>
// Please verify this is the correct syntax
Dispatcher.Run(() =>
{
// At second look, this might need to be locked too
// EDIT: This probably will just add overhead now that it's not running async.
// You can probably remove this lock
lock(oc)
{
oc.Clear();
for (int i = 0; i < itemsToRender.Count; i++)
{
// I didn't notice it before, but why are you checking the count again?
// items.Count());
oc.Add(itemsToRender[i]);
}
}
});
}
}
EDIT2:
Since your AddItem method is already on a background thread, I don't think you need to run the Dispatcher.RunAsync. Instead, I think it might be desirable for it to block so you don't end up with multiple calls to that section of code. Try using Dispatcher.Run instead. I've updated the code example above to show the changes. You shouldn't need the lock on the oc anymore since the lock on the items is good enough. Also, verify the syntax for Dispatcher.Run is correct.