I had some performance issues when using an UICollectionView where the cell was showing an Image (loaded from the HD).
I solved that issue by loading the image in the background.
Basically, in my "GetCell" method I check if the image is in my ImageCache.
If so, set the image on my ImageView in the cell.
If not, load the image in the background and request a reload for that specific
item (I request a reload because I don't know if the cell is recycled
in the meanwhile, so it's not safe to directly set the image)
Snippet of my background process:
ThreadPool.QueueUserWorkItem (delegate {
ImagesCache.AddImageToCache(""+indexPath.Row,ImageForPosition(indexPath.Row));
InvokeOnMainThread (delegate {
collectionView.ReloadItems(Converter.ToIndexPathArray(indexPath));
});
});
It works ok, but if you scroll fast, it will load all these async tasks and the problem is that it will execute the requests in order (FIFO). So when you scroll fast, images for invisible cells will be loaded before images from visible cells.
Does anyone has an idea how I could optimise this process to get a better user experience?
Because if I would extend this to include images from the internet, the problem will be worse (because of the download).
Increasing the max amount of simultaneously threads will allow the later added threads to already start immediately but it will decrease the overall performance/download speed, so that's also not a real solution.
Thanks,
Matt
My project's solution in short: one thread to download images with queue support. Plus checking in target UI control, that it wasn't dequeued for reuse.
Long version:
Implement queue with methods Start/Stop. When Start is calling, start background thread, which in busy loop (while true { DoSomething(); }) will try to dequeue request from queue. If not dequeued, sleep a bit. If dequeued, execute it (download image). Stop method should say thread to exit from loop:
public void Start()
{
if (started) {
return;
}
started = true;
new Thread (new ThreadStart (() => {
while (started) {
var request = GetRequest();
if (request != null) {
request.State = RequestState.Executing;
Download (request);
} else {
Thread.Sleep (QueueSleepTime);
}
}
})).Start ();
}
public void Stop()
{
started = false;
}
Then, make a private method in queue to download image with such logic: check for image in file cache. If file is available, read and return it. If not, download it, save it to file, return it (call Action<UIImage> onDownload) or error (call Action<Exception> onError). Call this method in queue's busy loop. Name it Download:
public Download(Request request)
{
try {
var image = GetImageFromCache(request.Url);
if (image == null) {
image = DownloadImageFromServer(request.Url); // Could be synchronous
}
request.OnDownload(image);
} catch (Exception e) {
request.OnError(e);
}
}
Then, make a public method to add request to queue. Pattern Command is useful for wrapping requests for queue: storing Actions, current State. Name it DownloadImageAsync:
public DownloadImageAsync(string imageUrl, Action<UIImage> onDownload, Action<Exception> onError)
{
var request = MakeRequestCommand(imageUrl, onDownload, onError);
queue.Enqueue(request);
}
When UITableViewCell is preparing to show and request to download image:
// Custom UITableViewCell method, which is called in `UITableViewSource`'s `GetCell`
public void PrepareToShow()
{
var imageURLClosure = imageURL;
queue.DownloadImageAsync(imageURL, (UIImage image) => {
if (imageURLClosure == imageURL) {
// Cell was not dequeued. URL from request and from cell are equals.
imageView.Image = image;
} else {
// Do nothing. Cell was dequeued, imageURL was changed.
}
}, (Exception e) => {
// Set default image. Log.
});
}
Checking for (imageURLClosure == imageURL) is imprtant to avoid showing multiple images in one UIImageView when scrolls fast. One cell could initialize multiple request, but only last one result should be used.
Futher improvements:
LIFO execution. If no requests are already run, add new one to begin.
Using Action<byte[]> onDownload instead of Action<UIImage> onDownload for cross-platform code compatibility;
Availability to cancel download image request when cell become insivisble (WillMoveToSuperview). Well, it's not very necessary. After first downloading, image will be in cache, so any further requests for image will be done fast. Thanks to cache;
In-memory cache. Thus, in worst case, chain will be:
Find image in in-memory cache -> Find image in file cache -> Downloading from server.
Related
I am writing an application that depends on fast image manipulation. It might sound strange but I'm doing this C# and not in C++. So far this has not been a limitation, I can process an image realtime. While I do quite some complex things with the image and I do this under 30ms.
I changed the program to make sure that the image stream would never queue
by simply checking a boolean to check if a current frame is not being processed. Normally this wouldn't happen, but in some cases it did. For example when run the app in VS2010 debug mode, or when the PC is doing also other heavy tasks, and has less CPU resources.
In such case I would like to skip new frame processing, so processing them won't queue. In such cases it would be better to just work with last known data which is still being processed, and thus waiting would be the fastest method then to retrieve an answer.
So I started with something like:
private void Camera_FrameReady(object Sender, ImageEvent e)
{
if (!IsImageReady) return; // global var
IsImageReady = false;
//... do stuff with the image
IsImageReady=true;
}
This didn't workout, as I had hoped. And I think it has to do with the threading nature of events within the C# compiler. So then I tried to resolve it by de-registering and re-registering the Camera_FrameReady ready, but the camera takes to much time to restart, so that didn't workout.
Strangely now it seams to work with the code below but I'm not sure why it does.
private void Camera_FrameReady(object Sender, ImageEvent e)
{
Update_Image(e)
}
private void Update_Image(e)
{
if (!IsImageReady) return; // global var
IsImageReady = false;
//... do stuff with the image
IsImageReady=true;
}
This makes me wonder about how C# gets compiled. Does it work like that whenever Camera_FrameReady is called it has a "world view" of the current global values? Or that global variables are only updated after the processing of an event?
The first thing came in my head is that the Camera_FrameReady event blocks the acquisition thread. But that doesn't explain why the second solution works..
So if you want to process the images parallel to the acquisition thread, you should create a new thread for processing.
For example: When there is a new image, check if the processing thread is busy. If the processing thread is busy, you shouldn't wait or queue (like you wanted) but just skip this image. If the processing thread is waiting for work, store the image in a 'global' variable, so the processing thread can access it and signal the processing thread.
I made an example for you: (pseudo)
// the thread for the processing.
private Thread _processingThread;
// a signal to check if the workerthread is busy with an image
private ManualResetEvent _workerThreadIsBusy = new ManualResetEvent(false);
// request for terminating
private ManualResetEvent _terminating = new ManualResetEvent(false);
// confirm terminated
private ManualResetEvent _terminated = new ManualResetEvent(false);
// store the current image.
private Image _myImage;
// event callback for new images
private void Camera_FrameReady(object Sender, ImageEvent e)
{
// is the workerthread already processing an image? return.. (skip this image)
if (_workerThreadIsBusy.WaitOne(0))
return; // skip frame.
//create a 'global' ref so the workerthread can access it.
/* BE CAREFULL HERE. You might experience trouble with the instance of this image.
* You are creating another reference to the SAME instance of the image
* to process on another thread. When the Camera is reusing this
* image (for speed), the image might screwed-up. In that case,
* you have to create a copy!
* (personally I would reuse the image which makes the image not available outside the event callback) */
_myImage = e.Image;
// signal the workerthread, so it starts processing the current image.
_workerThreadIsBusy.Set();
}
private void ImageProcessingThread()
{
var waitHandles = new WaitHandle[] { _terminating, _workerThreadIsBusy };
var run = true;
while (run)
{
switch (EventWaitHandle.WaitAny(waitHandles))
{
case 0:
// terminating.
run = false;
break;
case 1:
// process _myImage
ProcessImage(_myImage);
_workerThreadIsBusy.Reset();
break;
}
}
_terminated.Set();
}
private void ProcessImage(Image _myImage)
{
// whatever...
}
// constructor
public MyCameraProcessor()
{
// define the thread.
_processingThread = new Thread(ImageProcessingThread);
_processingThread.Start();
}
public void Dispose()
{
_terminating.Set();
_terminated.WaitOne();
}
}
Your code is not multithreading safe
if (!IsImageReady) return; // global var
IsImageReady = false;
//... do stuff with the image
IsImageReady=true;
2 threads can read IsImageReady at the same time, see that it is true and both set it then to false. You might also get problems if the processor is reading IsImageReady from cache and not from memory. You can avoid these kind of problems with the Interlocked class, which reads and changes the value in one operation. It also ensures that the cache doesn't cause problems.
private int IsImageReady= 0;
private void Camera_FrameReady(object Sender, ImageEvent e){
int wasImageReady = Interlocked.Exchange(ref IsImageReady, 1);
if (wasImageReady ==1) return;
//do something
IsImageReady= 0;
}
}
Although I am not sure if that is your only problem. You might have also others. To be sure, you have to debug your code properly, which is very difficult when it involves multithreading. Read my article Codeproject: Debugging multithreaded code in real time how you can do it.
i have a thread pool and all threads have a fixed set of images. Let's say they have each 100 images. I need to detect which image contains another image that reside in an image bank.
thread 1 - 100 images
thread 2 - 100 images
thread 3 - 100 images
thread 4 - 100 images
Image Base - 50 images
Now i need all the threads to look inside the Image base to see if one of the images they hold resembles one of the image base. I have my image matching done, what i am worried about is if multiple threads might open the same image file. What would be the proper way of tackling this ? It would be nice not to "lock" all the other threads for each IO.
Thanks !
What about something like this, where each thread has a reference to the image bank and provides a delegate that will be called for each of the files in the image bank? Here is a skeleton of what the image bank might look like.
public class ImageBank {
public delegate bool ImageProcessorDelegate(String imageName);
private readonly List<String> _imageBank;
public ImageBank()
{
// initialize _imageBank with list of image file names
}
private int ImageBankCount {
get {
lock(_imageBank) {
return _imageBank.Count;
}
}
}
private List<String> FilterImageBank(ISet<String> exclude)
{
lock(_imageBank)
{
return _imageBank.Where(name => !exclude.Contains(name)).ToList();
}
}
public void ProcessImages(ImageProcessorDelegate imageProcessor)
{
var continueProcessing = true;
var processedImages = new HashSet<String>();
var remainingImages = new List<String>();
do
{
remainingImages = FilterImageBank(processedImages);
while(continueProcessing && remainingImages.Any())
{
var currentImageName = remainingImages[0];
remainingImages.RemoveAt(0);
// protect this image being accessed by multiple threads.
var mutex = new Mutex(false, currentImageName);
if (mutex.WaitOne(0))
{
try
{
// break out of loop of the processor found what it was looking for.
continueProcessing = imageProcessor(currentImageName);
}
catch (Exception)
{
// exception thrown by the imageProcessor... do something about it.
}
finally
{
// add the current name to the set of images we've alread seen and reset the mutex we acquired
processedImages.Add(currentImageName);
mutex.ReleaseMutex();
}
}
}
}
while(continueProcessing);
}
}
Then, each thread would have its list of images (_myImageList) and a ThreadProc that looks something like this:
void ThreadProc(object bank)
{
var imageBank = bank as ImageBank;
foreach(var myImage in _myImageList)
{
imageBank.ProcessImages(imageName =>
{
// do something with imageName and myImage
// return true to continue with the next image from the image bank
// or false to stop processing more images from the image bank
}
);
}
}
Assuming all threads have the same set of images they have to work on, and assuming the pahts to these files are in a list or some other collection, you could try something like this:
// A shared collection
List<string> paths = new List<string>();
// Fill this collection with your fixed set.
IEnumerator<T> e = paths.GetEnumerator();
// Now create all threads and use e as the parameter. Now all threads have the same enumerator.
// Inside each thread you can do this:
while(true)
{
string path;
lock(e)
{
if (!e.MoveNext())
return; // Exit the thread.
path = e.Current;
}
// From here, open the file, read the image, process it, etc.
}
In this example you only put a lock on the enumerator. Only one thread can read from it at the same time. So each time it is called, a different path will come out of it.
Outside the lock you can do all the processing, I/O, etc.
Of course the collection could also be of another type, like an array.
I'm downloading two JSON files from the webs, after which I want to allow loading two pages, but not before. However, the ManualResetEvent that is required to be set in order to load the page never "fires". Even though I know that it gets set, WaitOne never returns.
Method that launches the downloads:
private void Application_Launching(object sender, LaunchingEventArgs e)
{
PhoneApplicationService.Current.State["doneList"] = new List<int>();
PhoneApplicationService.Current.State["manualResetEvent"] = new ManualResetEvent(false);
Helpers.DownloadAndStoreJsonObject<ArticleList>("http://arkad.tlth.se/api/get_posts/", "articleList");
Helpers.DownloadAndStoreJsonObject<CompanyList>("http://arkad.tlth.se/api/get_posts/?postType=webbkatalog", "catalog");
}
The downloading method, that sets the ManualResetEvent
public static void DownloadAndStoreJsonObject<T>(string url, string objName)
{
var webClient = new WebClient();
webClient.DownloadStringCompleted += (sender, e) =>
{
if (!string.IsNullOrEmpty(e.Result))
{
var obj = ProcessJson<T>(e.Result);
PhoneApplicationService.Current.State[objName] = obj;
var doneList = PhoneApplicationService.Current.State["doneList"] as List<int>;
doneList.Add(0);
if (doneList.Count == 2) // Two items loaded
{
(PhoneApplicationService.Current.State["manualResetEvent"] as ManualResetEvent).Set(); // Signal that it's done
}
}
};
webClient.DownloadStringAsync(new Uri(url));
}
The waiting method (constructor in this case)
public SenastePage()
{
InitializeComponent();
if ((PhoneApplicationService.Current.State["doneList"] as List<int>).Count < 2)
{
(PhoneApplicationService.Current.State["manualResetEvent"] as ManualResetEvent).WaitOne();
}
SenasteArticleList.ItemsSource = (PhoneApplicationService.Current.State["articleList"] as ArticleList).posts;
}
If I wait before trying to access that constructor, it easily passes the if-statement and doesn't get caught in the WaitOne, but if I call it immediately, I get stuck, and it never returns...
Any ideas?
Blocking the UI thread must be prevented at all costs. Especially when downloading data: don't forget that your application is executing on a phone, which has a very instable network. If the data takes two minutes to load, then the UI will be freezed for two minutes. It would be an awful user experience.
There's many ways to prevent that. For instance, you can keep the same logic but waiting in a background thread instead of the UI thread:
public SenastePage()
{
// Write the XAML of your page to display the loading animation per default
InitializeComponent();
Task.Factory.StartNew(LoadData);
}
private void LoadData()
{
((ManualResetEvent)PhoneApplicationService.Current.State["manualResetEvent"]).WaitOne();
Dispatcher.BeginInvoke(() =>
{
SenasteArticleList.ItemsSource = ((ArticleList)PhoneApplicationService.Current.State["articleList"]).posts;
// Hide the loading animation
}
}
That's just a quick and dirty way to reach the result you want. You could also rewrite your code using tasks, and using Task.WhenAll to trigger an action when they're all finished.
Perhaps there is a logic problem. In the SenastePage() constructor you are waiting for the set event only if the doneList count is less than two. However, you don't fire the set event until the doneList count is equal to two. You are listening for the set event before it can ever fire.
I'm developing a plugin for a 3D modelling application. For this application, there is also a third party plugin (a render engine) that I would like to automate.
What I do is create a list of Camera List<Camera> cameraViews , iterate trough all of them and tell the render engine to start rendering
foreach ( Camera camera in cameraViews )
{
// tell the modellingApplication to apply camera
modellingApplication.ApplyCameraToView(camera);
// tell the render engine to render the image
string path = "somePathWhereIWantToSaveTheImage"
renderEngine.renderCurrentScene(path)
// .renderCurrentScene() seems to be async, because my code, which is on the UI thread
// continues... so:
// make sure that the image is saved before continuing to the next image
while ( !File.Exists(path) )
{
Thread.Sleep(500);
}
}
However, this wont work. The renderingplugin seems to do some async work but, when doing this async work, it is calling the main thread for retrieving information.
I found a workaround for this: Right after calling the render engine to render, call a MessageBox. This will block the code from continuing but async calls are still beïng handled. I know, this is weird behaviour. Whats even weirder is the fact that my MessageBox gets automatically closed when the renderengine has done calling the UI thread for information and continues in his own process. Making my code continue to the while loop to check if the image is saved on the disk.
foreach ( Camera camera in cameraViews )
{
// tell the modellingApplication to apply camera
modellingApplication.ApplyCameraToView(camera);
// tell the render engine to render the image
string path = "somePathWhereIWantToSaveTheImage"
renderEngine.renderCurrentScene(path)
// .renderCurrentScene() seems to be async, because my code, which is on the UI thread
// continues... so:
// show the messagebox, as this will block the code but not the renderengine.. (?)
MessageBox.Show("Currently processed: " + path);
// hmm, messagebox gets automatically closed, that's great, but weird...
// make sure that the image is saved before continuing to the next image
while ( !File.Exists(path) )
{
Thread.Sleep(500);
}
}
This is wonderful, except for the messagebox part. I don't want to show a messagebox, I just want to pause my code without blocking the entire thread (as calls from the renderengine to the ui thread are still accepted)..
It would've been much easier if the renderengine didn't do his work async..
I don't feel this is the best answer, but it hopefully it's what you are looking for. This is how you block a thread from continuing.
// Your UI thread should already have a Dispatcher object. If you do this elsewhere, then you will need your class to inherit DispatcherObject.
private DispatcherFrame ThisFrame;
public void Main()
{
// Pausing the Thread
Pause();
}
public void Pause()
{
ThisFrame = new DispatcherFrame(true);
Dispatcher.PushFrame(ThisFrame);
}
public void UnPause()
{
if (ThisFrame != null && ThisFrame.Continue)
{
ThisFrame.Continue = false;
ThisFrame = null;
}
}
If you want to still receive and do actions on that thread while blocking intermediately, you can do something like this. This feels, um... kinda hacky, so don't just copy and paste without making sure I didn't make some major mistake typing this out. I haven't had my coffee yet.
// Used while a work item is processing. If you have something that you want to wait on this process. Or you could use event handlers or something.
private DispatcherFrame CompleteFrame;
// Controls blocking of the thread.
private DispatcherFrame TaskFrame;
// Set to true to stop the task manager.
private bool Close;
// A collection of tasks you want to queue up on this specific thread.
private List<jTask> TaskCollection;
public void QueueTask(jTask newTask)
{
//Task Queued.
lock (TaskCollection) { TaskCollection.Add(newTask); }
if (TaskFrame != null) { TaskFrame.Continue = false; }
}
// Call this method when you want to start the task manager and let it wait for a task.
private void FireTaskManager()
{
do
{
if (TaskCollection != null)
{
if (TaskCollection.Count > 0 && TaskCollection[0] != null)
{
ProcessWorkItem(TaskCollection[0]);
lock (TaskCollection) { TaskCollection.RemoveAt(0); }
}
else { WaitForTask(); }
}
}
while (!Close);
}
// Call if you are waiting for something to complete.
private void WaitForTask()
{
if (CompleteFrame != null) { CompleteFrame.Continue = false; }
// Waiting For Task.
TaskFrame = new DispatcherFrame(true);
Dispatcher.PushFrame(TaskFrame);
TaskFrame = null;
}
/// <summary>
/// Pumping block will release when all queued tasks are complete.
/// </summary>
private void WaitForComplete()
{
if (TaskCollection.Count > 0)
{
CompleteFrame = new DispatcherFrame(true);
Dispatcher.PushFrame(CompleteFrame);
CompleteFrame = null;
}
}
private void ProcessWorkItem(jTask taskItem)
{
if (taskItem != null) { object obj = taskItem.Go(); }
}
I'd like to download a picture and afterwards show it in a picturebox.
at first I did like this:
WebClient client = new WebClient();
client.DownloadFile(url, localFile);
pictureBox2.Picture = localFile;
But that wasn't perfect because for the time while the download is performed the app is kinda freezing.
Then I changed to this:
public class ParamForDownload
{
public string Url { get; set; }
public string LocalFile { get; set; }
}
ParamForDownload param = new ParamForDownload()
{
Url = url,
LocalFile = localFile
};
ThreadStart starter = delegate { DownloadMap (param); };
new Thread(starter).Start();
pictureBox2.Picture = localFile;
private static void DownloadMap(ParamForDownload p)
{
WebClient client = new WebClient();
client.DownloadFile(p.Url, p.LocalFile);
}
But now I have to do something like a "wait for thread ending" because the file is accessed in the thread and to same time there's downloaded something to the file by the DownloadMap method.
What would be the best wait to solve that problem?
Basically, What was happening originally, was the UI Thread was completing the download, and because it was working away on that, it couldn't be refreshed or painted (working synchronously). Now what is happening is that you're starting the thread then the UI thread is continuing, then trying to assign the local file (which hasn't finished downloading) to the picture box. You should try either of the following:
You should use a background worker to complete your download task.
It has events that will be very handy. DoWork, where you can start the download.
There is also a RunWorkerCompleted event that is fired when the Work has completed. Where you can set the image there (pictureBox2.Picture = localFile;).
It's definitely worth checking out, I think it's the most appropriate way to complete what you are trying to achieve.
Or
If you want to stick with using a Thread. You could take out the Image assignment after you've done the Thread.Start(), and put this in to your Worker Thread function:
private delegate void MyFunctionCaller();
private static void DownloadMap(ParamForDownload p)
{
WebClient client = new WebClient();
client.DownloadFile(p.Url, p.LocalFile);
DownloadMapComplete(p);
}
private void DownloadMapComplete(ParamForDownload p)
{
if (InvokeRequired == true)
{
MyFunctionCaller InvokeCall = delegate { DownloadMapComplete(p); };
Invoke(InvokeCall);
}
else
{
pictureBox2.Picture = p.LocalFile;
}
}
The simplest solution would be to just use the Picturebox.LoadAsync() method and let the Picturebox worry about loading it in the background. If you need to check for errors, use the LoadCompleted event (of the picturebox).
A line like:
pictureBox1.LoadAsync(#"http://imgs.xkcd.com/comics/tech_support_cheat_sheet.png");
is all you need.
When the user initiates the process what you need to do is:
1) Update the UI to indicate something is happening. IE: Disable all the fields and put up a message saying "I am downloading the file, please wait...". Preferentially with some kind of progress indicator (sorry I am not sure if the WebClient supports progress etc but you need to update the UI as the download make take a while).
2) Attach an event handler to the WebClient's 'DownloadFileCompleted'.
3) Use the WebClient's DownloadFileAsync method to start the download on another thread. You don't need to spin threads up yourself.
4) When the 'DownloadFileCompleted' event is fired route the call back to the UI thread by using the form's invoke method. THIS IS VERY IMPORTANT.
See: http://weblogs.asp.net/justin_rogers/pages/126345.aspx
and: http://msdn.microsoft.com/en-us/library/zyzhdc6b.aspx
5) Once the event has been routed back onto the UI thread open the file and update the UI as required (IE: Re-enable fields etc).
Leather.
Not related to the threading issue, but if you don't have any other requirements for saving the photo to the disk you can do this:
WebClient client = new WebClient();
byte[] data = client.DownloadData(item.Url);
MemoryStream ms = new MemoryStream(data);
Bitmap img = new Bitmap(ms);
pictureBox.Image = img;