We Keep Coding

Return stream immediately and then write to stream asynchronously

In my current code I have a method like this to read data from a device (pseudo code):
public async Task<string> ReadAllDataFromDevice()
{
var buffer = "";
using (var device = new Device())
{
while(device.HasMoreData)
{
buffer += await device.ReadLineAsync();
}
}
return buffer;
}
I then want to send all that data via the network to some receiver. The amount of data can be really large. So clearly the above design is not very memory-efficient since it requires to read all the data before I can start sending it to the network socket.
So what I'd like to have is a function that returns a stream instead. Something like this:
public async Task<Stream> ReadAllDataFromDevice()
{
var stream = new MemoryStream();
using (var device = new Device())
using (var streamWriter = new StreamWriter(stream, new UTF8Encoding(), 512, true))
{
while(device.HasMoreData)
{
var line = await device.ReadLineAsync();
await streamWriter.WriteLineAsync(line);
}
await streamWriter.FlushAsync();
}
return stream;
}
This returns a stream but it clearly does not solve my problem, because the stream is returned only after all the data has been read from the device.
So I came up with this:
public Stream ReadAllDataFromDevice()
{
var stream = new MemoryStream();
Task.Run(async () => {
using (var device = new Device())
using (var streamWriter = new StreamWriter(stream, new UTF8Encoding(), 512, true))
{
while(device.HasMoreData)
{
var line = await device.ReadLineAsync();
await streamWriter.WriteLineAsync(line);
}
await streamWriter.FlushAsync();
}
});
return stream;
}
Is this a good design? I'm especially concerned about thread-safety, lifetime of the stream object used in the lambda, and exception handling.
Or is there a better pattern for this kind of problem?
Edit
Actually I just came up with another design that looks much cleaner to me. Instead of having the ReadAllDataFromDevice() function returning a stream, I let the consumer of the data provide the stream, like this:
public async Task ReadAllDataFromDevice(Stream stream)
{
using (var device = new Device())
using (var streamWriter = new StreamWriter(stream, new UTF8Encoding(), 512, true))
{
while(device.HasMoreData)
{
var line = await device.ReadLineAsync();
await streamWriter.WriteLineAsync(line);
}
await streamWriter.FlushAsync();
}
}

This is the design I'm using now:
public async Task ReadAllDataFromDevice(Func<Stream, Task> readCallback)
{
using (var device = new Device())
{
await device.Initialize();
using (var stream = new DeviceStream(device))
{
await readCallback(stream);
}
}
}
The line-by-line device access is encapsulated in the custom DeviceStream class (not shown here).
The consumer of the data would look something like this:
await ReadAllDataFromDevice(async stream => {
using (var streamReader(stream))
{
var data = await streamReader.ReadToEndAsync();
// do something with data
}
});

Using Async to save a FileStream

I am trying to save a file(BitmapImage) to a certain location, but as soon as I use async & await I get an error saying the file is in use:
The process cannot access the file 'C:\Users\ ... \image1.jpg' because
it is being used by another process.
My coding:
BitmapImage image = new BitmapImage(new Uri(oldImagePath));
var encoder = new JpegBitmapEncoder() { QualityLevel = 17 };
encoder.Frames.Add(BitmapFrame.Create(image));
using (var filestream = new FileStream(GetImageLocation(), FileMode.Create))
await Task.Run(() => encoder.Save(filestream)); //Error here
When I use the code without the await, the code works perfectly fine. I think it might be because another thread might be using it, but can anyone help or explain to me a work around for my issue? Thank you.

In your case when you use Task with await another thread is used to save your encoder. But your encoder is also used by your main thread so new thread can't use it.
Change your code a little:
await Task.Run(() =>
{
using (var filestream = new FileStream(GetImageLocation(), FileMode.Create))
{
BitmapImage image = new BitmapImage(new Uri(oldImagePath));
var encoder = new JpegBitmapEncoder() { QualityLevel = 17 };
encoder.Frames.Add(BitmapFrame.Create(image));
encoder.Save(filestream);
}
}
Now you create and save your encoder in the same task and it will use only one thread.

I think you should move the code inside the Task.Run, because it gets called inside another thread.

You could encode to a MemoryStream, get the byte array, use WriteAsync on the FileStream, and avoid using Task.Run altogether.
BitmapImage image = new BitmapImage(new Uri(oldImagePath));
var encoder = new JpegBitmapEncoder() { QualityLevel = 17 };
encoder.Frames.Add(BitmapFrame.Create(image));
using (var mem = new MemoryStream())
using (var filestream = new FileStream(GetImageLocation(), FileMode.Create))
{
encoder.Save(mem);
var data = mem.ToArray();
await filestream.WriteAsync(date, 0, data.Length);
}
Note that this will block your thread during the encoding and will use more memory.

Why does this async method lock up the user interface?

I am using the following code to read all the images on a network drive and poplate an ImageControl with each, and then display them on the screen.
The problem I'm having is that regardless of making PopulateImages() an async method, and and running Task.WaitAll the user interface is still locked up until all the images render.
Am I doing the async/await incorrectly? What do I need to do to resolve this?
public MainWindow()
{
InitializeComponent();
Loaded += (s, e) => PopulateImages();
}
private async void PopulateImages()
{
string StartDirectory = #"//path/to/network/folder";
Task.WaitAll(Directory
.EnumerateFiles(StartDirectory)
.Select(filename => Task.Run(async () =>
{
Bitmap resizedImage;
using (var sourceStream = File.Open(filename, FileMode.Open))
{
using (var destinationStream = new MemoryStream())
{
await sourceStream.CopyToAsync(destinationStream);
resizedImage = ResizeImage(new Bitmap(destinationStream), 96, 96);
}
}
Dispatcher.BeginInvoke(new Action(() =>
{
var imgControl = new ImageControl(filename, resizedImage);
stackpanelContainer.Children.Add(imgControl);
}));
})).ToArray());
}

You're using Task.WaitAll - that blocks until all the tasks have completed.
Instead, you should use Task.WhenAll, which returns a Task which will itself complete when all the other tasks have completed. You can then await that.
await Task.WhenAll(...);
Although to be honest, unless you need to do anything when the tasks have all completed, you don't need to wait for them at all.

Instead of registering to the Loaded event, consider overriding FrameworkElement.OnInitialized method. That way, you can await on PopulateImage, save the Task.WaitAll and possibly remove the need to use Task.Run, if your ResizeImage isn't too CPU heavy:
public override async void OnInitialized()
{
await PopulateImages();
base.OnInitialized();
}
private async Task PopulateImages()
{
string StartDirectory = #"//path/to/network/folder";
Directory.EnumerateFiles(StartDirectory)
.Select(filename => async () =>
{
Bitmap resizedImage;
using (var sourceStream = File.Open(filename, FileMode.Open))
using (var destinationStream = new MemoryStream())
{
await sourceStream.CopyToAsync(destinationStream);
resizedImage = ResizeImage(new Bitmap(destinationStream), 96, 96);
}
}
var imgControl = new ImageControl(filename, resizedImage);
stackpanelContainer.Children.Add(imgControl);
}

Using Task.Factory in Image class

I am trying to make images load in different thread but image is never updated.
public class MyImage : System.Windows.Controls.Image
{
public MyImage()
{
this.Loaded += new RoutedEventHandler(MyImage_Loaded);
}
void MyImage_Loaded(object sender, RoutedEventArgs e)
{
//mypath=#"c:\test.jpg";
var t = Task<ImageSource>.Factory.StartNew(() => GetImage(mypath));
Source = t.Result;
}
Following works but it is the UI thread:
Source = GetImage(mypath);
I tried the same with BackgroundWorker but the result is the same.
Is it possible to do it like this without MVVM?

Because you create ImageSource on different thread then the one you want to use it on you get
The calling thread cannot access this object because a different thread owns it
exception and to solve your issue you should call Freeze() on your ImageSource. However even though you load image on different thread your code blocks UI thread until it's ready:
var t = Task<ImageSource>.Factory.StartNew(() => GetImage(mypath));
Source = t.Result; //this line blocks UI thread until Result is ready
If you want to avoid that change it to:
Task.Factory.StartNew(() =>
{
var source = GetImage(mypath);
source.Freeze();
this.Dispatcher.Invoke(() => this.Source = source);
});
This should load image on different thread and update Source when it's ready without blocking UI thread

Try returning a byte[] or a stream from your GetImage (or from its result), then use Dispatcher.Invoke to call a helper on the UI thread that reconstructs the image from the byte[] or stream and updates MyImage's Source.
This is my working POC. You want to examine it for proper disposal of the stream and so on.
(Note: disp is the Dispatcher to "post" to the UI)
Task.Factory.StartNew(() =>
{
var result = GetImage();
var bitmap = new BitmapImage();
var stream = result;
bitmap.BeginInit();
bitmap.CacheOption = BitmapCacheOption.OnLoad;
bitmap.StreamSource = stream;
bitmap.EndInit();
bitmap.Freeze();
stream.Dispose();
disp.Invoke(new Action(()=> MyImage.Source = bitmap));
});
And the GetImage():
private MemoryStream GetImage()
{
System.Threading.Thread.Sleep(35000);
var filePath = #"C:\temp\2013-01-08-235341.jpg";
MemoryStream memoryStream = null;
if (File.Exists(filePath))
{
memoryStream = new MemoryStream();
byte[] fileBytes = File.ReadAllBytes(filePath);
memoryStream.Write(fileBytes, 0, fileBytes.Length);
memoryStream.Position = 0;
}
return memoryStream;
}

Using statement in Task

In an ASP.NET Web API controller I want to return an image. For streaming the image I need a MemoryStream. Normally I would wrap it in a using statement in order to make sure it gets properly disposed afterwards. However, as this executes asynchronously in a Task this doesn't work:
public class ImagesController : ApiController
{
private HttpContent GetPngBitmap(Stream stream)
{
var pngBitmapEncoder = new PngBitmapEncoder();
pngBitmapEncoder.Save(stream);
stream.Seek(0, SeekOrigin.Begin);
return new StreamContent(stream);
}
// GET api/images
public Task<HttpResponseMessage> Get(string id, string path)
{
//do stuff
return Task.Factory.StartNew(() =>
{
var stream = new MemoryStream(); //as it's asynchronous we can't use a using statement here!
{
var response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = GetPngBitmap(stream)
};
response.Content.Headers.ContentType =
new System.Net.Http.Headers.MediaTypeHeaderValue("image/png");
return response;
}
//how can I dispose stream???
});
}
}

MemoryStream is one of the classes that implement IDisposable, because their base class does. MemoryStream doesn't hold any resources (apart from managed memory), so disposing it is actually unnecessary.
HttpResponseMessage is disposable. This means that when the whole response it sent, that object is disposed. Doing that disposes the contained HttpContent, which in the case of StreamContent disposes the underlying Stream. So even if you had a Stream that should be disposed, you don't have to worry about it in this case.

use "using" then it will disposed automatically.
public class ImagesController : ApiController
{
// GET api/images
public Task<HttpResponseMessage> Get(string id, string path)
{
//do stuff
return Task.Factory.StartNew(() =>
{
using( stream = new MemoryStream()) //as it's asynchronous we can't use a using statement here!
{
{
var response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = GetPngBitmap(stream)
};
response.Content.Headers.ContentType =
new System.Net.Http.Headers.MediaTypeHeaderValue("image/png");
return response;
}
//how can I dispose stream???
}
});
}
}

Maybe return a Tuple<HttpResponseMessage, Stream> and attach a continuation task to dispose of the Stream?
public Task<Tuple<HttpResponseMessage,Stream>> Get(string id, string path)
{
//do stuff
return Task.Factory.StartNew(() =>
{
var stream = new MemoryStream();
{
var response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = GetPngBitmap(stream)
};
response.Content.Headers.ContentType =
new System.Net.Http.Headers.MediaTypeHeaderValue("image/png");
return Tuple.Create(response, stream);
}
})
.ContinueWith(t => t.Result.Item2.Close());
}

Maybe this sounds naif, but can't you just dispose it at the end of the task?
var stream = new MemoryStream();
//use the stream...
stream.Dispose();
edit: or "Close()", it's the same on MemoryStream.

You can do it like this. This will execute your initialRun first and after the async call, return in and it will execute the continue with.
Task<System.IO.MemoryStream> initialRun = new Task<System.IO.MemoryStream>(() =>
{
System.IO.MemoryStream stream = new System.IO.MemoryStream();
// use stream
return stream;
});
initialRun.ContinueWith(new Action<Task<System.IO.MemoryStream>>((stream) =>
{
stream.Dispose();
}),
TaskScheduler.FromCurrentSynchronizationContext());
initialRun.Start();