We Keep Coding

Streaming lines of text using IAsyncEnumerable

I want to stream an arbitrary amount of lines of plain text from an ASP.NET server to a Blazor WebAssembly client (.NET 6.0).
For testing I implemented the following dummy API:
[HttpGet("lines")]
public async IAsyncEnumerable<string> GetLines() {
for (var i = 0; i < 10; ++i) {
yield return "test\n";
await Task.Delay(1000);
}
}
On the client I tried the following approach (following these ideas):
public async IAsyncEnumerable<string?> GetLines() {
var response = await HttpClient.GetAsync($"{apiRoot}/lines", HttpCompletionOption.ResponseHeadersRead);
if (response.IsSuccessStatusCode) {
var responseStream = await response.Content.ReadAsStreamAsync();
var lines = JsonSerializer.DeserializeAsyncEnumerable<string>(responseStream);
await foreach (var line in lines) {
yield return line;
}
}
else {
Log.Error($"Server response code: {response.StatusCode}");
yield return null;
}
}
Unfortunately, instead of returning immediately, response.Content.ReadAsStreamAsync() buffers the entire stream (i.e. 10 lines of "test\n"), taking 10 s in this case, before the buffered content gets deserialized as an IAsyncEnumerable<string>.
The same behavior can be observed using HttpClient.GetStreamAsync:
public async IAsyncEnumerable<string?> GetLines() {
var responseStream = await HttpClient.GetStreamAsync($"{apiRoot}/lines"); // buffers for 10 s
var linesAsync = JsonSerializer.DeserializeAsyncEnumerable<string>(responseStream);
await foreach (var line in lines) {
yield return line;
}
}
How can I change this so that every line sent from the server is processed immediately on the client, without any buffering?
Is this an issue on the client or the server side? Is there a way to disable this buffering behavior?
Edit: After some more experimentation, I found that calling the API directly (e.g. via the browser) does indeed show the expected streaming behavior, i.e. the individual lines pop up one after the other with a 1.0 s delay. So it seems to be a client-side issue, indeed.

I found a workaround that works for me, because I don't need any JSON deserialization as I want to stream raw strings.
The following implementation solves the client-side streaming issues:
public async IAsyncEnumerable<string?> GetLines() {
using var request = new HttpRequestMessage(HttpMethod.Get, $"{apiRoot}/lines");
request.SetBrowserResponseStreamingEnabled(true);
var response = await HttpClient.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
if (response.IsSuccessStatusCode) {
using var responseStream = await response.Content.ReadAsStreamAsync();
using var reader = new StreamReader(responseStream);
string? line = null;
while ((line = await reader.ReadLineAsync()) != null) {
yield return line;
}
}
else {
Log.Error($"Server response code: {response.StatusCode}");
yield return null;
}
}

httpclient GetAsync doesn't work for large content

I am trying to download the string content from an endpoint in question using xamarin and http requests. The endpoint I'm supporting with python flask, as well as the content of the page, which is similar to this:
[{"name":"test1","timestamp":12312,"type":"ultrasonic","value":65535},
{"name":"test2","timestamp":3123123,"type":"ultrasonic","value":65535}...]
When there are not many lines in the content of the page, I can access the content and save it to a file inside the android device. When the content is very long, with many lines, the application simply closes.
My code is as follows:
public async Task<string> GetNews()
{
List<string> valores = new List<string>();
var response = await client.GetAsync(get_url);
var responseData = await response.Content.ReadAsStringAsync();
string nome = DateTime.Now.ToString("HH:mm:ss");
//creating the subsequent file
string filename = System.IO.Path.Combine(
Environment.GetFolderPath(Environment.SpecialFolder.Personal),
"Sensor File - " + nome + ".json");
//inserting the content of the GET into the generated text file
System.IO.File.WriteAllText(filename, responseData);
return null;
}
public async void Download_Clicked(object sender, EventArgs e)
{
try
{
await GetNews();
//informing in the first label the directory of the saved file
lb.Text = "File saved successfully!";
}
catch
{
lb.Text = "Connection not possible, enter the correct URL.";
}
}
I did add .ConfigureAwait(false) to each await line, but it didn't work... What am I doing wrong?
Thank you!

When you used HttpClient to download large amounts of data (50 megabytes or more), then the app should stream those downloads and not use the default buffering. If the default buffering is used the client memory usage will get very large, potentially resulting in substantially reduced performance.
You could try to get the response body and load into memory.
using (var response = await client.GetAsync(get_url))
using (Stream streamToReadFrom =
await response.Content.ReadAsStreamAsync())
{
...........
//string fileToWriteTo = Path.GetTempFileName();
//using (Stream streamToWriteTo = File.Open(fileToWriteTo, FileMode.Create))
//{
// await streamToReadFrom.CopyToAsync(streamToWriteTo);
//}
}

Web API MultipartAsync Task no result

The goal is to upload a single file to my webserver and then store it to mssql database by using multipartcontent. While the file is uploading, a progress bar should be displayed in the client (WPF application). The following code sample shows only the upload to memorystream (no database connection).
The connection from client to server works, the upload to the MemoryStream at server-side works and receiving receiving the percentage to client side works (section ContinueWith in my sample code). The problem is, the client doesn't receive the final CreateResponse request - like a timeout or lost connection, I am not sure because i doesn't get an error/exception. The client never receives the task final result.
WebApi:
public class AttachmentsController : ApiController
{
[HttpPost]
[Route("api/Attachments/Upload")]
public async Task<HttpResponseMessage> Upload()
{
if (!Request.Content.IsMimeMultipartContent())
throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
try
{
AttachmentUpload _resultAttachmentUpload = null;
var _provider = new InMemoryMultipartFormDataStreamProvider();
await Request.Content.ReadAsMultipartAsync(_provider)
.ContinueWith(t =>
{
if (t.IsFaulted || t.IsCanceled)
{
throw new HttpResponseException(
HttpStatusCode.InternalServerError);
}
return new AttachmentUpload()
{
FileName = _provider.Files[0].Headers.ContentDisposition
.FileName.Trim('\"'),
Size = _provider.Files[0].ReadAsStringAsync().Result
.Length / 1024
};
});
return Request.CreateResponse<AttachmentUpload>(HttpStatusCode.Accepted,
_resultAttachmentUpload);
}
catch (Exception e)
{
return Request.CreateErrorResponse(HttpStatusCode.InternalServerError,
e.ToString());
}
}
}
WPF Client UploadService.cs:
private async Task<Attachment> UploadAttachment(AttachmentUpload uploadData,
string filePath)
{
try
{
var _encoding = Encoding.UTF8;
MultipartFormDataContent _multipartFormDataContent =
new MultipartFormDataContent();
_multipartFormDataContent.Add(new StreamContent(new MemoryStream(
File.ReadAllBytes(filePath))), uploadData.FileName, uploadData.FileName);
_multipartFormDataContent.Add(new StringContent(uploadData.Id.ToString()),
"AttachmentId", "AttachmentId");
_multipartFormDataContent.Add(new StringContent(
uploadData.Upload.ToString(CultureInfo.CurrentCulture)), "AttachmentUpload",
"AttachmentUpload");
_multipartFormDataContent.Add(new StringContent(
uploadData.DocumentId.ToString()), "DocumentId", "DocumentId");
_multipartFormDataContent.Add(new StringContent(
uploadData.User, _encoding), "User", "User");
//ProgressMessageHandler is instantiate in ctor to show progressbar
var _client = new HttpClient(ProgressMessageHandler);
_client.DefaultRequestHeaders.Accept.Add(
new MediaTypeWithQualityHeaderValue("multipart/form-data"));
_client.Timeout = TimeSpan.FromMinutes(5);
var _requestUri = new Uri(BaseAddress + "api/Attachments/Upload");
var _httpRequestTask = await _client.PostAsync(_requestUri,
_multipartFormDataContent)
.ContinueWith<AttachmentUpload>(request =>
{
var _httpResponse = request.Result;
if (!_httpResponse.IsSuccessStatusCode)
{
throw new Exception();
}
var _response = _httpResponse.Content.ReadAsStringAsync();
AttachmentUpload _upload =
JsonConvert.DeserializeObject<AttachmentUpload>(_response.Result);
return _upload;
});
var _resultAttachment = _httpRequestTask;
return new Attachment()
{
Id = _resultAttachment.Id,
FileName = _resultAttachment.FileName,
Comment = _resultAttachment.Comment,
Upload = _resultAttachment.Upload,
};
}
catch (Exception e)
{
//Handle exceptions
//file not found, access denied, no internet connection etc etc
var tmp = e;
throw e;
}
}
The program lacks at var _httpRequestTask = await _client.PostAsync(...).
The debugger never reaches the line var _resultAttachment = _httpRequestTask;.
Thank you very much for your help.

First of all, don't mix await and ContinueWith, the introduction of async / await effectively renders ContinueWith obsolete.
The reason that var _resultAttachment = _httpRequestTask; is never hit is because you have created a deadlock.
WPF has a synchronisation context, that ensures continuations resume on the UI thread.
In the line AttachmentUpload _upload = JsonConvert.DeserializeObject<AttachmentUpload>(_response.Result);, _response.Result is a blocking call; it blocks the current thread, until the Task referenced by _response has completed.
The ReadAsStringAsync method, which generated the Task, will attempt to resume once the asynchronous work has completed, and the WPF synchronisation context will force it to use the UI thread, which has been blocked by _response.Result, hence the deadlock.
To remedy this, use the await keyword for every asynchronous call:
var _httpResponse = await _client.PostAsync(_requestUri, _multipartFormDataContent);
if (!_httpResponse.IsSuccessStatusCode)
{
throw new Exception();
}
var _response = await _httpResponse.Content.ReadAsStringAsync();
var _resultAttachment = JsonConvert.DeserializeObject<AttachmentUpload>(_response);
You should also notice that the code is much more readable without the ContinueWith.

WriteToStreamAsync cancel does not work

I am running a Task, which copies from one stream to another. This works without problems, including progress reporting. But i cant cancel the task. If i fire the CancellationToken, the copy progress runs till its completion, then the task is cancelled, but this is of course to late. Here is my actual code
private async Task Download(Uri uriToWork, CancellationToken cts)
{
HttpClient httpClient = new HttpClient();
HttpRequestMessage requestAction = new HttpRequestMessage();
requestAction.Method = new HttpMethod("GET");
requestAction.RequestUri = uriToWork;
HttpResponseMessage httpResponseContent = await httpClient.SendRequestAsync(requestAction, HttpCompletionOption.ResponseHeadersRead);
using (Stream streamToRead = (await httpResponseContent.Content.ReadAsInputStreamAsync()).AsStreamForRead())
{
string fileToWrite = Path.GetTempFileName();
using (Stream streamToWrite = File.Open(fileToWrite, FileMode.Create))
{
await httpResponseContent.Content.WriteToStreamAsync(streamToWrite.AsOutputStream()).AsTask(cts, progressDownload);
await streamToWrite.FlushAsync();
//streamToWrite.Dispose();
}
await streamToRead.FlushAsync();
//streamToRead.Dispose();
}
httpClient.Dispose();
}
Can someone help me please, or can explain, why it does not work?

Is it this operation that continues until it completes ?
await httpResponseContent.Content.WriteToStreamAsync(streamToWrite.AsOutputStream()).AsTask(cts, progressDownload);
Or is it this one ?
await streamToWrite.FlushAsync();
I think the latter needs probably to have the CancellationToken as well:
await streamToWrite.FlushAsync(cts);
Unfortunately I cannot answer why this cancel does not occur. However, a solution that consists in writing the Stream in chunks may help.
Here is something very dirty that works:
private async Task Download(Uri uriToWork, CancellationToken cts) {
using(HttpClient httpClient = new HttpClient()) {
HttpRequestMessage requestAction = new HttpRequestMessage();
requestAction.Method = new HttpMethod("GET");
requestAction.RequestUri = uriToWork;
HttpResponseMessage httpResponseContent = await httpClient.SendRequestAsync(requestAction, HttpCompletionOption.ResponseHeadersRead);
string fileToWrite = Path.GetTempFileName();
using(Stream streamToWrite = File.Open(fileToWrite, FileMode.Create)) {
// Disposes streamToWrite to force any write operation to fail
cts.Register(() => streamToWrite.Dispose());
try {
await httpResponseContent.Content.WriteToStreamAsync(streamToWrite.AsOutputStream()).AsTask(cts, p);
}
catch(TaskCanceledException) {
return; // "gracefully" exit when the token is cancelled
}
await streamToWrite.FlushAsync();
}
}
}
I enclosed the httpClient in a using so a return disposes it properly.
I removed the streamToRead which was not used at all
Now here is the horror: I added a delegate that executes when the token is cancelled: it disposes streamToWrite while it is written to (ughhhh), which triggers an TaskCancelledException when WriteToStreamAsync cannot longer write in this disposed stream.
Please dont throw a puke bag at me yet, I am not experienced enough in this "Universal" Framework which looks very different as the usual one.
Here is a chunked stream solution that looks more acceptable. I shortened a bit the original code and added the IProgress as a parameter.
async Task Download(Uri uriToWork, CancellationToken cts, IProgress<int> progress) {
using(HttpClient httpClient = new HttpClient()) {
var chunkSize = 1024;
var buffer = new byte[chunkSize];
int count = 0;
string fileToWrite = Path.GetTempFileName();
using(var inputStream = await httpClient.GetInputStreamAsync(uriToWork)) {
using(var streamToRead = inputStream.AsStreamForRead()) {
using(Stream streamToWrite = File.OpenWrite(fileToWrite)) {
int size;
while((size = await streamToRead.ReadAsync(buffer, 0, chunkSize, cts).ConfigureAwait(false)) > 0) {
count += size;
await Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () => progress.Report(count));
// progress.Report(count);
await streamToWrite.WriteAsync(buffer, 0, size, cts).ConfigureAwait(false);
}
}
}
}
}
}

The blocking operation is most probably not WriteToStreamAsync() but FlushAsync(), so #Larry's assumption should be right, the FlushAsync method needs the cancellation token as well.

Unknown / empty background task after calling BackgroundDownloadAsync or BackgroundUploadAsync from Live SDK

Any idea why there will be an "empty" task listed under Lifecycle Events whenever a call to BackgrouDownloadAsync or BackgroundUploadAsync is made?
I can't post a picture yet. Please refer to the originally post in http://social.msdn.microsoft.com/Forums/en-US/39944e7d-feb2-4e06-b980-6ff41588ec50/unknown-empty-background-task?forum=wpdevelop

Here's what I did to replace BackgroundDownloadAsync and BackgroundUploadAsync, omitting any try-catch and error checking codes. These calls will not cause the empty entry in "Lifecycle Events" drop-down. Not sure if it is due to issue in VS 2013.
To download (this is messy, there must be another way)
LiveConnectClient connectClient = new LiveConnectClient(this.Session);
LiveOperationResult _opResult = await connectClient.GetAsync(FileID + "/content");
dynamic _result = _opResult.Result;
CancellationTokenSource cts = new CancellationTokenSource();
HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, new Uri(_result.location as string, UriKind.Absolute));
HttpResponseMessage response = await httpClient.SendRequestAsync(request, HttpCompletionOption.ResponseHeadersRead).AsTask(cts.Token);
using (var _stream = (await OutputFile.OpenStreamForWriteAsync()).AsOutputStream())
{
await response.Content.WriteToStreamAsync(_stream).AsTask(cts.Token);
await _stream.FlushAsync();
}
To upload
LiveConnectClient connectClient = new LiveConnectClient(this.Session);
using (Stream stream = await InputFile.OpenStreamForReadAsync())
{
using (StreamReader reader = new StreamReader(stream))
{
LiveOperationResult _opResult = await connectClient.PutAsync(Awesome2FolderID + "/files/" + OneDriveFilename, reader.ReadToEnd());
}
}