Stream Extensions to convert Stream content into String or Byte array - c#

Using C# 10 I am creating Stream extensions to get content into a String or Byte array.
Something similar to File.ReadAllTextAsync in Microsoft's Net 6.
public static async Task<string> ReadAllTextAsync(this Stream stream). {
string result;
using (var reader = new StreamReader(stream)) {
result = await reader.ReadToEndAsync().ConfigureAwait(false);
}
return result;
}
public static async Task<byte[]> ReadAllBytesAsync(this Stream stream) {
using (var content = new MemoryStream()) {
var buffer = new byte[4096];
int read = await stream.ReadAsync(buffer, 0, 4096).ConfigureAwait(false);
while (read > 0) {
content.Write(buffer, 0, read);
read = await stream.ReadAsync(buffer, 0, 4096).ConfigureAwait(false);
}
return content.ToArray();
}
}
public static async Task<List<string>> ReadAllLinesAsync(this Stream stream) {
var lines = new List<string>();
using (var reader = new StreamReader(stream)) {
string line;
while ((line = await reader.ReadLineAsync().ConfigureAwait(false)) != null) {
lines.Add(line);
}
}
return lines;
}
Is there a better way to do this?
I am not sure about the ConfigureAwait(false) that I picked on some code online.

A better alternative for the ReadAllBytesAsync is
public static async Task<byte[]> ReadAllBytesAsync(this Stream stream)
{
switch (stream)
{
case MemoryStream mem:
return mem.ToArray();
default:
using var m = new MemoryStream();
await stream.CopyToAsync(m);
return mem.ToArray();
}
}
For the ReadAllLinesAsync, the async stream in C# 8 can make the code cleaner:
public IAsyncEnumerable<string> ReadAllLinesAsync(this Stream stream)
{
using var reader = new StreamReader(stream)
while (await reader.ReadLineAsync() is { } line)
{
yield return line;
}
}
notice that the empty brace { } here is actually a property pattern that is only available after C# 8, it checks whether reader.ReadLineAsync() is null, if it's not, assign it to the line variable.
Usage:
var lines = await stream.ReadAllLinesAsync();
await foreach (var line in lines)
{
// write your own logic here
}
P.S.:
The ConfigureAwait(false) is kinda useless if your app is single-threaded like console apps, it instructs the awaiter not to capture the SynchronizationContext and let continuation run on the thread that runs the await statement, this method is useful when you're writing a library or SDK, since your user may use your library in a GUI application, and the combination of block waiting such as calling Task.Wait() and the capturing of SynchronizationContext often leads to deadlock, and ConfigureAwait(false) solves this. For detail explanation see ConfigureAwait FAQ

Related

Azure blob storage V12 - Example of using specialized class BlockBlobStorage

I have hard time with new version of Azure Storage Blobs client library for .NET.
What I need is create stream where I can write data and let's say, after stream reach size of 4MB, then I need to upload it. I found BlockBlobClient. There are two methods CommitBlockListAsync and StageBlockAsync. This methods looks like what I need, but I can't find some examples of usage.
Do you know about some scenario similar to my needs? Or can you someone help me understand this client?
Something like this I need, Every 4MB stage, clear stream and continue to write:
public class MyStreamWrapper : Stream
{
readonly BlockBlobClient _blockBlobClient;
readonly Stream _wrappedStream;
bool _isCommited;
readonly List<string> _blockIds;
public MyStreamWrapper (BlockBlobClient blockBlobClient)
{
_wrappedStream = new MemoryStream();
_blockBlobClient = blockBlobClient;
_isCommited = false;
_blockIds = new List<string>();
}
public override async Task WriteAsync(byte[] buffer, int offset, int count, CancellationToken cancellationToken)
{
if ((_wrappedStream.Length + buffer.Length) / 1024 > 4) // check size if
{
int byteCount = (int)(_wrappedStream.Length - buffer.Length);
if (byteCount > 0)
{
_wrappedStream.Write(buffer, offset, byteCount);
offset += byteCount;
}
string base64Id = Convert.ToBase64String(buffer);
_blockIds.Add(base64Id);
_blockBlobClient.StageBlock(base64Id, _wrappedStream);
_wrappedStream.Flush();
}
await _wrappedStream.WriteAsync(buffer, offset, count, cancellationToken);
}
}
For the APIs that don't appear in the samples folder, look at the tests.
eg
[Test]
public async Task CommitBlockListAsync()
{
await using DisposingContainer test = await GetTestContainerAsync();
// Arrange
BlockBlobClient blob = InstrumentClient(test.Container.GetBlockBlobClient(GetNewBlobName()));
var data = GetRandomBuffer(Size);
var firstBlockName = GetNewBlockName();
var secondBlockName = GetNewBlockName();
var thirdBlockName = GetNewBlockName();
// Act
// Stage blocks
using (var stream = new MemoryStream(data))
{
await blob.StageBlockAsync(ToBase64(firstBlockName), stream);
}
using (var stream = new MemoryStream(data))
{
await blob.StageBlockAsync(ToBase64(secondBlockName), stream);
}
// Commit first two Blocks
var commitList = new string[]
{
ToBase64(firstBlockName),
ToBase64(secondBlockName)
};
await blob.CommitBlockListAsync(commitList);
// Stage 3rd Block
using (var stream = new MemoryStream(data))
{
await blob.StageBlockAsync(ToBase64(thirdBlockName), stream);
}
// Assert
Response<BlockList> blobList = await blob.GetBlockListAsync(BlockListTypes.All);
Assert.AreEqual(2, blobList.Value.CommittedBlocks.Count());
Assert.AreEqual(ToBase64(firstBlockName), blobList.Value.CommittedBlocks.First().Name);
Assert.AreEqual(ToBase64(secondBlockName), blobList.Value.CommittedBlocks.ElementAt(1).Name);
Assert.AreEqual(1, blobList.Value.UncommittedBlocks.Count());
Assert.AreEqual(ToBase64(thirdBlockName), blobList.Value.UncommittedBlocks.First().Name);
}
https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/storage/Azure.Storage.Blobs/tests/BlockBlobClientTests.cs
Also familiarize yourself with the REST API, for which the client libraries are wrappers. You are using a "lower level" API approach, that maps directly to Put Block and Put Block List REST APIs.

Return stream immediately and then write to stream asynchronously

In my current code I have a method like this to read data from a device (pseudo code):
public async Task<string> ReadAllDataFromDevice()
{
var buffer = "";
using (var device = new Device())
{
while(device.HasMoreData)
{
buffer += await device.ReadLineAsync();
}
}
return buffer;
}
I then want to send all that data via the network to some receiver. The amount of data can be really large. So clearly the above design is not very memory-efficient since it requires to read all the data before I can start sending it to the network socket.
So what I'd like to have is a function that returns a stream instead. Something like this:
public async Task<Stream> ReadAllDataFromDevice()
{
var stream = new MemoryStream();
using (var device = new Device())
using (var streamWriter = new StreamWriter(stream, new UTF8Encoding(), 512, true))
{
while(device.HasMoreData)
{
var line = await device.ReadLineAsync();
await streamWriter.WriteLineAsync(line);
}
await streamWriter.FlushAsync();
}
return stream;
}
This returns a stream but it clearly does not solve my problem, because the stream is returned only after all the data has been read from the device.
So I came up with this:
public Stream ReadAllDataFromDevice()
{
var stream = new MemoryStream();
Task.Run(async () => {
using (var device = new Device())
using (var streamWriter = new StreamWriter(stream, new UTF8Encoding(), 512, true))
{
while(device.HasMoreData)
{
var line = await device.ReadLineAsync();
await streamWriter.WriteLineAsync(line);
}
await streamWriter.FlushAsync();
}
});
return stream;
}
Is this a good design? I'm especially concerned about thread-safety, lifetime of the stream object used in the lambda, and exception handling.
Or is there a better pattern for this kind of problem?
Edit
Actually I just came up with another design that looks much cleaner to me. Instead of having the ReadAllDataFromDevice() function returning a stream, I let the consumer of the data provide the stream, like this:
public async Task ReadAllDataFromDevice(Stream stream)
{
using (var device = new Device())
using (var streamWriter = new StreamWriter(stream, new UTF8Encoding(), 512, true))
{
while(device.HasMoreData)
{
var line = await device.ReadLineAsync();
await streamWriter.WriteLineAsync(line);
}
await streamWriter.FlushAsync();
}
}
This is the design I'm using now:
public async Task ReadAllDataFromDevice(Func<Stream, Task> readCallback)
{
using (var device = new Device())
{
await device.Initialize();
using (var stream = new DeviceStream(device))
{
await readCallback(stream);
}
}
}
The line-by-line device access is encapsulated in the custom DeviceStream class (not shown here).
The consumer of the data would look something like this:
await ReadAllDataFromDevice(async stream => {
using (var streamReader(stream))
{
var data = await streamReader.ReadToEndAsync();
// do something with data
}
});

Asynchronous Programming in C# with Tasks [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
I started to learn about async programming and I stumbled across this problem. I call these functions over an Dispatcher.BeginInvoke.
I'm expecting to get file content form ReadText method, but all I get back is "System.Threading.Tasks.Task`1[System.String]"
So question is what's wrong with my code and which line should I fix?
Sadly I can't figure out where my problem is because I am looking at this peace of code for quite some time now.
I thought because I get back the time of the object and not the object itself that my ReadText method is wrong, but I don't see where. It seems there is something wrong with my part below the Stringbuilder or the way I made these methods async.
If you wonder why I used two methods is to get known with await and calling async Task methods. I also tried making the method a Tast method but that only resulted in even more problems.
Thanks for your help in advance.
public async void ReadFile()
{
string filePath = #"SampleFile.txt";
if (File.Exists(filePath) == false)
{
MessageBox.Show(filePath + " not found", "File Error", MessageBoxButton.OK);
}
else
{
try
{
string text = await ReadText(filePath);
txtContents.Text = text;
}
catch (Exception ex)
{
Debug.WriteLine(ex.Message);
}
}
}
private async Task<string> ReadText(string filePath)
{
Task Readfile = Task.Run(() =>
{
using (FileStream sourceStream = new FileStream(filePath,
FileMode.Open, FileAccess.Read, FileShare.Read,
bufferSize: 4096, useAsync: true))
{
StringBuilder sb = new StringBuilder();
byte[] buffer = new byte[0x1000];
int numRead;
while ((numRead = sourceStream.Read(buffer, 0, buffer.Length)) != 0)
{
string text = Encoding.Unicode.GetString(buffer, 0, numRead);
sb.Append(text);
}
return sb.ToString();
}
}
);
await Readfile;
return Readfile.ToString();
}
You are returning a string, so make Readfile a Task<string>:
Task<string> Readfile = Task.Run<string>( ... )
Then you want to return Readfile.Result;, not Readfile.ToString();
But you can write this simpler:
return await Readfile;
or even:
return await Task.Run<string>( ... )
You don't need to use Task.Run for reading from the stream, there is convenient method ReadAsync that you can await:
public async Task ReadFile() {
string filePath = #"SampleFile.txt";
if (File.Exists(filePath) == false) {
MessageBox.Show(filePath + " not found", "File Error", MessageBoxButton.OK);
} else {
try {
string text = await ReadText(filePath);
txtContents.Text = text;
} catch (Exception ex) {
Debug.WriteLine(ex.Message);
}
}
}
private async Task<string> ReadText(string filePath) {
using (FileStream sourceStream = new FileStream(filePath,
FileMode.Open, FileAccess.Read, FileShare.Read,
bufferSize: 4096, useAsync: true)) {
StringBuilder sb = new StringBuilder();
byte[] buffer = new byte[0x1000];
int numRead;
while ((numRead = await sourceStream.ReadAsync(buffer, 0, buffer.Length)) != 0) {
string text = Encoding.Unicode.GetString(buffer, 0, numRead);
sb.Append(text);
}
return sb.ToString();
}
}
Also avoid async void is not recommended, try to avoid that.Async void methods have different error-handling semantics. When an exception is thrown out of an async Task or async Task method, that exception is captured and placed on the Task object. With async void methods, there is no Task object, so any exceptions thrown out of an async void method will be raised directly on the SynchronizationContext that was active when the async void method started.
Best practice for async methods

Creating a file asynchronously

How can I modify this method to call it asynchronously?
private void Write(string fileName, data)
{
File.WriteAllText(fileName, data);
}
Look into FileStream.WriteAsync (Note you have to use the proper overload which takes a bool indicating if it should run async:)
public async Task WriteAsync(string data)
{
var buffer = Encoding.UTF8.GetBytes(data);
using (var fs = new FileStream(#"File", FileMode.OpenOrCreate,
FileAccess.Write, FileShare.None, buffer.Length, true))
{
await fs.WriteAsync(buffer, 0, buffer.Length);
}
}
Edit
If you want to use your string data and avoid the transformation to a byte[], you can use the more abstracted and less verbose StreamWriter.WriteAsync overload which accepts a string:
public async Task WriteAsync(string data)
{
using (var sw = new StreamWriter(#"FileLocation"))
{
await sw.WriteAsync(data);
}
}
With .NetCore 2.0 you can just use File.WriteAllTextAsync

Windows Store App how to delete text from file

In a windows store app, how can I delete text from a file ? For example
If I have
StorageFile file = await roamingfolder.CreateFileAsync(filename,
CreationCollisionOption.OpenIfExists);
await FileIO.AppendTextAsync(file, temp);
How can i remove some text from this file ?
You generally read the text into a string, remove the text, and rewrite the file.
Here I get a file then I put the content to a stringbuilder then do some string operations, finally put the string back to the file using DataWriter
public static async Task UpdateTextContent(string contentItemId)
{
var storageFolder = await ApplicationData.Current.LocalFolder.GetFolderAsync(TARGET_FOLDER);
StorageFile sf = null;
try
{
//get content of the file make sure that it exist
sf = await storageFolder.GetFileAsync(TARGET_FILE);
}
catch (Exception ex)
{
}
if (sf != null)
{
var targettxtfile = await Windows.Storage.FileIO.ReadTextAsync(sf);
var sbProcessedTextToWrite = new StringBuilder(targettxtfile);
if (targettxtfile.IndexOf(contentItemId) >= 0)
{
string startmarker = new StringBuilder("[").Append(contentItemId).Append("#start]").ToString();
string endmarker = new StringBuilder("[").Append(contentItemId).Append("#end]").ToString();
int start = sbProcessedTextToWrite.ToString().IndexOf(startmarker);
int end = sbProcessedTextToWrite.ToString().IndexOf(endmarker);
int slen = end + endmarker.Length - start;
//compute position to remove
sbProcessedTextToWrite.Remove(start, slen);
}
using (IRandomAccessStream fileStream = await sf.OpenAsync(FileAccessMode.ReadWrite))
{
using (IOutputStream outputStream = fileStream.GetOutputStreamAt(0))
{
using (DataWriter dataWriter = new DataWriter(outputStream))
{
dataWriter.WriteString(sbProcessedTextToWrite.ToString());
await dataWriter.StoreAsync();
// For the in-memory stream implementation we are using, the flushAsync call
// is superfluous,but other types of streams may require it.
await dataWriter.FlushAsync();
// In order to prolong the lifetime of the stream, detach it from the
// DataWriter so that it will not be closed when Dispose() is called on
// dataWriter. Were we to fail to detach the stream, the call to
// dataWriter.Dispose() would close the underlying stream, preventing
// its subsequent use by the DataReader below.
dataWriter.DetachStream();
}
//same here flush the outputStream as well
await outputStream.FlushAsync();
}
}
}
}
Some references for this code

Categories