We Keep Coding

Proper way to use DisposeAsync on C# streams

I'm writing a method which asynchronously writes separate lines of text to a file. If it's cancelled it deletes the created file and jumps out of the loop.
This is the simplified code which works fine... And I marked 2 points which I'm not sure how they are being handled. I want the code to not block the thread in any case.
public async Task<IErrorResult> WriteToFileAsync(string filePath,
CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
using var stream = new FileStream(filePath, FileMode.Create);
using var writer = new StreamWriter(stream, Encoding.UTF8);
foreach (var line in Lines)
{
if (cancellationToken.IsCancellationRequested)
{
//
// [1] close, delete and throw if cancelled
//
writer.Close();
stream.Close();
if (File.Exists(filePath))
File.Delete(filePath);
throw new OperationCanceledException();
}
// write to the stream
await writer.WriteLineAsync(line.ToString());
}
//
// [2] flush and let them dispose
//
await writer.FlushAsync();
await stream.FlushAsync();
// await stream.DisposeAsync(); ??????
return null;
}
1
I'm calling Close() on FileStream and StreamWriter and I think it will run synchronously and blocks the thread. How can I improve this? I don't want to wait for it to flush the buffer into the file and then delete the file.
2
I suppose the Dispose method will be called and not DisposeAsync at the end of the using scope. (is this assumption correct?).
So Dispose blocks the thread and in order to prevent that I'm flushing first with FlushAsync so that Dispose would perform less things. (to what extent is this true?)
I could also remove using and instead I could write DisposeAsync manually in these two places. But it will decrease readability.
If I open the FileStream with useAsync = true would it automatically call DisposeAsync when using block ends?
Any explanation or a variation of the above code which performs better is appreciated.

As you have it, the using statement will call Dispose(), not DisposeAsync().
C# 8 brought a new await using syntax, but for some reason it's not mentioned in the What's new in C# 8.0 article.
But it's mentioned elsewhere.
await using var stream = new FileStream(filePath, FileMode.Create);
await using var writer = new StreamWriter(stream, Encoding.UTF8);
But also note that this will only work if:
You're using .NET Core 3.0+ since that's when IAsyncDisposable was introduced, or
Install the Microsoft.Bcl.AsyncInterfaces NuGet package. Although this only adds the interfaces and doesn't include the versions of the Stream types (FileStream, StreamWriter, etc.) that use it.
Even in the Announcing .NET Core 3.0 article, IAsyncDisposable is only mentioned in passing and never expanded on.
On another note, you don't need to do this (I see why now):
writer.Close();
stream.Close();
Since the documentation for Close says:
This method calls Dispose, specifying true to release all resources. You do not have to specifically call the Close method. Instead, ensure that every Stream object is properly disposed.
Since you're using using, Dispose() (or DisposeAsync()) will be called automatically and Close won't do anything that's not already happening.
So if you do need to specifically close the file, but want to do it asynchronously, just call DisposeAsync() instead. It does the same thing.
await writer.DisposeAsync();
public async Task<IErrorResult> WriteToFileAsync(string filePath,
CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
await using var stream = new FileStream(filePath, FileMode.Create);
await using var writer = new StreamWriter(stream, Encoding.UTF8);
foreach (var line in Lines)
{
if (cancellationToken.IsCancellationRequested)
{
// not possible to discard, FlushAsync is covered in DisposeAsync
await writer.DisposeAsync(); // use DisposeAsync instead of Close to not block
if (File.Exists(filePath))
File.Delete(filePath);
throw new OperationCanceledException();
}
// write to the stream
await writer.WriteLineAsync(line.ToString());
}
// FlushAsync is covered in DisposeAsync
return null;
}

How to write a test to expose returning a task from a using block?

I wrote the following buggy async method, which passes its unit test but fails in production (EDIT: in production it throws an ObjectDisposedException):
public class FileUtils {
public Task<string> ReadAllText(string path)
{
using (var stream = ReadStreamAsync(path))
using (var reader = new StreamReader(stream))
{
return reader.ReadToEndAsync();
}
}
private static FileStream ReadStreamAsync(string path)
{
return new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read,
4096, FileOptions.Asynchronous);
}
}
The error is that if you return a task from inside a using block, the code will only work if the ReadToEndAsync call happens to execute synchronously (which apparently happened in my unit test).
The correct code adds async and await as follows:
public async Task<string> ReadAllText(string path)
{
using (var stream = ReadStreamAsync(path))
using (var reader = new StreamReader(stream))
{
return await reader.ReadToEndAsync();
}
}
My question is: How would I write a unit test that would reliably fail for the incorrect version of the code?
EDIT: Here is the current (insufficient) unit test that does not expose this problem:
[Test]
public async Task GivenFileUtilsWhenReadAllTextThenGetsText()
{
var fileUtils = new FileUtils(); // the prod code above is in class FileUtils
var path = #"C:\tmp\foo.txt";
var expected = "foo";
File.WriteAllText(path, expected);
var text = await fileUtils.ReadAllText(path);
text.Should().Be(expected);
}

How would I write a unit test that would reliably fail for the incorrect version of the code?
Frame it in terms of requirements. This is what you want to test:
"The SUT should not dispose the stream immediately. It may dispose the stream after the stream has been completely read."
In order to do this, your unit test needs to control:
The stream being disposed, so it can detect when disposal happens.
When the stream is completely read.
Both of these can be handled using a custom stream type.
For the first stub requirement, your custom stream type can just have a bool Disposed property that is set to true when Dispose is called.
For the second stub requirement, your custom stream type can be implemented to only complete asynchronous operations after a signal is received. One type for an "asynchronous signal" is TaskCompletionSource<T> - you can create the instance inside the custom stream, have every async method await its Task property, and when your unit test is ready for the stream to complete, it can complete the TaskCompletionSource<T>.

The problem is that you do not put enough content in the file to cause it to be disposed when read asynchronously.
So it is not that your code executed synchronously. It's finishing its task too quickly to cause the issue experienced in production.
To prove it, I just added more content
for (int i = 0; i < 10; i++) { expected += expected; }
before writing to the file and ran the test
var fileUtils = new FileUtils(); // the prod code above is in class FileUtils
var path = #"foo.txt";
var expected = "foo";
for (int i = 0; i < 15; i++) {
expected += expected;
}
File.WriteAllText(path, expected);
var text = await fileUtils.ReadAllText(path);
text.Should().Be(expected);
and sure enough object disposed exception caused the a failed test.
You have already discovered the solution to the problem code by awaiting within the using block.
Writing tests to discover what you seek would be rather difficult, however Code Analyzers would have a better chance of discovering such mistakes at compile time.

Write to open FileStream using reactive programming

I am writing a small logger and I want to open the log file once, keep writing reactively as log messages arrive, and dispose of everything on program termination.
I am not sure how I can keep the FileStream open and reactively write the messages as they arrive.
I would like to update the design from my previous solution where I had a ConcurrentQueue acting as a buffer, and a loop inside the using statements that consumed the queue.
Specifically, I want to simultaneously take advantage of the using statement construct, so I don't have to explicitly close the stream and writer, and of the reactive, loopless programming style. Currently I only know how to use one of these constructs at once: either the using/loop combination, or the explicit-stream-close/reactive combination.
Here's my code:
BufferBlock<LogEntry> _buffer = new BufferBlock<LogEntry>();
// CONSTRUCTOR
public DefaultLogger(string folder)
{
var filePath = Path.Combine(folder, $"{DateTime.Now.ToString("yyyy.MM.dd")}.log");
_cancellation = new CancellationTokenSource();
var observable = _buffer.AsObservable();
using (var stream = File.Create(_filePath))
using (var writer = new StreamWriter(stream))
using (var subscription = observable.Subscribe(entry =>
writer.Write(GetFormattedString(entry))))
{
while (!_cancellation.IsCancellationRequested)
{
// what do I do here?
}
}
}

You need to use Observable.Using. It's designed to create an IDisposble resource that gets disposed when the sequence ends.
Try something like this:
IDisposable subscription =
Observable.Using(() => File.Create(_filePath),
stream => Observable.Using(() => new StreamWriter(stream),
writer => _buffer.AsObservable().Select(entry => new { entry, writer })))
.Subscribe(x => x.writer.Write(GetFormattedString(x.entry)));

Method that can be accessed by multiple threads

I want to write a method to open (or create if it does not exist) a file from different threads.
What would be the FileAccess and FileShare flags in this case? I tried both FileAccess.Read/Write and FileShare.Read/Write but don't see any differences. I used the following code to test, looks fine but not sure about the flags (last 2).
Can anybody clarify should I use FileAccess.ReadWrite or FileAccess.Read and FileShare.ReadWrite or FileShare.Read?
class Program
{
static void Main(string[] args)
{
Task first = Task.Run(() => AccessFile());
Task second = Task.Run(() => AccessFile());
Task third = Task.Run(() => AccessFile());
Task fourth = Task.Run(() => AccessFile());
Task.WaitAll(first, second, third, fourth);
Task[] tasks = new Task[100];
for (int i = 0; i < 100; i++)
{
tasks[i] = Task.Run(() => AccessFile());
}
Task.WaitAll(tasks);
}
public static void AccessFile()
{
string path = #"c:\temp\test.txt";
// FileShare.Write gives access violation
using (FileStream fs = File.Open(path, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite))
{
byte[] by = new byte[100];
fs.Read(by, 0, 100);
Console.WriteLine(Encoding.ASCII.GetString(by));
}
}
}

If multiple threads need to write to the file then you need an exclusive lock on it. If you could have some threads read and others write to the file you could use a ReaderWriterLockSlim.
The FileShare parameter is used to indicate how other threads/processes can access the file while a handle is being opened in the current thread. ReadWrite means that other threads will be able to read and write from this file which obviously means that if you try to write to the file from the current handle you will probably corrupt the contents.

Since you are only reading you should specify just that (FileAccess.Read). And you are OK with others reading so say that (FileShare.Read).
And here's nicer test code:
ParallelEnumerable.Range(0, 100000).ForAll(_ => AccessFile());

StreamWriter Creates Zero-Byte File

I have a Task that reads strings from a blocking collection and is supposed to write them out to a file. Trouble is, while the file is created, the size of the file is 0 bytes after the task completes.
While debugging, I see that non-empty lines are retrieved from the blocking collection, and the stream writer is wrapped in a using block.
For debugging I threw in a flush that should not be required and write the lines to the console. There are 100 non-empty lines of text read from the blocking collection.
// Stuff is placed in writeQueue from a different task
BlockingCollection<string> writeQueue = new BlockingCollection<string>();
Task writer = Task.Factory.StartNew(() =>
{
try
{
while (true)
{
using (FileStream fsOut = new FileStream(destinationPath, FileMode.Create, FileAccess.Write))
using (BufferedStream bsOut = new BufferedStream(fsOut))
using (StreamWriter sw = new StreamWriter(bsOut))
{
string line = writeQueue.Take();
Console.WriteLine(line); // Stuff is written to the console
sw.WriteLine(line);
sw.Flush(); // Just in case, makes no difference
}
}
}
catch (InvalidOperationException)
{
// We're done.
}
});
Stepping through in the debugger, I see that the program terminates in an orderly manner. There are no unhandled exceptions.
What might be going wrong here?

You are re-creating the file on every run of the loop. Change the FileMode.Create to FileMode.Append and it will keep the previous values you wrote on it.
Also, using exceptions to detect that you should stop is a really bad practice, if this a consumer-producer solution, you can easily do better by having the producer setting a thread safe flag variable signaling it has finished the work and will not produce anything else.