I have a Task that reads strings from a blocking collection and is supposed to write them out to a file. Trouble is, while the file is created, the size of the file is 0 bytes after the task completes.
While debugging, I see that non-empty lines are retrieved from the blocking collection, and the stream writer is wrapped in a using block.
For debugging I threw in a flush that should not be required and write the lines to the console. There are 100 non-empty lines of text read from the blocking collection.
// Stuff is placed in writeQueue from a different task
BlockingCollection<string> writeQueue = new BlockingCollection<string>();
Task writer = Task.Factory.StartNew(() =>
{
try
{
while (true)
{
using (FileStream fsOut = new FileStream(destinationPath, FileMode.Create, FileAccess.Write))
using (BufferedStream bsOut = new BufferedStream(fsOut))
using (StreamWriter sw = new StreamWriter(bsOut))
{
string line = writeQueue.Take();
Console.WriteLine(line); // Stuff is written to the console
sw.WriteLine(line);
sw.Flush(); // Just in case, makes no difference
}
}
}
catch (InvalidOperationException)
{
// We're done.
}
});
Stepping through in the debugger, I see that the program terminates in an orderly manner. There are no unhandled exceptions.
What might be going wrong here?
You are re-creating the file on every run of the loop. Change the FileMode.Create to FileMode.Append and it will keep the previous values you wrote on it.
Also, using exceptions to detect that you should stop is a really bad practice, if this a consumer-producer solution, you can easily do better by having the producer setting a thread safe flag variable signaling it has finished the work and will not produce anything else.
Related
I'm writing a method which asynchronously writes separate lines of text to a file. If it's cancelled it deletes the created file and jumps out of the loop.
This is the simplified code which works fine... And I marked 2 points which I'm not sure how they are being handled. I want the code to not block the thread in any case.
public async Task<IErrorResult> WriteToFileAsync(string filePath,
CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
using var stream = new FileStream(filePath, FileMode.Create);
using var writer = new StreamWriter(stream, Encoding.UTF8);
foreach (var line in Lines)
{
if (cancellationToken.IsCancellationRequested)
{
//
// [1] close, delete and throw if cancelled
//
writer.Close();
stream.Close();
if (File.Exists(filePath))
File.Delete(filePath);
throw new OperationCanceledException();
}
// write to the stream
await writer.WriteLineAsync(line.ToString());
}
//
// [2] flush and let them dispose
//
await writer.FlushAsync();
await stream.FlushAsync();
// await stream.DisposeAsync(); ??????
return null;
}
1
I'm calling Close() on FileStream and StreamWriter and I think it will run synchronously and blocks the thread. How can I improve this? I don't want to wait for it to flush the buffer into the file and then delete the file.
2
I suppose the Dispose method will be called and not DisposeAsync at the end of the using scope. (is this assumption correct?).
So Dispose blocks the thread and in order to prevent that I'm flushing first with FlushAsync so that Dispose would perform less things. (to what extent is this true?)
I could also remove using and instead I could write DisposeAsync manually in these two places. But it will decrease readability.
If I open the FileStream with useAsync = true would it automatically call DisposeAsync when using block ends?
Any explanation or a variation of the above code which performs better is appreciated.
As you have it, the using statement will call Dispose(), not DisposeAsync().
C# 8 brought a new await using syntax, but for some reason it's not mentioned in the What's new in C# 8.0 article.
But it's mentioned elsewhere.
await using var stream = new FileStream(filePath, FileMode.Create);
await using var writer = new StreamWriter(stream, Encoding.UTF8);
But also note that this will only work if:
You're using .NET Core 3.0+ since that's when IAsyncDisposable was introduced, or
Install the Microsoft.Bcl.AsyncInterfaces NuGet package. Although this only adds the interfaces and doesn't include the versions of the Stream types (FileStream, StreamWriter, etc.) that use it.
Even in the Announcing .NET Core 3.0 article, IAsyncDisposable is only mentioned in passing and never expanded on.
On another note, you don't need to do this (I see why now):
writer.Close();
stream.Close();
Since the documentation for Close says:
This method calls Dispose, specifying true to release all resources. You do not have to specifically call the Close method. Instead, ensure that every Stream object is properly disposed.
Since you're using using, Dispose() (or DisposeAsync()) will be called automatically and Close won't do anything that's not already happening.
So if you do need to specifically close the file, but want to do it asynchronously, just call DisposeAsync() instead. It does the same thing.
await writer.DisposeAsync();
public async Task<IErrorResult> WriteToFileAsync(string filePath,
CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
await using var stream = new FileStream(filePath, FileMode.Create);
await using var writer = new StreamWriter(stream, Encoding.UTF8);
foreach (var line in Lines)
{
if (cancellationToken.IsCancellationRequested)
{
// not possible to discard, FlushAsync is covered in DisposeAsync
await writer.DisposeAsync(); // use DisposeAsync instead of Close to not block
if (File.Exists(filePath))
File.Delete(filePath);
throw new OperationCanceledException();
}
// write to the stream
await writer.WriteLineAsync(line.ToString());
}
// FlushAsync is covered in DisposeAsync
return null;
}
I want to write to StreamWriter line async but I don't want to await on this.
for(int i= 0 ;i<1000;i++)
{
sw.WriteLineAsync(i.ToString());
}
But i got an error that i invoke to WriteLineAsync in same time.
What can I do to fix that?
I want to close this StreamWriter after this loop.
How can I verify that I now close without to write all data on StreamWriter, Or when I close the stream all the data that sent with WriteLineAsync will be write before the stream close?
If you want to write the lines using async, you should use await. Because, the file might be corrupted and you might encounter with possible stream errors like "the stream is already in use". In short, you should synchronize the write action.
So, I provided an example;
private async Task WriteToFileAsAsync()
{
string file = #"sample.txt";
using (FileStream stream = new FileStream(file, FileMode.Create, FileAccess.ReadWrite))
{
using (StreamWriter streamWriter = new StreamWriter(stream))
{
for (int i = 0; i < 1000; i++)
{
await streamWriter.WriteLineAsync(i.ToString());
}
}
}
}
Also, you can close and dispose the StreamWriter within using blocks.
EDIT
If you want to perform write action in separately from main thread, don't use async methods and just create a seperate Task and assign it an another thread.
private void WriteToFile()
{
string file = #"sample.txt";
using (FileStream stream = new FileStream(file, FileMode.Create, FileAccess.ReadWrite))
{
using (StreamWriter streamWriter = new StreamWriter(stream))
{
for (int i = 0; i < 1000; i++)
{
streamWriter.WriteLine(i.ToString());
}
}
}
}
Then call like this;
Task.Factory.StartNew(WriteToFile);
In your particular context there are two main issues in doing what you would like to do.
Technically speaking, if you call an async method, you need to await for it sooner or later, hence you can collect the task, and await for it later on. However, the WriteLineAsync method is not atomic, therefore calling it and performing other operations on the stream can corrupt the stream itself.
If you don't want to await, then don't call the Async method..
If you want to close a stream after using, use using with it.
using (StreamWriter writer = new StreamWriter("temp.txt"))
{
for (int i = 0; i < 1000; i++)
{
writer.WriteLine(i.ToString());
}
}
If you want it to be asynchronous, then you have to call the Async version, otherwise it will corrupt the file.
I am writing a small logger and I want to open the log file once, keep writing reactively as log messages arrive, and dispose of everything on program termination.
I am not sure how I can keep the FileStream open and reactively write the messages as they arrive.
I would like to update the design from my previous solution where I had a ConcurrentQueue acting as a buffer, and a loop inside the using statements that consumed the queue.
Specifically, I want to simultaneously take advantage of the using statement construct, so I don't have to explicitly close the stream and writer, and of the reactive, loopless programming style. Currently I only know how to use one of these constructs at once: either the using/loop combination, or the explicit-stream-close/reactive combination.
Here's my code:
BufferBlock<LogEntry> _buffer = new BufferBlock<LogEntry>();
// CONSTRUCTOR
public DefaultLogger(string folder)
{
var filePath = Path.Combine(folder, $"{DateTime.Now.ToString("yyyy.MM.dd")}.log");
_cancellation = new CancellationTokenSource();
var observable = _buffer.AsObservable();
using (var stream = File.Create(_filePath))
using (var writer = new StreamWriter(stream))
using (var subscription = observable.Subscribe(entry =>
writer.Write(GetFormattedString(entry))))
{
while (!_cancellation.IsCancellationRequested)
{
// what do I do here?
}
}
}
You need to use Observable.Using. It's designed to create an IDisposble resource that gets disposed when the sequence ends.
Try something like this:
IDisposable subscription =
Observable.Using(() => File.Create(_filePath),
stream => Observable.Using(() => new StreamWriter(stream),
writer => _buffer.AsObservable().Select(entry => new { entry, writer })))
.Subscribe(x => x.writer.Write(GetFormattedString(x.entry)));
I've the following code, that from the first Stream is reading from a file, and doing some interpretations to the content and write them to the second file, I'm facing a problem that when I've a big file the GUI in WPF is sticking, I tried to put the reading and the writing actions in:
Application.Current.Dispatcher.BeginInvoke(DispatcherPriority.Normal, new Action(() =>
{
// Here
});
This in the follwoing code:
using (StreamReader streamReader = new StreamReader(File.Open(fileName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)))
using (StreamWriter streamWriter = new StreamWriter(File.Open("Compressed_" + splitFilePath[splitFilePath.Length - 1], FileMode.Create, FileAccess.Write, FileShare.ReadWrite)))
{
// Here are the interpretations of the code
while ((dataSize = streamReader.ReadBlock(buffer, 0, BufferSize)) > 0)
{
streamWriter.Write(.....);
}
}
Can anyone help me??
Thanks
You need to move the writing into a Background thread if you want to avoid blocking the UI.
This can be done via Task.Factory.StartNew:
var task = Task.Factory.StartNew( () =>
{
using (StreamReader streamReader //.. Your code
});
This will, by default, cause this to run on a ThreadPool thread. If you need to update your user interface when this completes, you can use a continuation on the UI thread:
task.ContinueWith(t =>
{
// Update UI here
}, TaskScheduler.FromCurrentSynchronizationContext());
You need to understand, that even with BeginInvoke your code is executed on the SAME UI dispatcher thread, thus freezing your GUI. Try using tasks to run your logic in background.
I am downloading a file and want to execute the install only after the download is complete. How do I accomplish this? Seems like FileSystemWatcher onCreate event would do this but this happens in a different thread, is there a simple way to force the waiting part to happen in the same thread.
Code I have so far
FileSystemWatcher w = new FileSystemWatcher(#"C:/downloads");
w.EnableRaisingEvents = true;
w.Created += new FileSystemEventHandler(FileDownloaded);
static void FileDownloaded(object source, FileSystemEventArgs e)
{
InstallMSI(e.FullPath);
}
I looked at SynchronizingObject and WaitForChangedResult but didn't get a solid working sample.
Try:
FileInfo fInfo = new FileInfo(e.FullPath);
while(IsFileLocked(fInfo)){
Thread.Sleep(500);
}
InstallMSI(e.FullPath);
static bool IsFileLocked(FileInfo file)
{
FileStream stream = null;
try {
stream = file.Open(FileMode.Open, FileAccess.ReadWrite, FileShare.None);
}
catch (IOException) {
return true;
}
finally {
if (stream != null)
stream.Close();
}
return false;
}
If you insist on using FileSystemWatcher you would probably have to account for the fact that a file of some size isn't created (uploaded) in one single operation. The filesystem is likely to produce 1 created and x changed events before the file is ready for use.
You could catch the created events and create new (dedicated) threads (unless you already have an ongoing thread for that file) in which you loop and periodically try to open the file exclusively. If you succeed, the file is ready.
One technique would be to download to the temporary directory, and then move it into C:/downloads once it was complete.
If you are using WebClient to download, you can use set the client's DownloadFileCompleted eventhandler.
If you do it this way you can also use client.DownloadFileAsync() to make it download asynchronously.