I've the following code, that from the first Stream is reading from a file, and doing some interpretations to the content and write them to the second file, I'm facing a problem that when I've a big file the GUI in WPF is sticking, I tried to put the reading and the writing actions in:
Application.Current.Dispatcher.BeginInvoke(DispatcherPriority.Normal, new Action(() =>
{
// Here
});
This in the follwoing code:
using (StreamReader streamReader = new StreamReader(File.Open(fileName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)))
using (StreamWriter streamWriter = new StreamWriter(File.Open("Compressed_" + splitFilePath[splitFilePath.Length - 1], FileMode.Create, FileAccess.Write, FileShare.ReadWrite)))
{
// Here are the interpretations of the code
while ((dataSize = streamReader.ReadBlock(buffer, 0, BufferSize)) > 0)
{
streamWriter.Write(.....);
}
}
Can anyone help me??
Thanks
You need to move the writing into a Background thread if you want to avoid blocking the UI.
This can be done via Task.Factory.StartNew:
var task = Task.Factory.StartNew( () =>
{
using (StreamReader streamReader //.. Your code
});
This will, by default, cause this to run on a ThreadPool thread. If you need to update your user interface when this completes, you can use a continuation on the UI thread:
task.ContinueWith(t =>
{
// Update UI here
}, TaskScheduler.FromCurrentSynchronizationContext());
You need to understand, that even with BeginInvoke your code is executed on the SAME UI dispatcher thread, thus freezing your GUI. Try using tasks to run your logic in background.
Related
I'm writing a method which asynchronously writes separate lines of text to a file. If it's cancelled it deletes the created file and jumps out of the loop.
This is the simplified code which works fine... And I marked 2 points which I'm not sure how they are being handled. I want the code to not block the thread in any case.
public async Task<IErrorResult> WriteToFileAsync(string filePath,
CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
using var stream = new FileStream(filePath, FileMode.Create);
using var writer = new StreamWriter(stream, Encoding.UTF8);
foreach (var line in Lines)
{
if (cancellationToken.IsCancellationRequested)
{
//
// [1] close, delete and throw if cancelled
//
writer.Close();
stream.Close();
if (File.Exists(filePath))
File.Delete(filePath);
throw new OperationCanceledException();
}
// write to the stream
await writer.WriteLineAsync(line.ToString());
}
//
// [2] flush and let them dispose
//
await writer.FlushAsync();
await stream.FlushAsync();
// await stream.DisposeAsync(); ??????
return null;
}
1
I'm calling Close() on FileStream and StreamWriter and I think it will run synchronously and blocks the thread. How can I improve this? I don't want to wait for it to flush the buffer into the file and then delete the file.
2
I suppose the Dispose method will be called and not DisposeAsync at the end of the using scope. (is this assumption correct?).
So Dispose blocks the thread and in order to prevent that I'm flushing first with FlushAsync so that Dispose would perform less things. (to what extent is this true?)
I could also remove using and instead I could write DisposeAsync manually in these two places. But it will decrease readability.
If I open the FileStream with useAsync = true would it automatically call DisposeAsync when using block ends?
Any explanation or a variation of the above code which performs better is appreciated.
As you have it, the using statement will call Dispose(), not DisposeAsync().
C# 8 brought a new await using syntax, but for some reason it's not mentioned in the What's new in C# 8.0 article.
But it's mentioned elsewhere.
await using var stream = new FileStream(filePath, FileMode.Create);
await using var writer = new StreamWriter(stream, Encoding.UTF8);
But also note that this will only work if:
You're using .NET Core 3.0+ since that's when IAsyncDisposable was introduced, or
Install the Microsoft.Bcl.AsyncInterfaces NuGet package. Although this only adds the interfaces and doesn't include the versions of the Stream types (FileStream, StreamWriter, etc.) that use it.
Even in the Announcing .NET Core 3.0 article, IAsyncDisposable is only mentioned in passing and never expanded on.
On another note, you don't need to do this (I see why now):
writer.Close();
stream.Close();
Since the documentation for Close says:
This method calls Dispose, specifying true to release all resources. You do not have to specifically call the Close method. Instead, ensure that every Stream object is properly disposed.
Since you're using using, Dispose() (or DisposeAsync()) will be called automatically and Close won't do anything that's not already happening.
So if you do need to specifically close the file, but want to do it asynchronously, just call DisposeAsync() instead. It does the same thing.
await writer.DisposeAsync();
public async Task<IErrorResult> WriteToFileAsync(string filePath,
CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
await using var stream = new FileStream(filePath, FileMode.Create);
await using var writer = new StreamWriter(stream, Encoding.UTF8);
foreach (var line in Lines)
{
if (cancellationToken.IsCancellationRequested)
{
// not possible to discard, FlushAsync is covered in DisposeAsync
await writer.DisposeAsync(); // use DisposeAsync instead of Close to not block
if (File.Exists(filePath))
File.Delete(filePath);
throw new OperationCanceledException();
}
// write to the stream
await writer.WriteLineAsync(line.ToString());
}
// FlushAsync is covered in DisposeAsync
return null;
}
I want to write to StreamWriter line async but I don't want to await on this.
for(int i= 0 ;i<1000;i++)
{
sw.WriteLineAsync(i.ToString());
}
But i got an error that i invoke to WriteLineAsync in same time.
What can I do to fix that?
I want to close this StreamWriter after this loop.
How can I verify that I now close without to write all data on StreamWriter, Or when I close the stream all the data that sent with WriteLineAsync will be write before the stream close?
If you want to write the lines using async, you should use await. Because, the file might be corrupted and you might encounter with possible stream errors like "the stream is already in use". In short, you should synchronize the write action.
So, I provided an example;
private async Task WriteToFileAsAsync()
{
string file = #"sample.txt";
using (FileStream stream = new FileStream(file, FileMode.Create, FileAccess.ReadWrite))
{
using (StreamWriter streamWriter = new StreamWriter(stream))
{
for (int i = 0; i < 1000; i++)
{
await streamWriter.WriteLineAsync(i.ToString());
}
}
}
}
Also, you can close and dispose the StreamWriter within using blocks.
EDIT
If you want to perform write action in separately from main thread, don't use async methods and just create a seperate Task and assign it an another thread.
private void WriteToFile()
{
string file = #"sample.txt";
using (FileStream stream = new FileStream(file, FileMode.Create, FileAccess.ReadWrite))
{
using (StreamWriter streamWriter = new StreamWriter(stream))
{
for (int i = 0; i < 1000; i++)
{
streamWriter.WriteLine(i.ToString());
}
}
}
}
Then call like this;
Task.Factory.StartNew(WriteToFile);
In your particular context there are two main issues in doing what you would like to do.
Technically speaking, if you call an async method, you need to await for it sooner or later, hence you can collect the task, and await for it later on. However, the WriteLineAsync method is not atomic, therefore calling it and performing other operations on the stream can corrupt the stream itself.
If you don't want to await, then don't call the Async method..
If you want to close a stream after using, use using with it.
using (StreamWriter writer = new StreamWriter("temp.txt"))
{
for (int i = 0; i < 1000; i++)
{
writer.WriteLine(i.ToString());
}
}
If you want it to be asynchronous, then you have to call the Async version, otherwise it will corrupt the file.
I am writing a small logger and I want to open the log file once, keep writing reactively as log messages arrive, and dispose of everything on program termination.
I am not sure how I can keep the FileStream open and reactively write the messages as they arrive.
I would like to update the design from my previous solution where I had a ConcurrentQueue acting as a buffer, and a loop inside the using statements that consumed the queue.
Specifically, I want to simultaneously take advantage of the using statement construct, so I don't have to explicitly close the stream and writer, and of the reactive, loopless programming style. Currently I only know how to use one of these constructs at once: either the using/loop combination, or the explicit-stream-close/reactive combination.
Here's my code:
BufferBlock<LogEntry> _buffer = new BufferBlock<LogEntry>();
// CONSTRUCTOR
public DefaultLogger(string folder)
{
var filePath = Path.Combine(folder, $"{DateTime.Now.ToString("yyyy.MM.dd")}.log");
_cancellation = new CancellationTokenSource();
var observable = _buffer.AsObservable();
using (var stream = File.Create(_filePath))
using (var writer = new StreamWriter(stream))
using (var subscription = observable.Subscribe(entry =>
writer.Write(GetFormattedString(entry))))
{
while (!_cancellation.IsCancellationRequested)
{
// what do I do here?
}
}
}
You need to use Observable.Using. It's designed to create an IDisposble resource that gets disposed when the sequence ends.
Try something like this:
IDisposable subscription =
Observable.Using(() => File.Create(_filePath),
stream => Observable.Using(() => new StreamWriter(stream),
writer => _buffer.AsObservable().Select(entry => new { entry, writer })))
.Subscribe(x => x.writer.Write(GetFormattedString(x.entry)));
I want to write a method to open (or create if it does not exist) a file from different threads.
What would be the FileAccess and FileShare flags in this case? I tried both FileAccess.Read/Write and FileShare.Read/Write but don't see any differences. I used the following code to test, looks fine but not sure about the flags (last 2).
Can anybody clarify should I use FileAccess.ReadWrite or FileAccess.Read and FileShare.ReadWrite or FileShare.Read?
class Program
{
static void Main(string[] args)
{
Task first = Task.Run(() => AccessFile());
Task second = Task.Run(() => AccessFile());
Task third = Task.Run(() => AccessFile());
Task fourth = Task.Run(() => AccessFile());
Task.WaitAll(first, second, third, fourth);
Task[] tasks = new Task[100];
for (int i = 0; i < 100; i++)
{
tasks[i] = Task.Run(() => AccessFile());
}
Task.WaitAll(tasks);
}
public static void AccessFile()
{
string path = #"c:\temp\test.txt";
// FileShare.Write gives access violation
using (FileStream fs = File.Open(path, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite))
{
byte[] by = new byte[100];
fs.Read(by, 0, 100);
Console.WriteLine(Encoding.ASCII.GetString(by));
}
}
}
If multiple threads need to write to the file then you need an exclusive lock on it. If you could have some threads read and others write to the file you could use a ReaderWriterLockSlim.
The FileShare parameter is used to indicate how other threads/processes can access the file while a handle is being opened in the current thread. ReadWrite means that other threads will be able to read and write from this file which obviously means that if you try to write to the file from the current handle you will probably corrupt the contents.
Since you are only reading you should specify just that (FileAccess.Read). And you are OK with others reading so say that (FileShare.Read).
And here's nicer test code:
ParallelEnumerable.Range(0, 100000).ForAll(_ => AccessFile());
I have a Task that reads strings from a blocking collection and is supposed to write them out to a file. Trouble is, while the file is created, the size of the file is 0 bytes after the task completes.
While debugging, I see that non-empty lines are retrieved from the blocking collection, and the stream writer is wrapped in a using block.
For debugging I threw in a flush that should not be required and write the lines to the console. There are 100 non-empty lines of text read from the blocking collection.
// Stuff is placed in writeQueue from a different task
BlockingCollection<string> writeQueue = new BlockingCollection<string>();
Task writer = Task.Factory.StartNew(() =>
{
try
{
while (true)
{
using (FileStream fsOut = new FileStream(destinationPath, FileMode.Create, FileAccess.Write))
using (BufferedStream bsOut = new BufferedStream(fsOut))
using (StreamWriter sw = new StreamWriter(bsOut))
{
string line = writeQueue.Take();
Console.WriteLine(line); // Stuff is written to the console
sw.WriteLine(line);
sw.Flush(); // Just in case, makes no difference
}
}
}
catch (InvalidOperationException)
{
// We're done.
}
});
Stepping through in the debugger, I see that the program terminates in an orderly manner. There are no unhandled exceptions.
What might be going wrong here?
You are re-creating the file on every run of the loop. Change the FileMode.Create to FileMode.Append and it will keep the previous values you wrote on it.
Also, using exceptions to detect that you should stop is a really bad practice, if this a consumer-producer solution, you can easily do better by having the producer setting a thread safe flag variable signaling it has finished the work and will not produce anything else.