I am trying to read the contents of a directory that has over 100k+ files. Each file will be parsed eventually. Since this is a long time consuming task I want to provide feedback to the end user that the program has not "frozen", This is an MVP pattern project setup.
I am using async Task and IProgress to provide feedback to the UI, but either methods below lock up the UI and I can't figure out why..
private async void MainFormViewOnStartProcessingFiles(object sender, EventArgs eventArgs)
{
// The Progress<T> constructor captures our UI context,
// so the lambda will be run on the UI thread.
var asyncProgress = new Progress<int>(percent =>
{
EventAggregator.Instance.Publish(new DisplayMainFormWaitBarMessage($"Async found {percent} files..."));
});
// GetDirectoryFiles is run in the thread pool
await Task.Run(() => GetDirectoryFilesAsync(asyncProgress));
var syncProgress = new Progress<int>(percent =>
{
EventAggregator.Instance.Publish(new DisplayMainFormWaitBarMessage($"Sync found {percent} files..."));
});
GetDirectoryFiles(syncProgress);
}
This updates the UI
private void DisplayWaitingBarMessage(DisplayMainFormWaitBarMessage obj)
{
_mainFormView.DisplayWaitBarMessage(obj.DisplayText);
}
This is my Async and Non Async code to read the files into a queue
private async Task GetDirectoryFilesAsync(IProgress<int> taskProgress)
{
await Task.Run(() =>
{
try
{
var directoryPath = Directory.GetCurrentDirectory() + #"\..\..\..\FUG_2017\";
var _listOfDirectoryFiles = Directory.GetFiles(directoryPath).Take(100);
var fileCount = 0;
foreach (var filePath in _listOfDirectoryFiles)
{
_filesToProcessQueue.Enqueue(filePath);
fileCount++;
taskProgress?.Report(fileCount);
}
}
catch (Exception exc)
{
FlushAndExit(exc);
}
});
}
private void GetDirectoryFiles(IProgress<int> taskProgress)
{
try
{
var directoryPath = Directory.GetCurrentDirectory() + #"\..\..\..\FUG_2017\";
EventAggregator.Instance.Publish(new DisplayMainFormWaitBarWithMessage(ElementVisibility.Visible, $"Inspecting path {directoryPath}"));
var _listOfDirectoryFiles = Directory.GetFiles(directoryPath).Take(1000);
var fileCount = 0;
foreach (var filePath in _listOfDirectoryFiles)
{
_filesToProcessQueue.Enqueue(filePath);
fileCount++;
EventAggregator.Instance.Publish(new DisplayMainFormWaitBarWithMessage(ElementVisibility.Visible, $"Loaded file {fileCount} of {_listOfDirectoryFiles.Count()} to Queue..."));
taskProgress?.Report(fileCount);
}
EventAggregator.Instance.Publish(new DisplayMainFormWaitBarWithMessage(ElementVisibility.Visible, $"Loaded {_listOfDirectoryFiles.Count()} files to Queue..."));
}
catch (Exception exc)
{
FlushAndExit(exc);
}
}
Related
I am implementing a run() function that can not be async. The goal is to send a get request to a server, and then download some files based on the result of the get request and return the number of files downloaded. I have written an async function to do that but I want to essentially "await" it before the rest of my main function continues. I am unable to achieve this behavior currently as the function just hangs. Not sure why its hanging :(
I think I just need some insights into Task and why this isn't working as expected. I am familiar with promises in JS so I thought this wouldn't be that difficult.
Thank you!
public int run(){
FilesManager test = new FilesManager();
string path = Path.Combine("C:\Users\username\Documents", "Temp");
Task<int> T_count = test.Downloadfiles(path); //TODO: trying to "await" this before the messageBoxes
Task.WaitAll(T_count);
int count = T_count.Result;
MessageBox.Show("File Downloaded");
MessageBox.Show(count.ToString());
}
public async Task<int> Downloadfiles(string path)
{
String[] response = await getClient.GetFromJsonAsync<string[]>("http://localhost:3000");
int counter = 0;
try
{
foreach (string url in response)
{
Uri uri = new Uri(url);
var response2 = await getClient.GetAsync(uri);
using (var fs = new FileStream(
path + counter.ToString(),
FileMode.Create))
{
await response2.Content.CopyToAsync(fs);
}
counter++;
}
return counter;
}catch(Exception e)
{
while (e != null)
{
MessageBox.Show(e.Message);
e = e.InnerException;
}
return 0;
}
}
EDIT:
Still not able to get the task.WaitAll(T_count) to work. With some more debugging, the execution jumps from the response2 = await getClient.GetAsync... straight into the waitAll, never hitting the copyToAsync or counter++.
Sync-over-async is a fundamentally difficult problem, because you need to guarantee that continuations never try to run on the thread you are blocking on, otherwise you will get a deadlock as you have seen. Ideally you would never block on async code, but sometimes that is not possible.
Task.Run(...)..GetAwaiter().GetResult() is normally fine to use for this purpose, although there are still some circumstances when it can deadlock.
Do not call the UI from inside the async function, therefore you must move the catch with MessageBox.Show to the outer function.
You can also make this more efficient, by using HttpCompletionOption.ResponseHeadersRead, and you are missing a using on the response2 object.
public int run()
{
FilesManager test = new FilesManager();
string path = Path.Combine("C:\Users\username\Documents", "Temp");
try
{
int count = Task.Run(() => test.Downloadfiles(path)).GetAwaiter().GetResult();
MessageBox.Show("File Downloaded");
MessageBox.Show(count.ToString());
return count;
}
catch(Exception e)
{
while (e != null)
{
MessageBox.Show(e.Message);
e = e.InnerException;
}
return 0;
}
}
public async Task<int> Downloadfiles(string path)
{
String[] response = await getClient.GetFromJsonAsync<string[]>("http://localhost:3000");
int counter = 0;
try
{
foreach (string url in response)
{
using (var response2 = await getClient.GetAsync(url, HttpCompletionOption.ResponseHeadersRead))
using (var fs = new FileStream(
path + counter.ToString(),
FileMode.Create))
{
await response2.Content.CopyToAsync(fs);
}
counter++;
}
return counter;
}
}
Another option is to remove and afterwards restore the SynchronizationContext, as shown in this answer.
During the UWP conversion of an application that was previously designed as a winform, errors occurred in my logging method and I had to change the System.IO reference to Storage. As a result of this change, when the synchronous method started to work asynchronously, I got errors about the request to use the file at the same time. Then I came up with a simple solution by creating the class I mentioned below but this quick amateurish approach do you think is correct?
public class Logging_UWP
{
private List<string> CompletedLog = new List<string>();
private string machineCode = string.Empty;
public Logging_UWP(string MachineCode)
{
machineCode = MachineCode;
Task.Factory.StartNew(()=> WriteLog());
}
public void WriteLogSync(string NewLog)
{
CompletedLog.Add(NewLog);
}
private async void WriteLog()
{
while (true)
{
try
{
foreach (string NewLog in CompletedLog)
{
StorageFolder storageFolder = ApplicationData.Current.LocalFolder;
StorageFile sampleFile = await storageFolder.CreateFileAsync(machineCode + ".log", CreationCollisionOption.OpenIfExists);
var stream = await sampleFile.OpenAsync(Windows.Storage.FileAccessMode.ReadWrite);
using (var outputStream = stream.GetOutputStreamAt(stream.Size + 1))
{
using (var dataWriter = new Windows.Storage.Streams.DataWriter(outputStream))
{
dataWriter.WriteString(String.Format("\n[{0}-{1}] >>> {2}", DateTime.Now.ToShortDateString(), DateTime.Now.ToLongTimeString(), NewLog));
await dataWriter.StoreAsync();
await outputStream.FlushAsync();
}
}
}
if (CompletedLog.Count > 0)
CompletedLog.Clear();
await Task.Delay(500);
}
catch (Exception)
{ }
}
}
} '''
First of all, sorry because I am so new at C# and I decided to make this question because I have been choked in this for hours.
I have an GUI that works with Google Cloud Speech services and make a Speech-to-Text operation. I share with you the whole method that runs when a button is clicked:
private async Task<object> StreamingMicRecognizeAsync(int seconds)
{
if (NAudio.Wave.WaveIn.DeviceCount < 1)
{
Console.WriteLine("No microphone!");
return -1;
}
GoogleCredential googleCredential;
using (Stream m = new FileStream(#"..\..\credentials.json", FileMode.Open))
googleCredential = GoogleCredential.FromStream(m);
var channel = new Grpc.Core.Channel(SpeechClient.DefaultEndpoint.Host,
googleCredential.ToChannelCredentials());
var speech = SpeechClient.Create(channel);
var streamingCall = speech.StreamingRecognize();
// Write the initial request with the config.
await streamingCall.WriteAsync(
new StreamingRecognizeRequest()
{
StreamingConfig = new StreamingRecognitionConfig()
{
Config = new RecognitionConfig()
{
Encoding =
RecognitionConfig.Types.AudioEncoding.Linear16,
SampleRateHertz = 48000,
LanguageCode = "es-ES",
},
InterimResults = true,
}
});
// Read from the microphone and stream to API.
object writeLock = new object();
bool writeMore = true;
var waveIn = new NAudio.Wave.WaveInEvent();
waveIn.DeviceNumber = 0;
waveIn.WaveFormat = new NAudio.Wave.WaveFormat(48000, 1);
waveIn.DataAvailable +=
(object sender, NAudio.Wave.WaveInEventArgs args) =>
{
lock (writeLock)
{
if (!writeMore) return;
streamingCall.WriteAsync(
new StreamingRecognizeRequest()
{
AudioContent = Google.Protobuf.ByteString
.CopyFrom(args.Buffer, 0, args.BytesRecorded)
}).Wait();
}
};
// Print responses as they arrive.
Task printResponses = Task.Run(async () =>
{
while (await streamingCall.ResponseStream.MoveNext(default(CancellationToken)))
{
foreach (var result in streamingCall.ResponseStream
.Current.Results)
{
foreach (var alternative in result.Alternatives)
{
Console.WriteLine(alternative.Transcript);
//Textbox1.Text = alternative.Transcript;
}
}
}
});
waveIn.StartRecording();
Console.WriteLine("Speak now.");
Result_Tone.Text = "Speak now:\n\n";
await Task.Delay(TimeSpan.FromSeconds(seconds));
// Stop recording and shut down.
waveIn.StopRecording();
lock (writeLock) writeMore = false;
await streamingCall.WriteCompleteAsync();
await printResponses;
return 0;
}
My problem is that I want to update the content of the Textbox1control but it doesn´t work. It writes perfectly the output into the console with the line Console.WriteLine(alternative.Transcript); but not into my textbox.
If someone could help I would appreciate so much his help.
The problem is that you're using Task.Run, which means your code will be running on a thread-pool thread.
Instead of calling Task.Run(), just move that code into a separate async method:
async Task DisplayResponses(IAsyncEnumerator<StreamingRecognizeResponse> responses)
{
while (await responses.MoveNext(default(CancellationToken)))
{
foreach (var result in responses.Current.Results)
{
foreach (var alternative in result.Alternatives)
{
Textbox1.Text = alternative.Transcript;
}
}
}
}
Then call that method directly (without Task.Run) from code that's already on the UI thread (e.g. an event handler).
The async machinery will make sure that after the await expression, you're back on the UI thread (the same synchronization context). So the assignment to the Text property will occur on the UI thread, and all should be well.
For example:
// This would be registered as the event handler for a button
void HandleButtonClick(object sender, EventArgs e)
{
var stream = client.StreamingRecognize();
// Send the initial config request
await stream.WriteAsync(...);
// Presumably you want to send audio data...
StartSendingAudioData(stream);
await DisplayResponses(stream.ResponseStream);
}
Tasks run on seperate threads, so you must Invoke an action that will be performed on the control's thread
Textbox1.Invoke(new Action(() =>
{
Textbox1.Text= "";
}));
Edit: For WPF, I believe the equivalent is
Textbox1.Dispatcher.Invoke(new Action(() =>
{
Textbox1.Text= "";
}));
have you tried using Dispatcher.InvokeASync()?
await Dispatcher.InvokeAsync(() => {while (await streamingCall.ResponseStream.MoveNext(default(CancellationToken)))
{
foreach (var result in streamingCall.ResponseStream
.Current.Results)
{
foreach (var alternative in result.Alternatives)
{
Textbox1.Text = alternative.Transcript;
}
}
}});
Why does saving a file fail so often on UWP?
I open and read the contents of a specified file when the application is initially launched (with the help of a BackgroundWorker). Then when the application is suspending I try to save new content to the same file, however this results almost always in the following exception:
Unable to remove the file to be replaced. (Exception from HRESULT: 0x80070497)
My loading method:
private async static void SymbolLoader_DoWork(object sender, DoWorkEventArgs e)
{
IStorageItem file = null;
if (!UseDefault)
{
var folder = await StorageFolder.GetFolderFromPathAsync(SavePath);
file = await folder.TryGetItemAsync(FileName);
}
if (file == null)
{
file = await Package.Current.InstalledLocation.GetFileAsync(#"Assets\Default.txt");
}
var text = await FileIO.ReadTextAsync((IStorageFile)file);
Data = DoSomething(text);
}
My OnSuspending method:
private async void OnSuspending(object sender, SuspendingEventArgs e)
{
var tries = 0;
var maxTries = 10;
var deferral = e.SuspendingOperation.GetDeferral();
while (tries < maxTries)
{
var completed = await Saver.Save();
if (completed)
break;
else
{
tries++;
await Task.Delay(TimeSpan.FromSeconds(0.1));
}
}
deferral.Complete();
}
Save method:
public static async Task<bool> Save()
{
//some formatting etc.
var folder = await StorageFolder.GetFolderFromPathAsync(SavePath);
var file = await folder.CreateFileAsync(FileName, CreationCollisionOption.ReplaceExisting);
try
{
await FileIO.WriteLinesAsync(file, lines);
return true;
}
catch(Exception e)
{
return false;
}
}
It is critical that the data is saved, which is why I "solved" the problem by trying to save the file multiple times with some delay. However my solution, presented in the OnSuspending method, does not seem like the correct way to do this.
What is the correct approach? Thanks in advance!
I am running a program where a file gets uploaded to a folder in IIS,and then is processed to extract some values from it. I use a WCF service to perform the process, and BackgroundUploader to upload the file to IIS. However, after the upload process is complete, I get the error "The process cannot access the file x because it is being used by another process." Based on similar questions asked here, I gathered that the file concerned needs to be in a using statement. I tried to modify my code to the following, but it didn't work, and I am not sure if it is even right.
namespace App17
{
public sealed partial class MainPage : Page, IDisposable
{
private CancellationTokenSource cts;
public void Dispose()
{
if (cts != null)
{
cts.Dispose();
cts = null;
}
GC.SuppressFinalize(this);
}
public MainPage()
{
this.InitializeComponent();
cts = new CancellationTokenSource();
}
public async void Button_Click(object sender, RoutedEventArgs e)
{
try
{
Uri uri = new Uri(serverAddressField.Text.Trim());
FileOpenPicker picker = new FileOpenPicker();
picker.FileTypeFilter.Add("*");
StorageFile file = await picker.PickSingleFileAsync();
using (var stream = await file.OpenAsync(FileAccessMode.Read))
{
GlobalClass.filecontent = file.Name;
GlobalClass.filepath = file.Path;
BackgroundUploader uploader = new BackgroundUploader();
uploader.SetRequestHeader("Filename", file.Name);
UploadOperation upload = uploader.CreateUpload(uri, file);
await HandleUploadAsync(upload, true);
stream.Dispose();
}
}
catch (Exception ex)
{
string message = ex.ToString();
var dialog = new MessageDialog(message);
await dialog.ShowAsync();
Log(message);
}
}
private void CancelAll(object sender, RoutedEventArgs e)
{
Log("Canceling all active uploads");
cts.Cancel();
cts.Dispose();
cts = new CancellationTokenSource();
}
private async Task HandleUploadAsync(UploadOperation upload, bool start)
{
try
{
Progress<UploadOperation> progressCallback = new Progress<UploadOperation>(UploadProgress);
if (start)
{
await upload.StartAsync().AsTask(cts.Token, progressCallback);
}
else
{
// The upload was already running when the application started, re-attach the progress handler.
await upload.AttachAsync().AsTask(cts.Token, progressCallback);
}
ResponseInformation response = upload.GetResponseInformation();
Log(String.Format("Completed: {0}, Status Code: {1}", upload.Guid, response.StatusCode));
cts.Dispose();
}
catch (TaskCanceledException)
{
Log("Upload cancelled.");
}
catch (Exception ex)
{
string message = ex.ToString();
var dialog = new MessageDialog(message);
await dialog.ShowAsync();
Log(message);
}
}
private void Log(string message)
{
outputField.Text += message + "\r\n";
}
private async void LogStatus(string message)
{
var dialog = new MessageDialog(message);
await dialog.ShowAsync();
Log(message);
}
private void UploadProgress(UploadOperation upload)
{
BackgroundUploadProgress currentProgress = upload.Progress;
MarshalLog(String.Format(CultureInfo.CurrentCulture, "Progress: {0}, Status: {1}", upload.Guid,
currentProgress.Status));
double percentSent = 100;
if (currentProgress.TotalBytesToSend > 0)
{
percentSent = currentProgress.BytesSent * 100 / currentProgress.TotalBytesToSend;
}
MarshalLog(String.Format(CultureInfo.CurrentCulture,
" - Sent bytes: {0} of {1} ({2}%), Received bytes: {3} of {4}", currentProgress.BytesSent,
currentProgress.TotalBytesToSend, percentSent, currentProgress.BytesReceived, currentProgress.TotalBytesToReceive));
if (currentProgress.HasRestarted)
{
MarshalLog(" - Upload restarted");
}
if (currentProgress.HasResponseChanged)
{
MarshalLog(" - Response updated; Header count: " + upload.GetResponseInformation().Headers.Count);
}
}
private void MarshalLog(string value)
{
var ignore = this.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
{
Log(value);
});
}
}
}
After this is done, the file name is sent to a WCF service which will access and process the uploaded file to extract certain values. It is at this point I receive the error. I would truly appreciate some help.
public async void Extract_Button_Click(object sender, RoutedEventArgs e)
{
ServiceReference1.Service1Client MyService = new ServiceReference1.Service1Client();
string filename = GlobalClass.filecontent;
string filepath = #"C:\Users\R\Documents\Visual Studio 2015\Projects\WCF\WCF\Uploads\"+ filename;
bool x = await MyService.ReadECGAsync(filename, filepath);
}
EDIT: Code before I added the using block
try
{
Uri uri = new Uri(serverAddressField.Text.Trim());
FileOpenPicker picker = new FileOpenPicker();
picker.FileTypeFilter.Add("*");
StorageFile file = await picker.PickSingleFileAsync();
GlobalClass.filecontent = file.Name;
GlobalClass.filepath = file.Path;
BackgroundUploader uploader = new BackgroundUploader();
uploader.SetRequestHeader("Filename", file.Name);
UploadOperation upload = uploader.CreateUpload(uri, file);
await HandleUploadAsync(upload, true);
}
When you work with stream writers you actually create a process, which you can close it from task manager. And after stream.Dispose() put stream.Close().
This should solve your problem.
You should also close the stream that writes the file to disk (look at your implementation of CreateUpload).
i got such error in DotNet Core 2 using this code:
await file.CopyToAsync(new FileStream(fullFileName, FileMode.Create));
counter++;
and this is how I managed to get rid of message (The process cannot access the file x because it is being used by another process):
using (FileStream DestinationStream = new FileStream(fullFileName, FileMode.Create))
{
await file.CopyToAsync(DestinationStream);
counter++;
}