Greetings stackoverflow members,
in a BackgroundWorker of a WPF Frontend i run sox (open source console sound processing tool) in a System.Diagnostics.Process. In that same way i use several other command line tools and parse their output to poulate progress bars in my frontend.
This works fine for the other tools but not for Sox since instead of spamming new lines for each progress step, it updates a single line on the console by only using carriage returns (\r) and no line feeds (\n). I tried both asynchronous and synchronous reads on process.StandardError.
Using async process.ErrorDataReceived += (sender, args) => FadeAudioOutputHandler(clip, args); in combination with process.BeginErrorReadLine(); doesn't produce any individual status updates because for some reason the carriage returns do not trigger ReadLine, even though the MSDN docs suggest that it should. The output is spit out in one chunk when the process finishes.
I then tried the following code for synchronous char by char reads on the stream:
char[] c;
var line = new StringBuilder();
while (process.StandardError.Peek() > -1)
{
c = new char[1];
process.StandardError.Read(c, 0, c.Length);
if (c[0] == '\r')
{
var percentage = 0;
var regex = new Regex(#"%\s([^\s]+)");
var match = regex.Match(line.ToString());
if (match.Success)
{
myProgressObject.ProgressType = ProgressType.FadingAudio
//... some calculations omitted for brevity
percentage = (int) Math.Round(result);
}
else
{
myProgressObject.ProgressType = ProgressType.UndefinedStep;
}
_backGroundWorker.ReportProgress(percentage, myProgressObject);
line.Clear();
}
else
{
line.Append(c[0]);
}
}
The above code does not seem to read the stream in realtime but will stall output for a while. Then it spams a small chunk and finally deadlocks half-way through the process.
Any hints towards the right direction would be greatly appreciated!
UPDATE with (sloppy?) solution:
This drove me crazy because nothing i tried on the C# side of things seemed to have any effect on the results. My original implementation, before changing it 15 times and introducing new dependencies, was fine.
The problem is with sox and RedirectStandardError alone. I discovered that after grabbing the sox source code and building my own version. First i removed all output of sox entirely except for the stuff i was really interested in and then changing the output to full lines followed by a newline \n . I assumed that this would fix my issues. Well, it didn't. I do not know enough c++ to actually find out why, but they seem to have tempered with how stdio writes to that stream, how it's buffered or do it in such a special way that the streamreader on the c# side is not flushed until the default 4096 byte buffer is full. I confirmed that by padding each line to at least 4096 byte. So in conclusion all i had to do was to manually flush stderr in sox.c after each fprintf(stderr, ...) call in display_status(...):
fflush(stderr);
Though, I'm not sure this is anywhere close to an elegant solution.
Thanks to Erik Dietrich for his answer which made me look at this from a different angle.
The situation you describe is a known problem - for a solution including source code see http://www.codeproject.com/KB/threads/ReadProcessStdoutStderr.aspx
It solves both problems (deadlock and the problem with \n)...
I've had to deal with a similar issue with a bespoke build tool in visual studio. I found that using a regex and doing the parsing in the same thread as the reading is a problem and the output processing grinds to a halt. I ended up with a standard consumer producer solution where you read lines from the output and stick them onto a Queue. Then have the queue be dequeued and processed on some other thread. I can't offer source code but this site has some fantastic resources: http://www.albahari.com/threading/part2.aspx
It's a little kludgy, but perhaps you could pipe the output of the uncooperative process to a process that does nothing but process input by characters, insert line feeds, and write to to standard out... So, in terms of (very) pseudo-code:
StartProcess("sox | littleguythatIwrote")
ReadStandardOutTheWayYouAleadyAre()
Could be that just moves the goalposts (I'm a lot more familiar with std in/out/err in the NIX world), but it's a different way to look at the problem, anyway.
Related
I have a somewhat weird problem. I have a couple of DLLs that I need to use in order to write and read with an NFC reader.
This works:
LV3_InitializeSystem(5);
setAuthCode();
MessageBox.Show(""); // I immediately click and close the box
short ret = LV3_CheckIssuer();
Console.WriteLine(ret); // 0 - Success
This doesn't work:
LV3_InitializeSystem(5);
setAuthCode();
short ret = LV3_CheckIssuer();
Console.WriteLine(ret); // 90 - Card reader can not be detected.
This also doesn't work:
LV3_InitializeSystem(5);
setAuthCode();
Thread.Sleep(5000);
short ret = LV3_CheckIssuer();
Console.WriteLine(ret); // 90 - Card reader can not be detected.
I have no idea what might be the problem. I tried using threads running the initialize part with no success. How does showing a MessageBox enable the initialization to complete but Thread.Sleep() doesn't?
The DLL is apparently posting some required messages on the Windows message queue. In order for the messages to be processed, the message queue must be emptied.
One way of ensuring these messages are processed is to use Application.DoEvents(). Generally Application.DoEvents() is frowned upon - see https://blog.codinghorror.com/is-doevents-evil/ for reasons why this is.
There are other ways to solve this without using Application.DoEvents(), but it would probably require restructuring your code - for example using async/await with a Task.Delay.
I've got a few huge xml files, 1+ gb. I need to do some filtering operations with them. The easiest idea I've come up with is to save them as txt and ReadAllText from them, and start doing some operations like
var a = File.ReadAllText("file path");
a = a.Replace("<", "\r\n<");
The moment I try to do that, however, the program crashes out of memory. I've looked at my task manager while I run it and the RAM usage climbs to 50% and the moment it reaches it the program dies.
Does anyone have any ideas on how I operate with this file avoiding the OutOfMemory exception or allow the program to pull on more of the memory.
If you can do it line by line, instead of saying "Read everything to memory" with File.ReadAllText, you can say "Yield me one line at time" with File.ReadLines.
This will return IEnumerable which uses deferred execution. You can do it like this:
using(StreamWriter sw = new StreamWriter(newFilePath))
foreach(var line in File.ReadLines(path))
{
sw.WriteLine(line.Replace("<", "\r\n<"));
sw.Flush();
}
If you want to learn more about deferred execution, you can check this github page.
I'm writing a WinForm application to control an encoder engine via serial port. The protocol is quite simple, I send the 1st command to ask engine moving, and the 2nd command to confirm it's been in new position, and so on for other places. Here is my code for this:
string dataRead ="";
serialPort1.Write("P.1=2950\r\n"); //Location register (2950)
serialPort1.Write("^.1\r\n"); //Moving command
while (dataRead.Contains("Px.1=2950") == false)
{
serialPort1.Write("?96.1\r\n"); //Ask for current location
respond3 = serialPort1.ReadExisting();
dataRead = string.Concat(dataRead, respond3);
}
//Keep moving to 710 after stop at 2950
serialPort1.Write("P.1=710\r\n"); //Location register (710)
serialPort1.Write("^.1\r\n"); //Moving command
The problem is, when I debug my app, it's stuck in the while-loop. But if I break all, and then continue again, it will pass. The respond3 is used to get output from the engine. Whenever it gets a correct respond, the while-loop will finish.
The problem with this code is how ReadExisting works. The important part is:
Reads all immediately available bytes, based on the encoding, in both the stream and the input buffer of the SerialPort object.
Emphasis mine.
When you're debugging, the IO port has more time to read data, so you'll get a long string in your input like so:
"?96.1\r\n?96.1\r\n?96.1\r\n?96.1\r\nPx.1=2950\r\n"
However, when you're not debugging the code runs faster, and you will get: "?" followed by "9" etc.
As such the immediately available string will never match the string "Px.1=2950" and your loop will never stop.
What you probably want to use, given the newline delimited string you are using, is the ReadLine method.
Im using Stockfish game engine to power Human Vs Computer games.
Here is first part of the code:
Process _proc= new Process();
_proc.StartInfo = new ProcessStartInfo(path);
_proc.StartInfo.RedirectStandardInput = true;
_proc.StartInfo.RedirectStandardOutput = true;
_proc.StartInfo.UseShellExecute = false;
_proc.StartInfo.CreateNoWindow = true;
_proc.Start();
_proc.StandardInput.WriteLine("uci");
_proc.StandardInput.WriteLine("ucinewgame");
At this point everything is ok, but when I try to read StandardOutput something weird happens.
string result = _proc.StandardOutput.ReadToEnd();
Stockfish.exe program pops-up my application is running but code after that line is not executing. When I press pause, it points at this line:
If I use:
while (!_proc.StandardOutput.EndOfStream)
{
result += _proc.StandardOutput.ReadLine();
}
Same thing happens only at while statement. result has its full value there, all the text is written into it.
Is there any way to overcome this without async reading?
Side problem:
Since this is all part of singleton class that is used over whole ASP.NET application, i dont feel like using async reading since Im not sure how can I protect (with locking) multiple threads writing into it. Also, I dont know how to stop current thread since the processing of command can last up to 10 sec.
I don't feel like using Thread.Sleep() to constantly check for end of reading output, not elegant.
Considering side problem, how could i avoid multithread problems if async is only solution?
My threading knowledge is weak, so please have that in mind when giving thread related answers. Thank you.
The call to StandardOutput.ReadToEnd will block until this process ends. Is the goal here to read, process, and respond to various text commands from the process you spawn as you receive them?
You must approach this via asynchronous reading.
For example, you could setup a listener to Process.OutputDataReceived. Then call Process.BeginOutputReadLine to start reading. Your code will continue execution. Meanwhile, the .NET Framework will handle incoming text messages on a separate thread.
I was wondering what the fastest way to send keystrokes using C# is. Currently I am using SendKeys.Send() and SendKeys.SendWait() with SendKeys.Flush().
I am using the following code to calculate the time of how long it takes for the both of them to work:
Stopwatch sw1 = new Stopwatch();
sw1.Start();
for (int a = 1; a <= 1000; a++)
{
SendKeys.Send("a");
SendKeys.Send("{ENTER}");
}
sw1.Stop();
And:
Stopwatch sw2 = new Stopwatch();
sw2.Start();
for (int b = 1; b <= 1000; b++)
{
SendKeys.SendWait("b");
SendKeys.SendWait("{ENTER}");
SendKeys.Flush();
}
sw2.Stop();
The results of the 2 are:
Result 1: 40119 milliseconds
Result 2: 41882 milliseconds
Now if we put the SendKeys.Flush() on the second test, outside of the loop we get:
Result 3: 46278 milliseconds
I was wondering why these changes in the code make the speed very different.
I was also wondering if there is a faster way of sending many keystrokes, as my application does it a lot. (These tests were done on a really slow netbook)
Thanks!
SendWait() is slower because it waits that the message has been processed by the target application. The Send() function instead doesn't wait and returns as soon as possible. If the application is somehow busy the difference can be even much more evident.
If you call Flush() you'll stop your application to process all events related to the keyboard that are queued in the message queue. It doesn't make too much sense if you sent them using SendWait() and you'll slow down a lot the application because it's inside the loop (imagine Flush() as a selective DoEvents() - yes with all its drawbacks - and it's called by SendWait() itself too).
If you're interested about its performance (but they'll always be limited to the speed at which your application can process the messages) please read this on MSDN. In sum, you can change the SendKeys class to use the SendInput function, rather than a journal hook. As quick reference, simply add this setting to your app.config file:
<appSettings>
<add key="SendKeys" value="SendInput"/>
</appSettings>
Anyway, the goal of the new implementation isn't the speed but consistent behavior across different versions of Windows and and options (the increased performance is kind of a side effect, I guess).
If you have a lot of text to push to a client, you may notice that SendKeys is really sluggish. You can vastly speed things up by using the clipboard. The idea is to put the text you wish to "type" into a target text box in the clipboard and then send a CTRL-V to the target application to paste that text. Here's an illustration:
Clipboard.Clear(); // Always clear the clipboard first
Clipboard.SetText(TextToSend);
SendKeys.SendWait("^v"); // Paste
I found this worked well for me with a cordless bar code scanner that talks via WiFi to a host app that sends long bar codes to a web app running in Google Chrome. It went from tediously pecking out 30 digits over about 4 seconds to instantly pasting all in under a second.
One obvious downside is that this can mess with your user's use of the clipboard. Another is that it doesn't help if you intend to send control codes like TAB or F5 instead of just plain old text.