I'm trying to use this newbie-handy console "outputer". When I try to apply the loop on it, it either goes nuts producing huge amounts of lines in output file or not working at all if try to use delay. Extremely simplified version of my issue:
C#:
using (StreamWriter writer = new StreamWriter())
{
Console.SetOut(writer);
while (true)
{
Console.WriteLine("1");
}
}
I have tried almost every possible way to make a 1sec delay (timers, delays,actions,sleeps). Every option works fine until I apply the StreamWriter. Using delay inside the loop does nothing - tonns of lines "1" in output file. Using delay outside the loop keeps the output file empty. "Try-Catch" says nothing. Guys, where is the trouble? Maybe StreamWriter is not delay-compatible?
Your file will only be closed and written for sure when the using block ends. Before, it's up to your operating system and disk's caching mechanism to actually write the file.
Your loop never ends. Your file might never get written. See to it that your loop can be terminated normally and the using block can close the file.
You can use Thread.Sleep and specify the timespan you want to sleep inside your loop. You also need to terminate the loop at some point, for example:
using (StreamWriter writer = new StreamWriter("c:\\temp\\temp.txt"))
{
Console.SetOut(writer);
for (int i = 0; i < 10; i++)
{
Thread.Sleep(TimeSpan.FromSeconds(1));
Console.WriteLine("{0}: {1}", DateTime.Now.ToShortTimeString(), i);
}
}
Related
I have a class that takes data from several sources and writes them to a ZIP file. I've benchmarked the class to check if using CompressionLevel.Optimal would be much slower than CompressionLevel.Fastest. But the benchmark throws an exception on different iterations and in different CompressionLevel values each time I run the benchmark.
I started removing the methods that add the file-content step by step until I ended up with the code below (inside the for-loop) which does basically nothing besides creating an empty zip-file and deleting it.
Simplified code:
var o = #"e:\test.zip";
var result = new FileInfo(o);
for (var i = 0; i < 1_000_000; i++)
{
// Alternate approach
// using(var archive = ZipFile.Open(o, ZipArchiveMode.Create))
using (var archive = new ZipArchive(result.OpenWrite(), ZipArchiveMode.Create, false, Encoding.UTF8))
{
}
result.Delete();
}
The loop runs about 100 to 15k iterations on my PC and then throws an IOException when trying to delete the file saying that the file (result) is locked.
So... did I miss something about how to use System.IO.Compression.ZipArchive? There is no close method for ZipArchive and using should dispose/close the archive... I've tried different .NET versions 4.6, 4.6.1, 4.7 and 4.7.2.
EDIT 1:
The result.Delete() is not part of the code that is benchmarked
EDIT 2:
Also tried to play around with Thread.Sleep(5/10/20) after the using block (therefore the result.Delete() to check if the lock persists) but up to 20ms the file is still locked at some point. Didnt tried higher values than 20ms.
EDIT 3:
Can't reprodurce the problem at home. Tried a dozen times at work and the loop never hit 20k iterations. Tried once here and it completed.
EDIT 4:
jdweng (see comments) was right. Thanks! Its somehow related to my "e:" partition on a local hdd. The same code runs fine on my "c:" partition on a local ssd and also on a network share.
In my experience files are may not be consistently unlocked when the dispose method for the stream returns. My best guess is that this is due to the file system doing some operation asynchronously. The best solution I have found is to retry the delete operation multiple times. i.e. something like this:
public static void DeleteRetrying(this FileInfo self, int delayMs = 100, int numberOfAttempts = 3)
{
for (int i = 0; i < numberOfAttempts-1; i++)
{
try
{
self.Delete();
}
catch (IOException)
{
// Consider making the method async and
// replace this with Task.Delay
Thread.Sleep(delayMs);
}
}
// Final attempt, let the exception propagate
self.Delete();
}
This is not an ideal solution, and I would love if someone could provide a better solution. But it might be good enough for testing where the impact of a non deleted file would be manageable.
I've stumbled upon extremely weird error. When using FileStream in the first using - application iterates through the loop and prints out "Done", however, then it exits with the error code 5. Try/Catch doesn't work either.
This seems to be extremely fragile error state because if I fiddle with the file names (for example C:\TFS\file1.xml.No.xml -> C:\TFS\file1.xml.N.xml) then it works fine.
If I use var tw = File.CreateText in the first using then the application exits with the code 1073741845. I've manage to reduce the problem significantly to just few lines of code below for a reproducible example.
Perhaps someone can explain why in the world this would behave so weirdly? I'm also interested in why I am not able to recover from this error state? I've tried [HandleProcessCorruptedStateExceptions] and [SecurityCritical] with no effect.
static void Main(string[] args)
{
var ds = new DataSet();
for (int i = 0; i <= 2; i++)
{
using (var fs = new FileStream(#"C:\TFS\file1.xml.No.xml", FileMode.Create))
{
}
using (var tw = File.CreateText(#"C:\TFS\file1.xml"))
{
ds.WriteXml(tw);
}
Console.WriteLine($"Pass {i} done.");
}
Console.WriteLine("Done");
Console.ReadLine();
}
Using .NET Framework 4.7 Console Application project.
EDIT:
If I put Thread.Sleep(2000) in each using statement - I then encounter this error after the 2nd pass - it prints Pass 1 done. and Pass 2 done. before has exited with code 5 (0x5) so the frequency of writing does not seem to be responsible for this behaviour.
Upon further tinkering with this small sample - I can reproduce the issue without using DataSet at all and just with creating StreamWriter instances. The below example should produce before exiting abruptly:
TW1 created.
TW1 flushed.
TW1 created.
TW2 flushed.
Pass 0 done.
static void Main(string[] args)
{
for (int i = 0; i <= 2; i++)
{
var tw1 = File.CreateText(#"C:\TFS\file1.xml.No.xml");
Console.WriteLine("TW1 created.");
tw1.Flush();
Console.WriteLine("TW1 flushed.");
Thread.Sleep(2000);
var tw2 = File.CreateText(#"C:\TFS\file1.xml");
Console.WriteLine("TW1 created.");
tw2.Flush();
Console.WriteLine("TW2 flushed.");
Thread.Sleep(2000);
Console.WriteLine($"Pass {i} done.");
}
Console.WriteLine("Done");
Console.ReadLine();
}
**EDIT2: **
So it appears for us this issue was caused by Kaspersky Endpoint Security for Windows v11.
The process exit code does not mean that much, you favor seeing the debugger stop to tell you about an unhandled exception. But sure, this isn't healthy. This is an anti-malware induced problem, they don't like XML files. Often a problem on a programmer's machine, they also don't like executable files appearing from seemingly no-where, created by a process that uses process interop like the IDE does to run msbuild. Strong malware signals. So first thing you want to do is temporarily turn it off to see if that solves the problem.
It surely will, next thing you'd do is switching to something a bit less aggressive. The anti-malware solution provided by the OS never gets in the way like that. If you use Avast or anything else that has a "deep scan" mode then uninstall asap.
And worry a bit about what your user might use, getting an IOException from FileStream is quite normal so a try/catch is pretty much required. In general you don't want to overwrite a file or delete a directory that you created milliseconds ago, luckily it is never a sensible thing to do.
I have the following code:
while (condition == true)
{
//First part
using (var stream = File.Create(audioPath))
{
using (WaveFileWriter writer = new WaveFileWriter(stream, waveFormat))
{
writer.Write(audioBytes.ToArray(), 0, audioBytes.ToArray().Length);
}
}
//Second part
using (Process.Start("cmd.exe", commands)) { };
Thread.Sleep(1000);
}
The first part saves a byte array to an audio file, then the second part of my code runs a .cmd file that does some processing on the code. However, this above code returns the error
the process cannot access the file (audioPath) because it is being used by another process.
I have read some other answers and have encountered this problem before but always managed to solve it with a using statement.
Both parts run correcly independently (when the other part is commented out). I am running this on Windows Server 2016 if that has any affect. I have added permissions to the folder/file also and because they both work independently, I doubt it's a permissions issue.
Is it possible that the using statement is not disposing correctly?
Are you really generating files by the same name?
audioPath doesn't seem to change.
while (condition == true)
{
//First part
using (var stream = File.Create(audioPath))
{
using (WaveFileWriter writer = new WaveFileWriter(stream, waveFormat))
{
writer.Write(audioBytes.ToArray(), 0, audioBytes.ToArray().Length);
}
}
//Second part
using (Process.Start("cmd.exe", commands)) { };
Thread.Sleep(1000);
}
Consider the following:
The auditPath is written.
The cmd-command starts using it - you don't wait for it to finish.
Since you don't wait for it to finish, the loop enters the next iteration
and a new auditPath is written while the cmd-command already is using
the "previous" one.
alternatively
Before the cmd-command actually has started (but after the Process.Start() has
already completed), the loop comes to the next iteration and opens a new "version"
of auditPath, writing to it.
The cmd-command finally starts to access the file and you get the seen error.
All in all you have a race condition here. Make sure you wait for Process to finish,
e.g.
using (var proc = Process.Start(...))
{
proc.WaitForExit();
// You might want to check `proc.ExitCode`, etc.
}
before running the next loop-cycle.
Key take-away: Process.Start() is not synchronous. If you need to wait for the launched command to finish, you need to explicitly do it, otherwise it continues to run in the background and might interfere with your other logic - as it currently does.
I don't know if this would help, but you could try getting rid out of one using and use other constructor of WaveFileWriter:
using (WaveFileWriter writer = new WaveFileWriter(fileName, waveFormat))
{
writer.WriteData(testSequence, 0, testSequence.Length);
}
I am been fighting with this problem the last couple of days, it works fine when I am on my dev machine, but on the client it is showing this error.
Now this is the code that I have that seems to be showing the error so any help or guidance would be amazing, thank you in advance.
private void document()
{
StreamWriter sWrite = new StreamWriter("C:\\Demo\\index.html");
//LOTS OF SWRITE LINES HERE
sWrite.Close();
System.Diagnostics.Process.Start("C:\\Demo\\index.html");
}
So I have no idea what it keeps telling me the file is already being used by another process if I run this method twice.
Some of it depends on the exact behavior. This could be for a few reasons: it could be, for example, due to an exception. The following code will produce the exception you've described.
for (int i = 0; i < 10; i++)
{
const string path = #"[path].xml";
try
{
// After the first exception, this call will start throwing
// an exception to the effect that the file is in use
StreamWriter sWrite = new StreamWriter(path, true);
// The first time I run this exception will be raised
throw new Exception();
// Close will never get called and now I'll get an exception saying that the file is still in use
// when I try to open it again. That's because the file lock was never released due to the exception
sWrite.Close();
}
catch (Exception e)
{
}
//LOTS OF SWRITE LINES HERE
Process.Start(path);
}
A "using" block will fix this because it's equivalent to:
try
{
//...
}
finally
{
stream.Dispose();
}
In the context of your code, if you're doing a whole bunch of line writes it actually does make sense to consider if (and when) you want to call Flush at some point. The question is whether the write should be "all or none" - i.e. if an exception occurs, do you want the previous lines to still be written? If not, just use a "using" block - it'll call "Flush" once at the end in the "Dispose." Otherwise, you can call "Flush" earlier. For example:
using (StreamWriter sw = new StreamWriter(...))
{
sw.WriteLine("your content");
// A bunch of writes
// Commit everything we've written so far to disc
// ONLY do this if you could stop writing at this point and have the file be in a valid state.
sw.Flush();
sw.WriteLine("more content");
// More writes
} // Now the using calls Dispose(), which calls Flush() again
A big possible bug is if you're doing this on multiple threads (especially if you're doing a lot of writes). If one thread calls your method and starts writing to the file, and then another thread calls it too and tries to start writing to the file as well, the second thread's call will fail because the first thread's still using the file. If this is the case, you'll need to use some kind of lock to make sure that the threads "take turns" writing to the file.
here is what you can do for example before trying to open the file from the Process.Start
var path = #"C:\Demo\index.html";
using (FileStream fs = new FileStream(path, FileMode.Append, FileAccess.Write))
using (StreamWriter sw = new StreamWriter(fs))
{
sw.WriteLine("Your contents to be written go here");
}
System.Diagnostics.Process.Start(path);
This is a simplified example
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
new Thread(() => Method1()).Start();
new Thread(() => Method2()).Start();
Console.Read();
}
private static void Method1()
{
using (StreamWriter sw = new StreamWriter(#"h:\data.txt"))
{
int i = 100000000;
while (true)
{
Thread.Sleep(100);
sw.WriteLine(i);
i++;
}
}
}
private static void Method2()
{
Thread.Sleep(6000);
throw null;
}
}
}
StreamWriter doesn't write the data into file if exception occurs too early and in another thread. File data.txt is empty at the time when exception occurs.
I played with this situation a little and found a bunch of workarounds:
If I increase the sleep interval for the exception's thread (and decrease interval between writings into file) the file will be filled with data. It is not a choice because I don't know when an exception occurs.
As a consequence of previous workaround I can decrease the buffer size of the stream writer. But it seems not working if I set it too small - for example, this code
FileStream fs = new FileStream(#"h:\data.txt", FileMode.Create);
using (StreamWriter sw = new StreamWriter(fs, Encoding.Default, 10))
doesn't work because the first writing operation occurs only when about 385 integers are waiting in the buffer to be written into file.
File will be filled if I close the writer before exception occurs. But that is not a good choice - I have to write into file from 1 to 10 times per second. It is not a good idea to open and close the writer so frequently, is it?
I can catch the exception like this
private static void Method2()
{
try
{
Thread.Sleep(6000);
throw null;
}
catch
{
Console.WriteLine("Exception!");
}
}
and all will be OK - no application termination and file will be filled pack by pack. But that is not the case also - I can't control when and where exceptions occur. I try to use try-catch everywhere but I can miss something.
So the situation is: StreamWriter's buffer is not full, exception occured in another thread and is not catched, so the application will be terminated. How not to lose this data and to write it into file?
As I understand your situation you are assuming that there is a bug somewhere and the process might be terminated at any time. You want to save as much data as possible.
You should be able to call Flush on the StreamWriter. This will push the data to the OS. If your process terminates the data will eventually be written by the OS.
In case you cannot convince StreamWriter to actually flush for some reason you can use a FileStream and write to that (pseudocode: fileStream.Write(Encoding.GetBytes(myString))). You can then flush the FileStream or use a buffer size of 1.
Of course it's best if you prevent the process from being terminated in the first place. This usually is straight forward with Task as opposed to using raw Threads.
Flushing the stream will ensure that all of it's content is pushed into it's underlying file.
That will take care of ensring all of the data is saved after you completed an operation and that a following exception will not make your application loose data.