We Keep Coding

System.IO.Compression.ZipArchive keeps file locked after dispose?

I have a class that takes data from several sources and writes them to a ZIP file. I've benchmarked the class to check if using CompressionLevel.Optimal would be much slower than CompressionLevel.Fastest. But the benchmark throws an exception on different iterations and in different CompressionLevel values each time I run the benchmark.
I started removing the methods that add the file-content step by step until I ended up with the code below (inside the for-loop) which does basically nothing besides creating an empty zip-file and deleting it.
Simplified code:
var o = #"e:\test.zip";
var result = new FileInfo(o);
for (var i = 0; i < 1_000_000; i++)
{
// Alternate approach
// using(var archive = ZipFile.Open(o, ZipArchiveMode.Create))
using (var archive = new ZipArchive(result.OpenWrite(), ZipArchiveMode.Create, false, Encoding.UTF8))
{
}
result.Delete();
}
The loop runs about 100 to 15k iterations on my PC and then throws an IOException when trying to delete the file saying that the file (result) is locked.
So... did I miss something about how to use System.IO.Compression.ZipArchive? There is no close method for ZipArchive and using should dispose/close the archive... I've tried different .NET versions 4.6, 4.6.1, 4.7 and 4.7.2.
EDIT 1:
The result.Delete() is not part of the code that is benchmarked
EDIT 2:
Also tried to play around with Thread.Sleep(5/10/20) after the using block (therefore the result.Delete() to check if the lock persists) but up to 20ms the file is still locked at some point. Didnt tried higher values than 20ms.
EDIT 3:
Can't reprodurce the problem at home. Tried a dozen times at work and the loop never hit 20k iterations. Tried once here and it completed.
EDIT 4:
jdweng (see comments) was right. Thanks! Its somehow related to my "e:" partition on a local hdd. The same code runs fine on my "c:" partition on a local ssd and also on a network share.

In my experience files are may not be consistently unlocked when the dispose method for the stream returns. My best guess is that this is due to the file system doing some operation asynchronously. The best solution I have found is to retry the delete operation multiple times. i.e. something like this:
public static void DeleteRetrying(this FileInfo self, int delayMs = 100, int numberOfAttempts = 3)
{
for (int i = 0; i < numberOfAttempts-1; i++)
{
try
{
self.Delete();
}
catch (IOException)
{
// Consider making the method async and
// replace this with Task.Delay
Thread.Sleep(delayMs);
}
}
// Final attempt, let the exception propagate
self.Delete();
}
This is not an ideal solution, and I would love if someone could provide a better solution. But it might be good enough for testing where the impact of a non deleted file would be manageable.

Creating StreamWriter instances crashes the application with RPC error

I've stumbled upon extremely weird error. When using FileStream in the first using - application iterates through the loop and prints out "Done", however, then it exits with the error code 5. Try/Catch doesn't work either.
This seems to be extremely fragile error state because if I fiddle with the file names (for example C:\TFS\file1.xml.No.xml -> C:\TFS\file1.xml.N.xml) then it works fine.
If I use var tw = File.CreateText in the first using then the application exits with the code 1073741845. I've manage to reduce the problem significantly to just few lines of code below for a reproducible example.
Perhaps someone can explain why in the world this would behave so weirdly? I'm also interested in why I am not able to recover from this error state? I've tried [HandleProcessCorruptedStateExceptions] and [SecurityCritical] with no effect.
static void Main(string[] args)
{
var ds = new DataSet();
for (int i = 0; i <= 2; i++)
{
using (var fs = new FileStream(#"C:\TFS\file1.xml.No.xml", FileMode.Create))
{
}
using (var tw = File.CreateText(#"C:\TFS\file1.xml"))
{
ds.WriteXml(tw);
}
Console.WriteLine($"Pass {i} done.");
}
Console.WriteLine("Done");
Console.ReadLine();
}
Using .NET Framework 4.7 Console Application project.
EDIT:
If I put Thread.Sleep(2000) in each using statement - I then encounter this error after the 2nd pass - it prints Pass 1 done. and Pass 2 done. before has exited with code 5 (0x5) so the frequency of writing does not seem to be responsible for this behaviour.
Upon further tinkering with this small sample - I can reproduce the issue without using DataSet at all and just with creating StreamWriter instances. The below example should produce before exiting abruptly:
TW1 created.
TW1 flushed.
TW1 created.
TW2 flushed.
Pass 0 done.
static void Main(string[] args)
{
for (int i = 0; i <= 2; i++)
{
var tw1 = File.CreateText(#"C:\TFS\file1.xml.No.xml");
Console.WriteLine("TW1 created.");
tw1.Flush();
Console.WriteLine("TW1 flushed.");
Thread.Sleep(2000);
var tw2 = File.CreateText(#"C:\TFS\file1.xml");
Console.WriteLine("TW1 created.");
tw2.Flush();
Console.WriteLine("TW2 flushed.");
Thread.Sleep(2000);
Console.WriteLine($"Pass {i} done.");
}
Console.WriteLine("Done");
Console.ReadLine();
}
**EDIT2: **
So it appears for us this issue was caused by Kaspersky Endpoint Security for Windows v11.

The process exit code does not mean that much, you favor seeing the debugger stop to tell you about an unhandled exception. But sure, this isn't healthy. This is an anti-malware induced problem, they don't like XML files. Often a problem on a programmer's machine, they also don't like executable files appearing from seemingly no-where, created by a process that uses process interop like the IDE does to run msbuild. Strong malware signals. So first thing you want to do is temporarily turn it off to see if that solves the problem.
It surely will, next thing you'd do is switching to something a bit less aggressive. The anti-malware solution provided by the OS never gets in the way like that. If you use Avast or anything else that has a "deep scan" mode then uninstall asap.
And worry a bit about what your user might use, getting an IOException from FileStream is quite normal so a try/catch is pretty much required. In general you don't want to overwrite a file or delete a directory that you created milliseconds ago, luckily it is never a sensible thing to do.

SshStream.Expect() breaks code when SshStream is no longer valid

Im trying to do automation over SSH but the server is faulty and does not implement exec channel properly so I ended up doing a workaround using CreateShellStream(). I can expect that upon running the program a connection wouldnt be available and disconnections are a thing. My solution:
while(!_cancellationToken.IsCancellationRequested))
{
ShellStream stream = null;
while(stream == null)
{
stream = await GetSshStream();
}
while(stream.CanWrite && stream.CanRead)
{
stream.WriteLine(command);
//this breaks everything if stream is not valid
var rep = stream.Expect(new Regex(#"[$>]"));
var delimiters = new[] { " ", "\r\n", "\t" };
var values = rep.Split(delimiters, StringSplitOptions.RemoveEmptyEntries);
DealWithValues(values);
}
}
This works and waits for connection and once connected starts coms. The problem arises with that stream.CanWrite && stream.CanRead is not enough to detect that stream is healthy and once connection is lost and stream becomes invalid and used with Expect(); everything breaks. Jumps out of all loops, goes through try {} catch{} and even makes Visual Studio debugger steping break down and continue the program in another thread (multi threaded program). Is there a way to stop this from happening and throwing execution back to first while? I could possibly create a new stream every time I need access to the server but since Im polling parameters about once a second I wouldnt want to have the overhead of reconnecting each time.

I misundertood that thread dissapearing ment code execution stopped. Instead thread sleeps waiting for input. Expect has a timeout overload and I should of been using it.

C# The process cannot access file 'XYZ' because it is being used by another process

I am been fighting with this problem the last couple of days, it works fine when I am on my dev machine, but on the client it is showing this error.
Now this is the code that I have that seems to be showing the error so any help or guidance would be amazing, thank you in advance.
private void document()
{
StreamWriter sWrite = new StreamWriter("C:\\Demo\\index.html");
//LOTS OF SWRITE LINES HERE
sWrite.Close();
System.Diagnostics.Process.Start("C:\\Demo\\index.html");
}
So I have no idea what it keeps telling me the file is already being used by another process if I run this method twice.

Some of it depends on the exact behavior. This could be for a few reasons: it could be, for example, due to an exception. The following code will produce the exception you've described.
for (int i = 0; i < 10; i++)
{
const string path = #"[path].xml";
try
{
// After the first exception, this call will start throwing
// an exception to the effect that the file is in use
StreamWriter sWrite = new StreamWriter(path, true);
// The first time I run this exception will be raised
throw new Exception();
// Close will never get called and now I'll get an exception saying that the file is still in use
// when I try to open it again. That's because the file lock was never released due to the exception
sWrite.Close();
}
catch (Exception e)
{
}
//LOTS OF SWRITE LINES HERE
Process.Start(path);
}
A "using" block will fix this because it's equivalent to:
try
{
//...
}
finally
{
stream.Dispose();
}
In the context of your code, if you're doing a whole bunch of line writes it actually does make sense to consider if (and when) you want to call Flush at some point. The question is whether the write should be "all or none" - i.e. if an exception occurs, do you want the previous lines to still be written? If not, just use a "using" block - it'll call "Flush" once at the end in the "Dispose." Otherwise, you can call "Flush" earlier. For example:
using (StreamWriter sw = new StreamWriter(...))
{
sw.WriteLine("your content");
// A bunch of writes
// Commit everything we've written so far to disc
// ONLY do this if you could stop writing at this point and have the file be in a valid state.
sw.Flush();
sw.WriteLine("more content");
// More writes
} // Now the using calls Dispose(), which calls Flush() again
A big possible bug is if you're doing this on multiple threads (especially if you're doing a lot of writes). If one thread calls your method and starts writing to the file, and then another thread calls it too and tries to start writing to the file as well, the second thread's call will fail because the first thread's still using the file. If this is the case, you'll need to use some kind of lock to make sure that the threads "take turns" writing to the file.

here is what you can do for example before trying to open the file from the Process.Start
var path = #"C:\Demo\index.html";
using (FileStream fs = new FileStream(path, FileMode.Append, FileAccess.Write))
using (StreamWriter sw = new StreamWriter(fs))
{
sw.WriteLine("Your contents to be written go here");
}
System.Diagnostics.Process.Start(path);

How to delay the StreamWriter loop?

I'm trying to use this newbie-handy console "outputer". When I try to apply the loop on it, it either goes nuts producing huge amounts of lines in output file or not working at all if try to use delay. Extremely simplified version of my issue:
C#:
using (StreamWriter writer = new StreamWriter())
{
Console.SetOut(writer);
while (true)
{
Console.WriteLine("1");
}
}
I have tried almost every possible way to make a 1sec delay (timers, delays,actions,sleeps). Every option works fine until I apply the StreamWriter. Using delay inside the loop does nothing - tonns of lines "1" in output file. Using delay outside the loop keeps the output file empty. "Try-Catch" says nothing. Guys, where is the trouble? Maybe StreamWriter is not delay-compatible?

Your file will only be closed and written for sure when the using block ends. Before, it's up to your operating system and disk's caching mechanism to actually write the file.
Your loop never ends. Your file might never get written. See to it that your loop can be terminated normally and the using block can close the file.

You can use Thread.Sleep and specify the timespan you want to sleep inside your loop. You also need to terminate the loop at some point, for example:
using (StreamWriter writer = new StreamWriter("c:\\temp\\temp.txt"))
{
Console.SetOut(writer);
for (int i = 0; i < 10; i++)
{
Thread.Sleep(TimeSpan.FromSeconds(1));
Console.WriteLine("{0}: {1}", DateTime.Now.ToShortTimeString(), i);
}
}