We Keep Coding

Accessing file with streamreader failed because it is being used by another process

I have a .NET Core application which is multithreaded. One aspect of the application is a health check which parses a log file for errors. This is the code used to access it:
using StreamReader reader = new StreamReader(GetLogFile);
I noticed that I occasionally get this error:
2021-01-12 11:15:14.890Z ERROR APP=2227 COMP=3789 [16] Health check Check logs for application issues threw an unhandled exception after 96.2407ms - Logger=Microsoft.Extensions.Diagnostics.HealthChecks.DefaultHealthCheckService,Level=ERROR,ThreadId=16,,Exception="System.IO.IOException: The process cannot access the file 'c:\apps\Cb.Publisher\Logs\Cb.Publisher.log' because it is being used by another process.
I changed my code to this:
using StreamReader reader = new StreamReader(File.OpenRead(GetLogFile));
In testing it I haven't encountered the issue but it occurred so rarely that I am not 100% sure it's resolved. Is my change likely to resolve this issue or is there a better way to do it?
Additional Info
This is the entire function:
private int LogLine(Regex reg)
{
GetLogFile = DefaultLogFile.GetLogFileName();
using StreamReader reader = new StreamReader(File.OpenRead(GetLogFile));
string line;
int lineNo = 0;
int errorLine = 0;
while ((line = reader.ReadLine()) != null)
{
Match match = reg.Match(line);
if (match.Success)
{
errorLine = lineNumber;
}
lineNo++;
}
return errorLine;
}
If I set a breakpoint on the while line in Visual Studio and run the function, then try to edit the file in Notepad I fails with the error The process cannot access the file because it is being used by another process.
After some investigation I'm wondering if this line could actually be the cause of my problems:
var fileTarget = (FileTarget)LogManager.Configuration.FindTargetByName("file-target");
It's in DefaultLogFile.GetLogFileName:
public string GetLogFileName()
{
var fileTarget = (FileTarget)LogManager.Configuration.FindTargetByName("file-target");
var logEventInfo = new LogEventInfo();
string fileName = fileTarget.FileName.Render(logEventInfo);
if (!File.Exists(fileName))
{
throw new Exception("Log file does not exist.");
}
return fileName;
}

You currently suggested solution will likely be enough, yes:
using StreamReader reader = new StreamReader(File.OpenRead(GetLogFile));
However, the proper solution is for you to check that the file is not being locked from wherever else you are using the file. This also means your logging framework (be it Log4Net, NLog, Serilog etc.) should be properly configured to not take an exclusive lock on the log file. I believe logging frameworks usually do not lock it from read access by default, so unless you have customized the configuration the logging framework should not be a problem.

C# infinite recursion during resource lookup

I have a problem with this code:
if (_updater.IsNewVersionAvailable())
{
_isolatedStorageFile.CreateDirectory("Folder");
_isolatedStorageFile.CreateDirectory("Folder2");
foreach (string file in Directory.GetFiles(_sharedFilesFolder + "\\Folder"))
{
string fileName = Path.GetFileName(file);
//_isolatedStorageFile.CreateFile(fileName); // <- same problem
using (var outputStream = _isolatedStorageFile.OpenFile("Folder/" + fileName, FileMode.Create, FileAccess.Write)) // <- here is the problem (I tried with backslash (\\) and also doesnt work.
{
using (var inputStream = File.OpenRead(file))
{
inputStream.CopyTo(outputStream);
}
}
}
}
When I run the MS Test which called this piece of code I get this error:
error1
error2
The folders inside isolated storage are created normally ( I cannot create a file)
The strangest thing is that once when I started the test the file has been created - it was 1/20 runs.
Any idea?

One thing you can try is, insert this in your code right before you're getting the infinite recursion issue (from here and here):
try
{
throw new NotImplementedException();
}
catch (NotImplementedException ex)
{
}
I was just trying to figure out an issue where we intended to retrieve something from isolated storage, and it got stuck in this inf recursion. I was browsing the .Net and MSTest sources, and it seems that:
The file doesn't exist. Its trying to throw a FileNotFoundException from FileStream.Init > WinIOError.
To throw the exception, it's trying to get a string with Environment.GetResourceString("IO.FileNotFound_FileName", str), str). From there you get to functions like InternalGetSatelliteAssembly. Its trying to locate mscorlib.
Meanwhile, MSTest has defined an AssemblyResolver listener, which get's called at this point. It will iterate over some paths, doing a File.Exists check on them.
File.Exist will check for permissions to that file. For code access permissions, it'll throw a SecurityException with the parameter: Environment.GetResourceString("Security_Generic").
Loop back to point 2.
(IsolatedStorage never gets to catch the FileNotFoundException, so it won't create a new one.)
The NotImplementedException or AgrumentException seems to force mscorlib to be loaded, and the loop is avoided. Maybe there's a another way of making it easier to find, though.

In my case exactly this happened on some of the machines when my code tried to create a new file in IsolatedStorage. After some research, it appeared to be really a bug and happens when the machine has non-English active locale set. Following code fixed the issue in my case:
var currentCulture = Thread.CurrentThread.CurrentCulture;
var currentUiCulture = Thread.CurrentThread.CurrentUICulture;
Thread.CurrentThread.CurrentCulture = CultureInfo.InvariantCulture;
Thread.CurrentThread.CurrentUICulture = CultureInfo.InvariantCulture;
var traceFileStream = new IsolatedStorageFileStream("system_log.txt", FileMode.OpenOrCreate, FileAccess.Write);
Thread.CurrentThread.CurrentCulture = currentCulture;
Thread.CurrentThread.CurrentUICulture = currentUiCulture;

Capturing output from powershell script

I have been working on converting a GUI script from another language to C# in VS2017 for a customer. With help from the folks here I am 95% of the way there, but have run into a couple of snags; just not sure I am doing things in the best way. I'm including just the relevant portions of code below, please let me know if I am not providing enough:
The majority of the code is centered on the wpf form, which collects data for low level technicians to batch deploy a number of Virtual Machines into the VMware environment. This number could easily range into the dozens or even a hundred VMs at once. The information for each VM is specified in the form, then collected in a listview. Once the listview is fully populated it is exported to a csv. Up to this point everything works just fine.
I've next been working on actually launching the powershell/powerCLI script (also working) and capturing output. The log file is opened with a specific reader application the customer uses, which updates in real time, and the captured output is fed to the log. It is important for the technicians to see the output from the code line by line so they can react if there is an issue.
I started with something like this as a test:
string sPSScript = "C:\\Users\\Admin\\Desktop\\TestC#.ps1";
string logFile = "C:\\Users\\Admin\\Desktop\\My.log";
string logReader = "C:\\Users\\Admin\\Documents\\CMTrace.exe";
string standard_output;
System.Diagnostics.Process PSScript = new System.Diagnostics.Process();
PSScript.StartInfo.FileName =
Environment.GetFolderPath(Environment.SpecialFolder.SystemX86) +
"\\WindowsPowerShell\\v1.0\\powershell.exe";
PSScript.StartInfo.Arguments = "-command . '" + sPSScript + "' " +
vCenter.Text;
PSScript.StartInfo.RedirectStandardOutput = true;
PSScript.StartInfo.UseShellExecute = false;
PSScript.Start();
System.Diagnostics.Process LogFile = new System.Diagnostics.Process();
LogFile.StartInfo.FileName = logReader;
LogFile.StartInfo.Arguments = logFile;
LogFile.Start(); while ((standard_output =
PSScript.StandardOutput.ReadLine()) != null)
{
if (standard_output != "")
{
using (StreamWriter file = new StreamWriter(logFile, append: true))
{
file.WriteLine(standard_output);
}
}
}
While this writes to the log file in real time as expected, it creates 100 instances of the logReader application. I understand why, since I am declaring a new StreamWriter object through every pass, but am unsure how better to go about this.
I tried creating the file outside the loop, like this:
StreamWriter file = new StreamWriter(logFile, append: true) { };
System.Diagnostics.Process LogFile = new System.Diagnostics.Process();
LogFile.StartInfo.FileName = logReader;
LogFile.StartInfo.Arguments = logFile;
System.Diagnostics.Process PSScript = new System.Diagnostics.Process();
PSScript.StartInfo.FileName = Environment.GetFolderPath(Environment.SpecialFolder.SystemX86) + "\\WindowsPowerShell\\v1.0\\powershell.exe";
PSScript.StartInfo.Arguments = "-command . '" + sPSScript + "' " + vCenter.Text;
PSScript.StartInfo.RedirectStandardOutput = true;
PSScript.StartInfo.UseShellExecute = false;
LogFile.Start();
PSScript.Start();
System.Threading.Thread.Sleep(1500);
while ((standard_output = PSScript.StandardOutput.ReadLine()) != null)
{
if (standard_output != "")
{
file.WriteLine(standard_output);
}
}
It doesn't create multiple instances, but it also does not update the log file in real time as the previous code does. It only updates once the script runs, and then only partially. The script produces ~1000 lines of output, and I consistently see only about 840 written to the log file.
I thought about doing something like this:
FileStream logFS;
logFS = new FileStream(logFile, FileMode.Append);
but it appears the only options available to me to write to the file are expecting a byte array.
I am sure that I am missing something stupid simple in this, but would appreciate any suggestions on the easiest way to create the log file, open it in the reader, and then update it with the standard output from the powershell script.

why did the previous code writes in real time?
because you are wrapping it with using. And at the end of using block its gonna call dispose which calls .Flush to write to disk
Your second code block calls WriteLine but never called Flush so it writes to the disk whenever the buffer is full. Just add a .Flush call after WriteLine and you will have real time logging

RemotingConfiguration lock my file, how to use log4net's RemotingAppender?

client.cs
RemotingAppender remotingAppender = new RemotingAppender();
remotingAppender.Sink = "tcp://localhost:15642/LoggingSinkInConsoleDaemon";
remotingAppender.BufferSize = 1;
remotingAppender.ActivateOptions();
BasicConfigurator.Configure(remotingAppender);
log.Info("everything is ok!");
server.cs
LogManager.GetRepository().PluginMap.Add(new Plugin.RemoteLoggingServerPlugin("LoggingSinkInConsoleDaemon"));
client.exe log to server.exe, everything is ok, and after client.exe exit, i use Unlocker.exe(which can be found here) find that client.exe is locked by server.exe(which means i can't delete client.exe yet because it is used by server.exe), i locate the error which is caused by RemotingAppender, but i don't know how to resolve.
i think Remoting handle the RemotingAppender's request, and it have locked client.exe, how can i release the lock?
======================update 1===================================
client.cs
var repo = LogManager.GetRepository();
var app = repo.GetAppenders().Where(x => x.GetType() == typeof(RemotingAppender)).FirstOrDefault();
var remotingAppender = app as RemotingAppender;
var root = ((log4net.Repository.Hierarchy.Hierarchy)LogManager.GetRepository()).Root;
var attachable = root as IAppenderAttachable;
attachable.RemoveAppender(remotingAppender);
i tried above it works, but i lose my log.
i search the source, RemoveAppender remove the logger from list, so it doesn't solve the problem

I'm not sure at all but have you tried remotingAppender.Close(); when you finish with it ?

How can I capture, log and display the console output of any arbitrary process with my own c# app?

I want to run on a C# program a specific running file and during it display the output on the screen and also saving it in the file.
I don't want to save the output in the file and later display it on screen.
I want them both to happen together.
I know a way to do it by "tee" but I failed each time I tried doing so.
Can anyone give me an example (that works) by using "tee"?

The main question is, do you have control over the source code of the program whose output you want logged and displayed?
If so, then you have a couple options. Probably the easiest would be to "hijack" the Console's output stream with a compound TextWriter:
public class CompoundWriter:TextWriter
{
public readonly List<TextWriter> Writers = new List<TextWriter>();
public override void WriteLine(string line)
{
if(Writers.Any())
foreach(var writer in Writers)
writer.WriteLine(line);
}
//override other TextWriter methods as necessary
}
...
//When the program starts, get the default Console output stream
var consoleOut = Console.Out;
//Then replace it with a Compound writer set up with a file writer and the normal Console out.
var compoundWriter = new CompoundWriter();
compoundWriter.Writers.Add(consoleOut);
compoundWriter.Writers.Add(new TextWriter("c:\temp\myLogFile.txt");
Console.SetOut(compoundWriter);
//From now on, any calls to Console's Write methods will go to your CompoundWriter,
//which will send them to the console and the file.
You can also use the Trace listeners to handle any output you want to go to both places:
Trace.Listeners.Clear();
Trace.Listeners.Add(new TextWriterTraceListener(Console.Out));
Trace.Listeners.Add(new TextWriterTraceListener(File.Open("C:\temp\myLogFile.txt");
//replace any call to Console.WriteLine() with Trace.WriteLine()
if you do NOT have control over the source code of the console app you want to "tee", and the console app does not require any input in the middle of its execution, then you can use a named pipe to get the output of the app and redirect it.
var appLocation = #"C:\temp\myApp.exe";
var pipeName = "ConsoleNamedPipe";
using(var namedPipe = new NamedPipeServerStream(pipeName, PipeDirection.In))
{
var info = new ProcessStartInfo
{
FileName = "cmd.exe",
Arguments = String.Format(#"/C {1} >>\\.\pipe\{0}",
pipeName, appLocation),
WindowStyle = ProcessWindowStyle.Hidden
};
ConsoleProcess = Process.Start(info);
pipe.WaitForConnection();
using (var reader = new StreamReader(pipe))
{
while (!reader.EndOfStream)
{
var line = reader.ReadLine();
Console.WriteLine(line);
myFileWriter.WriteLine(line);
}
}
}
This is a very simple example; you can provide more interaction by using a two-way pipe but it will require quite a bit more code on your end.

You can try the following:
C:\you_csharp_program.exe arg1 arg2 arg3 |tee filename

I'm not going to write out specific code examples but I will tell you that you can look at logging frameworks like log4net which has console and file appenders which will do what you want. You cant wrong log statements in your code Log.Debug("some message") setup the log4net config to use any number of appenders you want and have it write the message to all of the sources at once, so for example screen, file, db, and email you all at the same time.
I seem to have missed the last sentence of the question about making it work with Tee so my answer may not be valid.

Elaborating on KeithS's answer, this is a working implementation. It should work for all Write/WriteLine calls without having to override every overload because the current (4.0, and I assume earlier) implementation of TextWriter directs all writes through Write(char).
public class TeeTextWriter : TextWriter
{
readonly TextWriter[] _redirectTo;
public override Encoding Encoding { get { return Encoding.UTF8; } }
public TeeTextWriter(params TextWriter[] redirectTo)
{
_redirectTo = redirectTo ?? new TextWriter[0];
}
public override void Write(char value)
{
foreach (var textWriter in _redirectTo)
{
textWriter.Write(value);
}
}
}
Usage:
var realConsoleStream = Console.Out;
using (var fileOut = new StreamWriter(outFileName, false))
{
Console.SetOut(new TeeTextWriter(fileOut, realConsoleStream));
try
{
Console.WriteLine("Test");
}
finally
{
Console.SetOut(realConsoleStream);
}
}