Can logging to text file in ASP.net WEb server cause locks
Thi is my code:
var outStream = new FileStream(Server.MapPath("~\\bin\\error.txt"), FileMode.Append, FileAccess.Write, FileShare.ReadWrite);
As you see FileShare is both read and write.
So far I have not seen any issues. Are you aware of possible issues using this method. (I do not want to log error to windows event)
Not 100% sure I understand what you are asking, but any time something writes to a file, the file is locked by that application for the period of time that it takes to complete the write. If there is another application writing to the same file at times, there will be the potential for conflict. If both applications are coded to handle conflicts of this nature, then all will be fine. So if you are coding to handle this situation, you would put the write method in a TRY block within a WHILE block. If an error occurs writing, then it would stay in the while block and pause for a second or something and then try again. Once the write is successful, break out of the while block. You may also want to put in a counter and limit the number of tries.
Just be aware that if you do this that the thread will sit here until the write is successful, so if you don't want this to hold up your application it needs to be done with another thread.
I think you would just want to do a lock on a static object that wraps the File I/O. This static object would ensure the file updates are thread safe.
With respect to Mr. Hinkle, I had always heard one would not want to have multiple try/catch fails inside a while loop as pure design -- unless I'm missing something. I can see where practically this would work, but I always thought exceptions should not become part of the "main flow" in this way.
Related
I'm currently using filestreams to copy files form one location to another.
It all functioned as intended until now when I suddenly have the problemn that File.open freezes the thread that it is running in.
FileStream sourceStream = File.Open(filePath, FileMode.Open)
It only happens for 1 specific file (3 GB in size). The interesting thing is one day prior it functioned normally though for this file so it can't be the file size. Next thing I checked was if some sort of exception was thrown that I don't catch.
I put a try / catch block the whole thing (normally I use the calling method to catch the exceptions) and still same effect.
try
{
FileStream sourceStream = File.Open(filePath, FileMode.Open);
sourceStream.Close();
}
catch (Exception e)
{
Console.Write("A");
}
I also checked what happens if the file is being accessed already. Then an exception is thrown (tested it for other files as like I said for this specific file it always hangs up the thread now when I try to open it).
The file is located on the local harddrive and other files (smaller though) in the same folder don't show this problem.
As I'm now running out of ideas what the possible reason could be, my question is:
What could possible reasons for this unexpected behaviour be and how can they be adverted?
EDIT:
It now functions again (just when I tried to use the process monitor it started functioning again).
So in total no clue what could have caused the phenomenon. If anyone would have an idea what could be a possible reason for this it would be good to know to avoid a possible repeat of the problem in the future.
Also of note as one question brought it up before the File.Open I have an using block with:
using (var stream = new BufferedStream(File.OpenRead(filePath), 1024 * 1024))
{
//..do calculations
}
Which I use to make some hash calculations in regards to the file. THIS one had no issues at all with opening the file (only the later File.Open had the issues)
Edit:
I've just received an info from the sysadmins here that shines a new light onto the problem:
The system is set up in a way so that the whole system is backuped time and again file by file wihtout the OS having any knowledge of it. This means in the case of the backuped file that the OS thinks it is there and nobody accesses it when in reality it is currently being backuped (and thus accessed and unable to be accessed from within the OS according to how they described the backup process.....as the OS doesn't know about the backup happening nothing was shown in the resources hard drive access nor the task manager).
Thus with that information it could be that as the OS didnt know about the file being accessed it tried to access it (through the open command) and waited and waited and waited for the hard drive read head to come to the file which never happened as it was not accessible in reality).
Thus it would have had to run into a timeout which the file.open command doesn't have (at least my guess there with the new infos if I understood the sys admins accurately there)
tnx
A couple possible reasons:
Your antivirus. That thing hooks into the OS and replaces the I/O functions with its own. When you open a file, it can actually perform a virus check before returning back control to your application. You could have had a bad signature update which forced the AV to perform the check on your 3GB file, and a subsequent update could have fixed the problem.
A bad sector on your drive. This usually makes I/O perform very poorly, but your system could have relocated the bad sector into another one, so the performance went back to normal. You can run a chkdsk /R to see if you have bad sectors.
Another app that locks the file, though I'd rather expect an exception in this case.
The Problem stemmed not from c# or the Windows System, but from the architecture of how the PC was set up itself.
In this case it was set up so, that the files I tried to read could be inacessible (because they were being backed up) WITHOUT the OS of the local PC knowing it.
Thus the OS thought the file was accessible and C# received that answer from the OS when it tried to open the file. And as file operations in C# use their Windows aequivalents and those have no timeouts.... the whole Operation hanged / freezed until the file backup was finished.
In retrospect I would say: Lucas Trzesniewski answer should cover most situations where the freeze happens....my own Problem was not answerd by that only because I had such a Special Situation that caused the Problem in the end.
Are you absolutely sure that the freezing always occurs in File.Open()?
Given the absence of exceptions it appears that the problem may be at lower level. When you've experienced it you tried to open the file with a hex editor or some other tool to check that it is actually entirely readable? It could be a problem of access to a certain area of the hard drive.
Try to specify the access mode with FileAccess if you need read-only, write-only, etc.
See also this post for the actual usefulness of BufferedStream.
Have you check with File.Open() function with FileAccess & FileShare values ,
I think it's a file locking issue
I had a similar issue when sometimes File.Open hangs when trying to check if a file is locked.
I solved it like this:
public async Task<bool> IsLocked(FileInfo file)
{
var checkTask = Task.Run(() =>
{
try
{
using (file.Open(FileMode.Open, FileAccess.Read, FileShare.None)) { };
return false;
}
catch (Exception)
{
return true;
}
});
var delayTask = Task.Delay(1000);
var firstTask = await Task.WhenAny(checkTask, delayTask);
if (firstTask == delayTask)
{
return true;
}
else
{
return await checkTask;
}
}
I am trying to implement file based autoincrement identity value (at int value stored in TXT file) and I am trying to come up with the best way to handle concurrency issues. This identity will be used for unique ID for my content. When saving new content this file gets opened, the value gets read, incremented, new content is saved and the incremented value is written back to the file (whether we store the next available ID or the last issued one doesn't really matter). While this is being done another process might come along and try to save new content. The previous process opens the file with FileShare.None so no other process will be able to read the file until it is released by the first process. While the odds of this happening are minimal it could still happen.
Now when this does happen we have two options:
wait for the file to become available -
Emulate waiting on File.Open in C# when file is locked
we are talking about miliseconds here, so I guess this wouldn't be an issue as long as something strange happens and file never becomes available, then this solution would result in an infinite loop, so not an ideal solution
implement some sort of a queue and run all operations on files within a queue. My user experience requirements are such that at the time of saving/modifying files user should never be informed about exceptions or that something went wrong - he would get informed about them through a very friendly user interface later when operations would fail on the queue too.
At the moment of writing this, the solution should work within ASP.NET MVC application (both synchronously and async thru AJAX) but, if possible, it should use the concepts that could also work in Silverlight or Windows Forms or WPF application.
With regards to those two options which one do you think is better and for the second option what are possible technologies to implement this?
The ReaderWriterLockSlim class seems like a good solution for synchronizing access to the shared resource.
I have a text file and multiple threads/processes will write to it (it's a log file).
The file gets corrupted sometimes because of concurrent writings.
I want to use a file writing mode from all of threads which is sequential at file-system level itself.
I know it's possible to use locks (mutex for multiple processes) and synchronize writing to this file but I prefer to open the file in the correct mode and leave the task to System.IO.
Is it possible ? what's the best practice for this scenario ?
Your best bet is just to use locks/mutexex. It's a simple approach, it works and you can easily understand it and reason about it.
When it comes to synchronization it often pays to start with the simplest solution that could work and only try to refine if you hit problems.
To my knowledge, Windows doesn't have what you're looking for. There is no file handle object that does automatic synchronization by blocking all other users while one is writing to the file.
If your logging involves the three steps, open file, write, close file, then you can have your threads try to open the file in exclusive mode (FileShare.None), catch the exception if unable to open, and then try again until success. I've found that tedious at best.
In my programs that log from multiple threads, I created a TextWriter descendant that is essentially a queue. Threads call the Write or WriteLine methods on that object, which formats the output and places it into a queue (using a BlockingCollection). A separate logging thread services that queue--pulling things from it and writing them to the log file. This has a few benefits:
Threads don't have to wait on each other in order to log
Only one thread is writing to the file
It's trivial to rotate logs (i.e. start a new log file every hour, etc.)
There's zero chance of an error because I forgot to do the locking on some thread
Doing this across processes would be a lot more difficult. I've never even considered trying to share a log file across processes. Were I to need that, I would create a separate application (a logging service). That application would do the actual writes, with the other applications passing the strings to be written. Again, that ensures that I can't screw things up, and my code remains simple (i.e. no explicit locking code in the clients).
you might be able to use File.Open() with a FileShare value set to None, and make each thread wait if it can't get access to the file.
BACKGROUND:
I use an offset into a file and the Filestream lock/unlock menthods to control read/write access. I am using the following code to test if a lock is currently held on the file
try
{
fs.Lock( RESERVED_BYTE, 1 );
fs.Unlock( RESERVED_BYTE, 1 );
rc = 1;
}
catch
{
rc = 0;
}
QUESTION:
My goal is to eliminate the try/catch block. Is there some better way to see if the lock exists?
EDIT:
Note: This question is not about if the file exists. I already know it does. It is about synchronizing write access.
You can call the LockFile Windows API function through the P/Invoke layer directly. You would use the handle returned by the SafeFileHandle property on the FileStream.
Calling the API directly will allow you to check the return value for an error condition as opposed to resorting to catching an exception.
Noah asks if there is any overhead in making the call to the P/Invoke layer vs a try/catch.
The Lock file makes the same call through the P/Invoke layer and throws the exception if the call to LockFile returns 0. In your case, you aren't throwing an exception. In the event the file is locked, you will take less time because you aren't dealing with a stack unwind.
The actual P/Invoke setup is around seven instructions I believe (for comparison, COM interop is about 40), but that point is moot, since your call to LockFile is doing the same thing that the managed method does (use the P/Invoke layer).
Personally I would just catch a locked file when trying to open it. If it's unlocked now, it may be locked when you try to open it (even if it's just a few ms later).
My goal is to eliminate the try/catch block
Remember, the file system is volatile: just because your file is in one state for one operation doesn't mean it will be in the same state for the next operation. You have to be able to handle exceptions from the file system.
In some circumstances you can also use WCT, it's usually implmented by debugger's or profilers, however it can be used from any code as the usual debugger requirement of being the thread which has the debug port open is not a pre-requisit. As such WCT is a very comprehensive and precise information regarding lock contention.
A managed example (all-be-it somewhat trickey), show's utility for this specific sub-set of the native debug API's on the CLR.
I don't think it's possible without try, catch.
-What is the most foolproof way of ensuring the folder or file I want to manipulate is accessible (not read-only)?
-I know I can use ACL to add/set entries (make the file/folder non-readonly), but how would I know if I need to use security permissions to ensure file access? Or can I just add this in as an extra measure and handle the exception/negative scenario?
-How do I know when to close or just flush a stream? For example, should I try to use the streams once in a method and then flush/close/dipose at the end? If I use dispose(), do I still need to call flush() and close() explicitly?
I ask this question because constantly ensuring a file is available is a core requirement but it is difficult to guarantee this, so some tips in the design of my code would be good.
Thanks
There is no way to guarantee access to a file. I know this isn't a popular response but it's 100% true. You can never guarantee access to a file even if you have an exclusive non-sharing open on a Win32 machine.
There are too many ways this can fail that you simply cannot control. The classic example is a file opened over the network. Open it any way you'd like with any account, I'll simply walk over and yank the network cable. This will kill your access to the file.
I'm not saying this to be mean or arrogant. I'm saying this to make sure that people understand that operating on the file system is a very dangerous operation. You must accept that the operation can and will fail. It's imperative that you have a fallback scenario for any operation that touches disk.
-What is the most foolproof way of ensuring the folder or file I want to manipulate is accessible (not read-only)?
Opening them in write-mode?
Try and write a new file into the folder and catch any exceptions. Along with that do the normally sanity checks like folder/files exists etc.
You should never change the folder security in code as the environment could drastically change and cause major headaches. Rather ensure that the security is well documented and configured before hand. ALternatively use impersonation in your own code to ensure you are always running the required code as a user with full permissions to the folder/file.
Never call Dispose() unless you have no other choice. You always flush before closing the file or when you want to commit the content of the stream to the file/disk. The choice of when to do it depends on the amount of data that needs to be written and the time involved in writing the data.
100% foolproof way to ensure a folder is writable - create a file, close it, verify it is there, then delete it. A little tedious, but you asked for foolproof =)
Your better bet, which covers your question about ACL, is to handle the various exceptions if you cannot write to a file.
Also, I always call Close explicitly unless I need to read from a file before I'm done writing it (in which case I call flush then close).
Flush() - Synchronizes the in-memory buffer with the disk. Call when you want to write the buffer to the disk but keep the file open for further use.
Dispose(bool) - Releases the unmanaged resource (i.e. the OS file handle) and, if passed true, also releases the managed resources.
Close() - Calls Dispose(true) on the object.
Also, Dispose flushes the data before closing the handle so there is no need to call flush explicitly (although it might be a good idea to be flushing frequently anyway, depending on the amount and type of data you're handling).
If you're doing relatively atomic operations to files and don't need a long-running handle, the "using" paradigm is useful to ensure you are handling files properly, e.g.:
using (StreamReader reader = new StreamReader("filepath"))
{
// Do some stuff
} // CLR automagically handles flushing and releasing resources