I am using an asynchronous delegate that invokes a method which loads an xml file into an XPathDocument. If the xml is too big to fit into memory it never finishes loading. the code below works if the xml file is successfully loaded into the XPathDocument. I have been able to use a timer event that executes the asyncXpath.EndInvoke(result) statement and that works to end the CreateDocument method, but it does not stop the XPathDocument from loading. My conclusion is that the only thing I can do is to issue an Application.End statement to kill the application. Does anyone know how to stop a blackbox operation such as loading an XPathDocument.
delegate bool AsyncXpathQueryCaller(string xmlfile
bool found = false;
AsyncXpathQueryCaller asyncXpath = new
AsyncXpathQueryCaller(CreateDocument);
IAsyncResult result = asyncXpath.BeginInvoke(xmlfile, null, null);
while (!result.IsCompleted)
{
result.AsyncWaitHandle.WaitOne(100, false);
}
found = asyncXpath.EndInvoke(result);
private bool CreateDocument (string xmlfile)
{
XPathDocument doc = new XPathDocument(xmlfile);
}
What about using FileInfo before you try to load it and checking the size? If it's too big just skip it.
Something like this:
FileInfo fi = new FileInfo(xmlfile);
if(fi.Length < /*some huge number*/)
{
//load the file
}
You could declare a FileStream and give it to the constructor, but before you do, look at its Length property, and if it's too long, just return an error.
As proposed by Abe Miessler it is sensible to check the file size even before attempting to load it into an XPathDocument.
How would one decide what should be the limit?
There is no exact rule, but I have heard people say that you should multiply the file size by 5 and then the result is close to the memory that the XmlDocument will require in order for the text to be loaded/parsed.
EDIT: I just realized that KeithS has come close to a good answer. The basic idea is that you call the XPathDocument constructor that accepts a Stream which wraps a FileStream. The object you pass it should implement the Read(byte[], int, int) function to call the wrapped FileStream's Read function, or throw an exception if the operation has timed out. Here's a code sample:
class XmlStream : FileStream
{
DateTime deadline;
public XmlStream(string filename, TimeSpan timeout)
: base(filename, FileMode.Open)
{
deadline = DateTime.UtcNow + timeout;
}
public override int Read(byte[] array, int offset, int count)
{
if (DateTime.UtcNow > deadline)
throw new TimeoutException();
return base.Read(array, offset, count);
}
}
Here's some code that reads in document, but times out after 1 second:
bool found = true;
using(var stream = new XmlStream(document, TimeSpan.FromSeconds(1)))
try
{
xpath = new XPathDocument(stream);
}
catch (TimeoutException)
{
found = false;
}
If you create a separate thread instead of doing a BeginInvoke, you can just abort the thread when a timer ticks (or somebody clicks "Cancel"). While aborting threads is generally not advisable because it could be holding a lock or have global data in an inconsistent state, in this case it should be fine because your thread would not be holding a lock or accessing global data.
Here's the code for this method that does the same as the previous sample:
bool found = false;
thread = new Thread(() =>
{
xpath = new XPathDocument(document);
found = true;
});
thread.Start();
thread.Join(TimeSpan.FromSeconds(1));
thread.Abort();
If you're uncomfortable with aborting threads in your own app domain, you can create the document in another app domain and call AppDomain.Unload on it if it takes too long. That will require some marshalling, but probably won't have too much overhead.
The ultimate way to be able to kill a process is to run it in a separate process and use some sort of remoting interface to access it. That's probably even messier that the other options, though, as you have to worry about finding the executable, passing parameters, some user terminating it, and so on.
Related
I have a program that get user int input "1" and increment it based on the amount of files in a directory then stamp that int on each file( first is 1 and so on 1++). The foreach loop go in each directory gets its files, increment the input and call the stamp method until all files are done. In this process the order is important. However multitasking ( Parallel.ForEach) does't always guarantee order, in my understanding it returns which ever thread done first and maybe also damage the i++ functionality ( correct me if I'm wrong).
The question is how to apply multi threading in this case? i am thinking save the values of the foreach at the end, pass it to the stamping method and have the method stamp x amount of files at a time. I don't know if its possible or how to apply.
Here is my watermark method:
//text comes from the foreach already set.
public void waterMark(string text, string sourcePath, string destinationPathh)
{
using (Bitmap bitmap = new Bitmap(sourcePath))
{
//somecode
using (Graphics graphics = Graphics.FromImage(tempBitmap))
{
//somecode
tempBitmap.Save(destinationPathh, ImageFormat.Tiff);
//Erroe^: a generic error occurred in gdi+
//I think due to trying to save multiple files at once
}
}
}
The foreach loop:
var files = folder.GetFiles();
Parallel.ForEach(files, new ParallelOptions { MaxDegreeOfParallelism = 4 }, (file, state,indexer) =>
{
//somecode that calls the waterMark method in multiple spots as of now
});
Thank you in advance.
There is an overload of Parallel.ForEach that also provides an index for the item being processed:
Parallel.ForEach(someEnumerable, (val, state, idx) => Console.WriteLine(idx))
You can use it to keep track of the index in a thread-safe fashion.
As for the GDI+ stuff (Bitmap), I think you're safe as long as you use a single thread for all interactions with the bitmap. Don't try to do anything clever with async between instantiation and disposal.
I created a function to make sure an object is disposed of properly. This function includes setting the object to null. I am wondering if the line that sets the object to null is useless (and hence I will remove the line), and then add a line to set the object to null in the calling function. My example is for the FileStream object, but I any other object (I think) can take its place. I know I can trace the execution of the program and see what is happening, however, I would like to know more information on the inner mechanisms (garbage collection?), does this work for any object, etc.
//Called function:
public static void DiscardFile(System.IO.FileStream file)
{
file.Flush();
file.Close();
file.Dispose();
//Does this work?
//When the function returns, is the file object really set to null?
file = null;
}
//Calling function:
public static void WriteStringToFile(string s, string fileName)
{
System.IO.StreamWriter file = new System.IO.StreamWriter(fileName);
file.Write(s);
DiscardFile(file);
//Is this redundant?
//Or is the line in the called function the redundant line?
file = null;
}
Thanks!
I have a loop that writes a thousand strings to files within 30 seconds. (The program will be writing 400K+ strings when it completes its execution.) I see that the loop waits (every so often) at the file.Write(s) line, and that the memory footprint of the app increases. That is for another thread, but wanted to know the behavior of the above code.
Thanks!
Sorry, but your implementation is dangerous
public static void WriteStringToFile(string s, string fileName)
{
System.IO.StreamWriter file = new System.IO.StreamWriter(fileName);
file.Write(s); // <- the danger is here
DiscardFile(file);
//Is this redundant? Yes, it's redundant
//Or is the line in the called function the redundant line?
file = null;
}
Suppose you have an exception thrown on file.Write(s); it means that DiscardFile(file); will never be executed an you have resource leakage (HFILE - opened file handle).
Why not stick to standard using pattern:
public static void WriteStringToFile(string s, string fileName)
{
// Let system release all the resources acquired
using var file = new System.IO.StreamWriter(fileName);
{
file.Write(s);
} // <- here the resources will be released
}
In case of C# 8.0 you can get rid of pesky {...} and let the system release resources on leaving method's scope (see https://learn.microsoft.com/en-us/dotnet/csharp/whats-new/csharp-8#using-declarations):
public static void WriteStringToFile(string s, string fileName)
{
// Let system release all the resources acquired
using var file = new System.IO.StreamWriter(fileName);
file.Write(s);
} // <- here the resources will be released
I have an application where I need to create files with a unique and sequential number as part of the file name. My first thought was to use (since this application does not have any other data storage) a text file that would contain a number and I would increment this number so then my application would always create a file with a unique id.
Then I thought that maybe at a time when there are more than one user submitting to this application at the same time, one process might be reading the txt file before it has been written by the previous process. So then I am looking for a way to read and write to a file (with try catch so then I can know when it's being used by another process and then wait and try to read from it a few other times) in the same 'process' without unlocking the file in between.
If what I am saying above sounds like a bad option, could you please give me an alternative to this? How would you then keep track of unique identification numbers for an application like my case?
Thanks.
If it's a single application then you can store the current number in your application settings. Load that number at startup. Then with each request you can safely increment it and use the result. Save the sequential number when the program shuts down. For example:
private int _fileNumber;
// at application startup
_fileNumber = LoadFileNumberFromSettings();
// to increment
public int GetNextFile()
{
return Interlocked.Increment(ref _fileNumber);
}
// at application shutdown
SaveFileNumberToSettings(_fileNumber);
Or, you might want to make sure that the file number is saved whenever it's incremented. If so, change your GetNextFile method:
private readonly object _fileLock = new object();
public int GetNextFile()
{
lock (_fileLock)
{
int result = ++_fileNumber;
SaveFileNumbertoSettings(_fileNumber);
return result;
}
}
Note also that it might be reasonable to use the registry for this, rather than a file.
Edit: As Alireza pointed in the comments, it is not a valid way to lock between multiple applications.
You can always lock the access to the file (so you won't need to rely on exceptions).
e.g:
// Create a lock in your class
private static object LockObject = new object();
// and then lock on this object when you access the file like this:
lock(LockObject)
{
... access to the file
}
Edit2: It seems that you can use Mutex to perform inter-application signalling.
private static System.Threading.Mutex m = new System.Threading.Mutex(false, "LockMutex");
void AccessMethod()
{
try
{
m.WaitOne();
// Access the file
}
finally
{
m.ReleaseMutex();
}
}
But it's not the best pattern to generate unique ids. Maybe a sequence in a database would be better ? If you don't have a database, you can use Guids or a local database (even Access would be better I think)
I would prefer a complex and universal solution with the global mutex. It uses a mutex with name prefixed with "Global\" which makes it system-wide i.e. one mutex instance is shared across all processes. if your program runs in friendly environment or you can specify strict permissions limited to a user account you can trust then it works well.
Keep in mind that this solution is not transactional and is not protected against thread-abortion/process-termination.
Not transactional means that if your process/thread is caught in the middle of storage file modification and is terminated/aborted then the storage file will be left in unknown state. For instance it can be left empty. You can protect yourself against loss of data (loss of last used index) by writing the new value first, saving the file and only then removing the previous value. Reading procedure should expect a file with multiple numbers and should take the greatest.
Not protected against thread-abortion means that if a thread which obtained the mutex is aborted unexpectedly and/or you do not have proper exception handling then the mutex could stay locked for the life of the process that created that thread. In order to make solution abort-protected you will have to implement timeouts on obtaining the lock i.e. replace the following line which waits forever
blnResult = iLock.Mutex.WaitOne();
with something with timeout.
Summing this up I try to say that if you are looking for a really robust solution you will come to utilizing some kind of a transactional database or write a kind of such a database yourself :)
Here is the working code without timeout handling (I do not need it in my solution). It is robust enough to begin with.
using System;
using System.IO;
using System.Security.AccessControl;
using System.Security.Principal;
using System.Threading;
namespace ConsoleApplication31
{
class Program
{
//You only need one instance of that Mutex for each application domain (commonly each process).
private static SMutex mclsIOLock;
static void Main(string[] args)
{
//Initialize the mutex. Here you need to know the path to the file you use to store application data.
string strEnumStorageFilePath = Path.Combine(
Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData),
"MyAppEnumStorage.txt");
mclsIOLock = IOMutexGet(strEnumStorageFilePath);
}
//Template for the main processing routine.
public static void RequestProcess()
{
//This flag is used to protect against unwanted lock releases in case of recursive routines.
bool blnLockIsSet = false;
try
{
//Obtain the lock.
blnLockIsSet = IOLockSet(mclsIOLock);
//Read file data, update file data. Do not put much of long-running code here.
//Other processes may be waiting for the lock release.
}
finally
{
//Release the lock if it was obtained in this particular call stack frame.
IOLockRelease(mclsIOLock, blnLockIsSet);
}
//Put your long-running code here.
}
private static SMutex IOMutexGet(string iMutexNameBase)
{
SMutex clsResult = null;
clsResult = new SMutex();
string strSystemObjectName = #"Global\" + iMutexNameBase.Replace('\\', '_');
//Give permissions to all authenticated users.
SecurityIdentifier clsAuthenticatedUsers = new SecurityIdentifier(WellKnownSidType.AuthenticatedUserSid, null);
MutexSecurity clsMutexSecurity = new MutexSecurity();
MutexAccessRule clsMutexAccessRule = new MutexAccessRule(
clsAuthenticatedUsers,
MutexRights.FullControl,
AccessControlType.Allow);
clsMutexSecurity.AddAccessRule(clsMutexAccessRule);
//Create the mutex or open an existing one.
bool blnCreatedNew;
clsResult.Mutex = new Mutex(
false,
strSystemObjectName,
out blnCreatedNew,
clsMutexSecurity);
clsResult.IsMutexHeldByCurrentAppDomain = false;
return clsResult;
}
//Release IO lock.
private static void IOLockRelease(
SMutex iLock,
bool? iLockIsSetInCurrentStackFrame = null)
{
if (iLock != null)
{
lock (iLock)
{
if (iLock.IsMutexHeldByCurrentAppDomain &&
(!iLockIsSetInCurrentStackFrame.HasValue ||
iLockIsSetInCurrentStackFrame.Value))
{
iLock.MutexOwnerThread = null;
iLock.IsMutexHeldByCurrentAppDomain = false;
iLock.Mutex.ReleaseMutex();
}
}
}
}
//Set the IO lock.
private static bool IOLockSet(SMutex iLock)
{
bool blnResult = false;
try
{
if (iLock != null)
{
if (iLock.MutexOwnerThread != Thread.CurrentThread)
{
blnResult = iLock.Mutex.WaitOne();
iLock.IsMutexHeldByCurrentAppDomain = blnResult;
if (blnResult)
{
iLock.MutexOwnerThread = Thread.CurrentThread;
}
else
{
throw new ApplicationException("Failed to obtain the IO lock.");
}
}
}
}
catch (AbandonedMutexException iMutexAbandonedException)
{
blnResult = true;
iLock.IsMutexHeldByCurrentAppDomain = true;
iLock.MutexOwnerThread = Thread.CurrentThread;
}
return blnResult;
}
}
internal class SMutex
{
public Mutex Mutex;
public bool IsMutexHeldByCurrentAppDomain;
public Thread MutexOwnerThread;
}
}
I need a robust way of getting system uptime, and ended up using something as follows.
Added some comments to help people read it. I cannot use Task's as this has to run on a .NET 3.5 application.
// This is a structure, can't be marked as volatile
// need to implement MemoryBarrier manually as appropriate
private static TimeSpan _uptime;
private static TimeSpan GetUptime()
{
// Try and set the Uptime using per counters
var uptimeThread = new Thread(GetPerformanceCounterUptime);
uptimeThread.Start();
// If our thread hasn't finished in 5 seconds, perf counters are broken
if (!uptimeThread.Join(5 * 1000))
{
// Kill the thread and use Environment.TickCount
uptimeThread.Abort();
_uptime = TimeSpan.FromMilliseconds(
Environment.TickCount & Int32.MaxValue);
}
Thread.MemoryBarrier();
return _uptime;
}
// This sets the System uptime using the perf counters
// this gives the best result but on a system with corrupt perf counters
// it can freeze
private static void GetPerformanceCounterUptime()
{
using (var uptime = new PerformanceCounter("System", "System Up Time"))
{
uptime.NextValue();
_uptime = TimeSpan.FromSeconds(uptime.NextValue());
}
}
The part I am struggling with is where should Thread.MemoryBarrier() be placed?
I am placing it before reading the value, but either the current thread or a different thread could have written to it. Does the above look correct?
Edit, Answer based on Daniel
This is what I eneded up implementing, thank you both for the insight.
private static TimeSpan _uptime;
private static TimeSpan GetUptime()
{
var uptimeThread = new Thread(GetPerformanceCounterUptime);
uptimeThread.Start();
if (uptimeThread.Join(5*1000))
{
return _uptime;
}
else
{
uptimeThread.Abort();
return TimeSpan.FromMilliseconds(
Environment.TickCount & Int32.MaxValue);
}
}
private static void GetPerformanceCounterUptime()
{
using (var uptime = new PerformanceCounter("System", "System Up Time"))
{
uptime.NextValue();
_uptime = TimeSpan.FromSeconds(uptime.NextValue());
}
}
Edit 2
Updated based on Bob's comments.
private static DateTimeOffset _uptime;
private static DateTimeOffset GetUptime()
{
var uptimeThread = new Thread(GetPerformanceCounterUptime);
uptimeThread.Start();
if (uptimeThread.Join(5*1000))
{
return _uptime;
}
else
{
uptimeThread.Abort();
return DateTimeOffset.Now.Subtract(TimeSpan.FromMilliseconds(
Environment.TickCount & Int32.MaxValue));
}
}
private static void GetPerformanceCounterUptime()
{
if (_uptime != default(DateTimeOffset))
{
return;
}
using (var uptime = new PerformanceCounter("System", "System Up Time"))
{
uptime.NextValue();
_uptime = DateTimeOffset.Now.Subtract(
TimeSpan.FromSeconds(uptime.NextValue()));
}
}
Thread.Join already ensures that writes performed by the uptimeThread are visible on the main thread. You don't need any explicit memory barrier. (without the synchronization performed by Join, you'd need barriers on both threads - after the write and before the read)
However, there's a potential problem with your code: writing to a TimeSpan struct isn't atomic, and the main thread and the uptimeThread may write to it at the same time (Thread.Abort just signals abortion, but doesn't wait for the thread to finish aborting), causing a torn write.
My solution would be to not use the field at all when aborting. Also, multiple concurrent calls to GetUptime() may cause the same problem, so you should use an instance field instead.
private static TimeSpan GetUptime()
{
// Try and set the Uptime using per counters
var helper = new Helper();
var uptimeThread = new Thread(helper.GetPerformanceCounterUptime);
uptimeThread.Start();
// If our thread hasn't finished in 5 seconds, perf counters are broken
if (uptimeThread.Join(5 * 1000))
{
return helper._uptime;
} else {
// Kill the thread and use Environment.TickCount
uptimeThread.Abort();
return TimeSpan.FromMilliseconds(
Environment.TickCount & Int32.MaxValue);
}
}
class Helper
{
internal TimeSpan _uptime;
// This sets the System uptime using the perf counters
// this gives the best result but on a system with corrupt perf counters
// it can freeze
internal void GetPerformanceCounterUptime()
{
using (var uptime = new PerformanceCounter("System", "System Up Time"))
{
uptime.NextValue();
_uptime = TimeSpan.FromSeconds(uptime.NextValue());
}
}
}
However, I'm not sure if aborting the performance counter thread will work correctly at all - Thread.Abort() only aborts managed code execution. If the code is hanging within a Windows API call, the thread will keep running.
AFAIK writes in .NET are volatile, so the only place where you would need a memory fence would be before each read, since they are subject to reordering and/or caching. To quote from a post by Joe Duffy:
For reference, here are the rules as I have come to understand them
stated as simply as I can:
Rule 1: Data dependence among loads and stores is never violated.
Rule 2: All stores have release semantics, i.e. no load or store may move after one.
Rule 3: All volatile loads are acquire, i.e. no load or store may move before one.
Rule 4: No loads and stores may ever cross a full-barrier.
Rule 5: Loads and stores to the heap may never be introduced.
Rule 6: Loads and stores may only be deleted when coalescing adjacent loads and
stores from/to the same location.
Note that by this definition, non-volatile loads are not required to
have any sort of barrier associated with them. So loads may be freely
reordered, and writes may move after them (though not before, due to
Rule 2). With this model, the only true case where you’d truly need
the strength of a full-barrier provided by Rule 4 is to prevent
reordering in the case where a store is followed by a volatile load.
Without the barrier, the instructions may reorder.
i am extremely new to C# so excuse me if i don't explain this well.
i'm retrieving images from my computer camera, and along with displaying them in a PictureBox, i'm encoding them to jpegs and sending them to a shared dictionary. here's my code:
void CurrentCamera_OnImageCaptured(object sender, CameraEventArgs e)
{
this.pictureBoxMe.Image = e.Image;
if (myName != "" && Form1.PicSent)
{
SendPic sendP = new SendPic((Image)e.Image.Clone());
new System.Threading.Thread(new System.Threading.ThreadStart(sendP.send)).Start();
}
}
public class SendPic
{
Image im;
public SendPic (Image im)
{
this.im = im;
}
public void send(){
Form1.PicSent = false;
var memoryStream = new MemoryStream();
im.Save(memoryStream, ImageFormat.Jpeg);
var byteArray = memoryStream.ToArray();
Form1.sd["/" + myName + "/video"] = byteArray;
memoryStream.Close();
Form1.PicSent = true;
}
}
the problem is that i'm getting the "Object is currently in use elsewhere." error on the line: SendPic sendP = new SendPic((Image)e.Image.Clone());
based on other forum posts i've found, i already changed it so that the image is passed to the thread, and so that it's a clone. however i'm still getting the same error (though it lasts longer before crashing now).
i read something about locking? how do i implement that in this case? or is there something else i need to do?
thanks.
It behaves as though the OnImageCaptured method runs on a thread. Which isn't unlikely for camera interfaces. Set a breakpoint and use the debugger's Debug + Windows + Threads window to see what thread is running this code.
The failure mode is then that the UI thread is accessing the image to paint the picture box, simultaneously with this worker thread calling Clone(). GDI+ does not permit two threads accessing the same image object at the same time. It would indeed be flaky, no telling at what exact moment in the time the UI thread starts painting. PicSent is another accident waiting to happen.
One thing that catches my eye is that the SendPic class accesses your dictionary asynchronously (the line: Form1.sd["/" + myName + "/video"] = byteArray;).
However, dictionaries and hashtables are not guaranteed to be thread-safe for write operations. You should be safe if you make the code accessing the dictionary to be thread-safe. A simple lock would be a way to start.
Sort of like this:
public class SendPic
{
private object lockobj = new object();
// .... whatever other code ...
public void send()
{
// .... whatever previous code ...
lock(lockobj)
{
// assuming that the sd dictionary already has the relevant key
// otherwise you'd need to do a Form1.sd.Add(key, byteArray)
Form1.sd["/" + myName + "/video"] = byteArray;
}
// .... whatever following code ...
}
}
Quote from MSDN:
Thread Safety A Dictionary<(Of <(TKey, TValue>)>) can support
multiple readers concurrently, as long
as the collection is not modified.
Even so, enumerating through a
collection is intrinsically not a
thread-safe procedure. In the rare
case where an enumeration contends
with write accesses, the collection
must be locked during the entire
enumeration. To allow the collection
to be accessed by multiple threads for
reading and writing, you must
implement your own synchronization.