This question already has an answer here:
Monitoring Garbage Collector in C#
(1 answer)
Closed 5 years ago.
After I handle all memory leaks in my application I experience a time to time freeze , most of the freeze happens on weak PC's
I thinking it's exactly the time when GC starts to work.
can I monitor GC invoke time to be sure that that's the problem
Thanks.
There are a number of performance counter related to GC: https://msdn.microsoft.com/en-us/library/x2tyfybc(v=vs.71).aspx
In your case, you should watch the value of # Gen 2 Collections and see if it increases when you experience the freeze. Note that it could also be gen 0 or 1 collections, but they're much less likely to cause a noticeable freeze.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I have 20 text files stored in hard disk each contains millions of information's about an educational organization.Suppose i have a method which will iterate text files in a loop and process .Which is best way to do the work starting each thread for each text file(Factory.startnew()) or each process for each textfile(process.start())
EDIT
I have 8GB RAM ,8core server ,so thought of to process them in thread or process.Currently i am using process and i don't find any bottleneck as of now.But i am in dilemma for using threads or process
The reading speed of the harddisk will most likely be the bottleneck here.
So, depending on the processing you need to do on the data, it might or might not be interesting to use multiple threads (and I would certainly not use processes).
The most important thing however, will be to make sure that no multiple threads are accessing the same physical disk at the same time, because that would lead to a slowdown because of constantly switching and seeking of the hdd-heads.
I have done some testing with that recently, and in some cases (depending on the hdd and/or pc) the OS takes care of it and it doesn't make a big difference, but on another combination however, a slowdown could be seen to 1/10 of the normal speed.
So, if using multiple threads (only needed if the processing of your data takes longer than the reading from your hdd!), make sure you have a lock somewhere to prevent multiple threads reading from the disk at the same time.
You might also want to look into memory mapped files for this.
edit:
In case you are working with buffers, you could start one thread to continuously fill the buffers, while another thread processes the data.
edit2 (in answer to Micky):
"Process or thread which is best ,faster and take less memory?"
As I said, I would not use processes (due to the extra overhead). That leaves threads, or no threads at all - depending on the amount of processing that needs to be done on the data. If data is read directly from memory buffers (instead of using something like readline for example, where all bets would be off), one or max. two threads would probably be the best option (if the processing of the data is fast enough - testing and timing would be needed to be sure).
As for speed and memory usage: best option (for me) would be memory mapped files (with the files opened in forward only mode). This would not only take advantage of the efficiency of the OS disk cache, but would also access the kernel-memory directly - while, when working with (user)buffers, memory has to be copied from kernel- to userspace, which takes time and uses extra memory.
IOCP: ok, but depends on what the threads would be asking. For example, if 10 threads would be asking 100kB each time in turn (on the different files), 10 x 10ms seektime would be needed, while reading 100Kb would take less than 1ms. Seektimes for future requests would depend on how IOCP handles the caching, which would probably be the same as using memory-mapping, but I don't think IOCP would be any faster in this case.
And using IOCP, would probably also be copying/filling buffers in userspace (and probably harder to handle in general). But I have to say, while writing my answer I was thinking C/C++ (using direct access to memory buffers) only to see later that it was C#. Although the principles stay the same, maybe there's an easy way in C# to use async I/O with IOCP.
As for the speed-testing and avoiding the reading at the same time: I have done testing with more than 50 threads on large files (via memory mapping) - and if done correctly, no reading-speed is lost. On the other hand, when just firing some threads and letting them access the hdd at random (even in large blocks), total reading-speed could come down to 10% in some cases - and sometimes not at all. Same PC, other hdd, other results.
This question already has answers here:
Is there an event for when garbage collection occurs in .NET?
(5 answers)
Closed 7 years ago.
I was just wondering if there is an overridable callback for the garbage collector begin/end in the .NET runtime/C#. I will also state that I have no intentions of attempting to control the GC, I am just curious.
And if not, what would be the best way to replicate this behavior?
You can't get notification directly in manged code because managed code is suspended (or at least not guaranteed to run your thread in case of background GC) during GC.
Options:
You can get approximate notification with GC.RegisterForFullGCNotification as shown in
Garbage Collection Notifications article.
You can watch performance counters related to GC if you need approximate time.
You can also capture ETW events to analyze behavior off-line - Garbage Collection ETW Events. There are many links on how to do that - Windows Performance Toolkit and SO questions like Consuming "Event Tracing for Windows" events.
If really necessary you can host CLR yourself and than you'll get notifications about GC - see CLR Indie Out:Hosting CLR and CLR Hosting - part 3. The ultimate method you want when hosting CLR to capture start/end of GC - IHostGCManager::SuspensionStarting.
This question already has answers here:
How to get memory available or used in C#
(6 answers)
Closed 9 years ago.
I'm creating a server (Console App), but after doing some long-term testing I found out it grows eating RAM. For the local test suite, I am not working with much RAM
(8GB-DDR3 #2400MHz)Is there a way (In Program.cs, I assume) to restart the program if it is using over 'x' amount of RAM? Also, one way could be a timed loop/checkup?
You can use GC.GetTotalMemory. It returns an approximate value (long) of how much memory your program has allocated.
You can create a Timer object and make this comparison under the Tick event handler.
For more information, you can look here: http://msdn.microsoft.com/en-us/library/system.gc.gettotalmemory.aspx
I agree with what others have said about fixing your memory leak.
If you want to restart your program, create a second application that monitors the first process. Then, when memory gets too high in your original app, safely shut it down and allow the second application to launch it again.
This question already has answers here:
Are there any differences between Java's "synchronized" and C#'s "lock"?
(3 answers)
Closed 9 years ago.
I'm wondering if there is any difference in runtime at lock vs syncronized.
I have learnd that syncronized is a slow operation and outdated at Java.
Today I saw the lock at C# and I'm wondering if they are the same and lock is something I "want" to avoid same as in Java or maybe he is much faster and I want to use it...
Thanks!
1 synchronized is not outdated, java.util.concurrent.locks package simply provides extended functions which are not always needed.
2 Locking is done at CPU level and there is no difference between Java and C# in this regard
see http://www.cs.umd.edu/~pugh/java/memoryModel/jsr-133-faq.html
... special instructions, called memory barriers, are required to flush or invalidate the local processor cache in order to see writes made by other processors or make writes by this processor visible to others. These memory barriers are usually performed when lock and unlock actions are taken; they are invisible to programmers in a high level language.
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
C# memory leak?
Sorry if this is not the right place to post this, seeing as it's not strictly code related.
Basically, since the beginning of this work, a program which has worked flawlessly before has suddenly started throwing Out of Memory exceptions. After checking out the usage, I discovered the memory usage increases by a ridiculous 1mb per second! This happens on a part of the program that iterates over a collection, and calls many different methods - so it seems like these objects are staying in memory even after use.
What does this issue sound like to everyone else? I've been debugging the code, ensuring nothing's left open and all is disposed, but to no avail. I'm wondering if perhaps the .NET framework on the computer it runs from has become corrupt, and the garbage collection isn't running.
I plan to test on another computer later to hopefully prove this theory
Garbage collection will only clear up objects in the heap with no references left to them. Although you've ensured you're disposing everything it'll also be an idea to specifically null any of these references to objects after use.
Perhaps you can post the troublesome code?
Have a look at dotTrace tool. 10 days trial version is available.
Using it you can track what keeps the object reference to objects that should be removed from memory.