How do you calculate memory (RAM) bandwidth used? Which performance counters are required?
I came across a tool that was able to do it, the "Rightmark multi-threaded memory test". But unlike the rest of Rightmark's tests, I haven't found the source code for it, just the binaries
If your code can run on Linux, use Cachegrind:
Cachegrind is a cache profiler. It
performs detailed simulation of the
I1, D1 and L2 caches in your CPU and
so can accurately pinpoint the sources
of cache misses in your code. It
identifies the number of cache misses,
memory references and instructions
executed for each line of source code,
with per-function, per-module and
whole-program summaries. It is useful
with programs written in any language.
Cachegrind runs programs about
20--100x slower than normal.
You may want to use the KCacheGrind GUI.
It is very difficult to 'calculate' memory bandwidth usage. There are lots of non-trivial cache and MMU issues to contend with. The only real way to do it is either through the use of simulation or real-world measurements.
You can get a 'rough' idea by debugging the code and counting the number of memory load and store operations performed. However, knowing whether it was a cache hit/miss is another issue.
It depends on your purpose. If it is to obtain a guesstimate, you can use the rule of thumb that about 30% of general purpose code is memory loads and stores. If you're trying to get a worst case, you can assume that caches miss all the time and work it out.
One potential thing you could do is to look at virtualisation. There are several open source options (QEMU comes to mind). It may be possible to export certain hardware measurements from them.
Related
I am having problems figuring out the source of a memory problem in a complex C# based Windows service. Unfortunately the problem does not occur all the time and I still don't exactly know the conditions which cause it to happen. Sometimes when I check the system ressources used by the service, it takes up multiple gigabytes of memory until the point where it throws OutOfMemory-exceptions everywhere because there isn't any memory left.
I have a paid version of .NET Memory Profiler available but so far it has been useless because the whole system becomes slow and unstable when the service uses too much memory so I cannot attach the memory profiler to the application.
The solution of the application consists of more than 30 individual projects and hundreds of thousands lines of code so there is no way for me to find the source of the problem by simply looking through the source code.
So far the only thing I was able to do is creating a memory dump (.dmp file) of the process while it was using a lot of memory. Is there a way to analyze this dump or anything else that would help me narrow down the source of this problem?
If you could identify some central methods in the main classes of your legacy projects and you have some kind of logging already in place, you could log the total memory (managed and unmanaged, if your application opens such resources) by calling
Process.GetCurrentProcess().PrivateMemorySize64
At least that would give you a feeling if the memory problem is "diffuse", e. g. by not release objects for garbage collection etc., if if the memory problem occurs just in certain use cases (jump in memory consumption, if a certain action happens). Then you could nail it down by more logging and investigating the corresponding code sections. Its tedious, but when you can not use code instrumentation as you have said, I find it effective. If you want to analyze a specific situation with a memory dump, you can use WinDbg for analyzing it, but that takes some effort for the first time to learn, and would be a separate topic (see https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/debugger-download-tools).
I'm developing a benchmarking application using Universal Windows Platform that evaluates CPU and RAM performance of a Windows 10 system.
Although I found different algorithms to benchmark a CPU, I still didn't found any solid algorithm or solution to evaluate the write and read speeds of memory.
How can I achieve this in C#?
Thanks in advance :)
I don't see why this would not be possible from managed code. Array access code turns into normal x86 memory instructions. It's a thin abstraction. In particular I don't see why you would need a customized OS.
You should be able to test sequential memory speed by performing memcpy on big arrays. They must be bigger than the last level cache size.
You can test random access by randomly indexing into a big array. The index calculation must be cheap, unpredictable and there must be a dependency chain that serializes the memory instructions so that the CPU cannot parallelize them.
Honestly I don't think its possible. RAM benchmarks usually run off of dedicated OS's
RAM testing is different from RAM benchmarking.
C# doesn't give you that kind of control over RAM
Of course, just new up a big array and access it. Also, understand the overheads that are present. The only overhead is a range check.
The GC has no impact during the benchmark. It might be triggered by an allocation.
I know a similar question has already been asked here What features should a C#/.NET profiler have? but this thread is not only about the wishlist but also about how to go about implementing that wishlist.
So let me ask you this just once more. I am in the process of building a generic performance profiler. I know I can take in a dll as input and take the usual Stopwatch approach to profile the response times of methods in that dll. But this is very basic stuff. I am willing to use third party api(or do some code on my own too) to extract whatever useful information I can lay my hands on from that dll. I want to know everything that makes it slow. I want to know about it's memory leaks. Anything at all that would help me find bottlenecks of the application. I'd want similar approach to find expensive db operations. But all this, under one application.
So what approach do you suggest? Which tools can I bring under my umbrella so that I can use them in my project?
I want to make a 'single' application that will take generic inputs like dlls, can also take input as source code tree(solution, projects, .cs files) and emit out results in the form of response times, identifying bottlenecks, memory leaks, etc.
Anything at all that would help me find bottlenecks of the
application.
Be careful of the universal profiling assumption that measurements unerringly lead to where the bottlenecks are, because some of them can be found that way, but only some.
Then the remaining bottlenecks sit there consuming time needlessly, but the developer is oblivious to them because profiler measurements did not isolate them.
A simple example could be some sort of dictionary lookup that appears to be optimal, except that the words being looked up are highly non-random. If certain words are looked up much more frequently, that represents an opportunity for optimization, but to detect that you need to know something about the data. Measuring profilers don't look at the program's data.
A more extreme example is any sort of interpreter, whose data is the "instruction set" for another language. The bottlenecks could easily be in that other language, but since it is data the measuring profiler would not see them.
What does see problems of this sort are not measurements but a small number of samples of the program's state, where the developer can fully examine and characterize the content of each sample (call stack and data). This leads to a much better understanding of how and why the program is spending its time than putting measurements on methods or browsing a call graph.
I understand there are many questions related to this, so I'll be very specific.
I create Console application with two instructions. Create a List with some large capacity and fill it with sample data, and then clear that List or make it equal to null.
What I want to know is if there is a way for me to know/measure/profile while debugging or not, if the actual memory used by the application after the list was cleared and null-ed is about the same as before the list was created and populated. I know for sure that the application has disposed of the information and the GC has finished collecting, but can I know for sure how much memory my application would consume after this?
I understand that during the process of filling the list, a lot of memory is allocated and after it's been cleared that memory may become available to other process if it needs it, but is it possible to measure the real memory consumed by the application at the end?
Thanks
Edit: OK, here is my real scenario and objective. I work on a WPF application that works with large amounts of data read through USB device. At some point, the application allocates about 700+ MB of memory to store all the List data, which it parses, analyzes and then writes to the filesystem. When I write the data to the filesystem, I clear all the Lists and dispose all collections that previously held the large data, so I can do another data processing. I want to know that I won't run into performance issues or eventually use up all memory. I'm fine with my program using a lot of memory, but I'm not fine with it using it all after few USB processings.
How can I go around controlling this? Are memory or process profilers used in case like this? Simply using Task Manager, I see my application taking up 800 MB of memory, but after I clear the collections, the memory stays the same. I understand this won't go down unless windows needs it, so I was wondering if I can know for sure that the memory is cleared and free to be used (by my application or windows)
It is very hard to measure "real memory" usage on Windows if you mean physical memory. Most likley you want something else like:
Amount of memory allocated for the process (see Zooba's answer)
Amount of Managed memory allocated - CLR Profiler, or any other profiler listed in this one - Best .NET memory and performance profiler?
What Task Manager reports for your application
Note that it is not necessary that after garbage collection is finished amount of memory allocated for your process (1) changes - GC may keep allocated memory for future managed allocations (this behavior is not specific to CLR for memory allcation - most memory allocators keep free blocks for later usage unless forced to release it by some means). The http://blogs.msdn.com/b/maoni/ blog is excelent source for details on GC/memory.
Process Explorer will give you all the information you need. Specifically, you will probably be most interested in the "private bytes history" graph for your process.
Alternatively, it is possible to use Window's Performance Monitor to track your specific application. This should give identical information to Process Explorer, though it will let you write the actual numbers out to a separate file.
(A picture because I can...)
I personaly use SciTech Memory Profiler
It has a real time option that you can use to see your memory usage. It has help me find a number of problems with leaking memory.
Try ANTS Profiler. Its not free but you can try the trial version.
http://www.red-gate.com/products/dotnet-development/ants-performance-profiler/
I have a long running console app running through millions of iterations. I want to benchmark if memory usage increases linerally as the number of iterations increases.
What would be the best way to go about this?
I think I really only need to be concerned with Peak memory usage during a run right? I basically need to work out what is the max number of iterations I can run on this hardware given the memory on the Server.
I am going to setup a batch lot of runs and Log the results over different interation sizes and then graph the results to identify a memory usage trend which can then be extrapolated for any given hardware.
Looking for advice on the best way to implement this, what .net methods, classes to use or should I use external tools. This article http://www.itwriting.com/dotnetmem.php suggests I should profile my own app via code to factor out shared memory used by the .net runtime across other apps on the box.
Thanks
There are several ways to do this:
Perfmon UI
You can use the Performance Montior control panel applet (in Administrative Tools) that comes with Windows to monitor your application. Have a look at the .Net CLR Memory category and the counters inside it. You can also limit the monitoring to just your process. This is probably the easiest as it requires no code changes.
Perfmon API
You can programmatically use performance counters from .Net. For this, you'll need to use the PerformanceCounter class. This is just an API to the same underlying information that the above UI presents.
Memory Profiler
You can use a memory profiler to profile you application whilst it is running. The two I have used with success are the ANTS Memory Profiler from RedGate and the .Net Memory Profiler from SciTech. Again, this requires no code changes, but may cost you money (although there are free trial versions). There is also the CLR Profiler (a howto can be found here).
Roll Your Own
You can get hold of some limited memory information from the Process class. Get hold of the current process using Process.GetCurrentProcess(), and then look at the properties of it, specifically the ones relating to memory (MinWorkingSet, MaxWorkingSet, PagedMemorySize64, PeakPagedMemorySize64, PeakVirtualMemorySize64, PeakWorkingSet64, PrivateMemorySize64, VirtualMemorySize64, WorkingSet64). This is probably the worst solution as you have to do everything yourself, including data collection and reporting.
If all you are wanting to do is to verify your application doesn't linearly increase its memory usage as your the number of iterations increases, I would recommend monitoring using the Performance Monitor UI in Windows. It will show you what you need with minimal effort on your part.
Windows perfmon is great for this kind of stuff. You have counters for all the managed heaps and for private bytes, if you need to cross correlate with iteration counts you can publish your own perfmon counters from .Net.
when you want to profile the mem usage of your apps, here is a piece of code you can use.
First you have to declare an instance of the PerformanceCounter class
Assembly a = Assembly.GetExecutingAssembly();
_PerfCounter = new PerformanceCounter(".NET CLR Memory",
"# Bytes in all Heaps",
a.GetName().Name,
true);
and each time you want to log the memory consumption you can call
_PerfCounter.NextValue()
Hope it was helpful
Maybe this tool can do everything you want.
Better late than never, but in response to M3ntat's comment, you'll need to add ".vshost" onto the assembly name parameter if you're running from within Visual Studio, e.g.
a.GetName().Name + ".vshost"