In my website I have a rather complicated business logic part encapsulated in a single DLL. Because CPU usage hits the roof when I call certain methods of it I need to measure and optimize.
I know about the performance profiler, but I don't know how to set it up for a library.
Furthermore I don't seem to find any usefull resources about it.
How do you approach this problem?
You can create a simple exe that runs the main methods of your library.
Although it requires some understanding to know which method to call it can help you focus on speciifc scenarios and check their bottlenecks.
You can also put some performance counters: look into msdn, or open a debugger and use old system: create a Stopwatch and do Debug.Writeline to see what's happening.
As Dror says, run it stand-alone under a simple exe. I would add, run THAT under the IDE, and while it's being slow, just pause it, several times, and each time look in detail at what it's doing. It's counterintuitive but very effective.
Related
Is there a way/system to debug/monitor code without stopping execution?
In industrial automation control programming (PLC/PAC/DCS) it is possible to connect the debugger while the program is running, and see in the code editor the value of variables and expressions, without setting breakpoints or tracepoints.
As an example, let's have a F# multithreaded application, where code is executed in a continuous loop or triggered by timers. Is there a way to attach a debugger like Visual studio Debugger and see the values of variables and expressions (in the code editor or in a watch pane) WITHOUT interrupting the execution?
It doesn't matter if it's not synchronous, it's acceptable if the debugger/monitor does not capture all the code scans.
I am tasked to create an high level controller for a process plant and I would like to use C# or F# or even C++ with a managed or native application, instead of a PAC system. But being forced to interrupt execution to debug is a huge disadvantage in this kind of application.
UPDATE
First of all thanks to all for their answer.
Based on those answers, though, I realized that probably I need to reformulate my question as follows:
Is anyone aware of any library/framework/package/extension that allows to work with a native or managed application in windows or linux (C#, F# or C++) the exact same way as a PAC development platform, specifically:
1) Put the dev platform in "status" mode, where it shows automatically the runtime value for variables and expressions present in the code exceprt currently visible, without interrupting execution?
2) Create watch windows that show the runtime value of variables and expressions, again without interrupting execution?
Also, what I am looking for is something that (like any PAC platform) offers these features OUT OF THE BOX, without requiring any change in the application code (like adding log instructions).
Thank you in advance
UPDATE 2
It looks like there is something (see http://vsdevaids.webs.com/); does anyone know whether they are still available somewhere?
UPDATE 3
For those interested, I managed to download the last available release of VSDEVAIDS. I installed it and looks working, but it's pointless without a licence and couldn't find information on how to reach the author.
http://www.mediafire.com/file/vvdk2e0g6091r4h/VSDevAidsInstaller.msi
If somebody has better luck, please let me know.
this is a normal requirement - needing instrumentation / diagnostic data from a production system. Its not really a debugger. Its usually one of the first things you should establish in your system design.
Not knowing your system at all its hard to say what you need but generally they fall into 2 categories
human readable trace - something like log4net is what I would recommend
machine readable counters etc. Say 'number of widget shaving in last pass',..... This one is harder to generalize, you could layer it onto log4net too. Or invent your own pipe
With regards to your edited question, I can almost guarantee you that what you are looking for does not exist. Consequence-free debugging/monitoring of even moderate usefulness for production code with no prior effort? I'd have heard of it. Consider that both C++ and C# are extremely cross-platform. There are a few caveats:
There are almost certainly C++ compilers built for very specific hardware that do what you require. This hardware is likely to have very limited capabilities, and the compilers are likely to otherwise be inferior to their larger counterparts, such as gcc, clang, MSVC, to name a few.
Compile-time instrumentation can do what you require, although it affects speed and memory usage, and even stability, in my experience.
There ARE also frameworks that do what you require, but not without affecting your code. For example, if you are using WPF as your UI, it's possible to monitor anything directly related to the UI of your application. But...that's hardly a better solution than log4net.
Lastly, there are tools that can monitor EVERY system call your application makes for both Windows (procmon.exe/"Process Monitor" from SysInternals) and Linux (strace). There's very little you can't find out using these. That said, the ease of use is hardly what you're looking for, and strictly internal variables are still not going to be visible. Still might be something to consider if you know you'll be making system calls with the variables you're interested in and can set up adequate filtering.
Also, you should reconsider your "No impact on the code" requirement. There are .NET frameworks that can allow you to monitor an entire class merely by making a single function call during construction, or by deriving from a class in the framework. Many modern UIs are predicated on the UIs being able to be notified of any change to the data they are monitoring. Extensive effort has gone into making this as powerful and easy as possible. But it does require you to at least consider it when writing your code.
Many years ago (think 8 bit 6502/6809 days) you could buy (or usually rent, I seem to remember a figure of £40K to purchase one in the late 80s) a processor simulator, that would allow you replace the processor in your design with a pin compatible device that had a flying lead to the simulator box. this would allow things like capturing instructions/data leading up to a processor interrupt, or some other way of stopping the processor (even a 'push button to stop code' was possible). You could even step-backwards allowing you to see why an instruction or branch happened.
In these days of multi-core, nm-technology, I doubt there is such a thing.
I have been searching for this kind of features since quite a long time with no luck, unfortunately. Submitting the question to the StackOverflow community was sort of a "last resort", so now I'm ready to conclude that it doesn't exist.
VSDevAids (as #zzxyz pointed out) is not a solution, as it requires significant support from the application itself.
Pod cpu emulators (mentioned by #Neil) aka in-circuit emulators (ICE) and their evolutions are designed to thoroughly test the interaction between firmware and hardware, not so useful in high level programming (especially if managed like .NET).
Thanks for all contributions.
I know a similar question has already been asked here What features should a C#/.NET profiler have? but this thread is not only about the wishlist but also about how to go about implementing that wishlist.
So let me ask you this just once more. I am in the process of building a generic performance profiler. I know I can take in a dll as input and take the usual Stopwatch approach to profile the response times of methods in that dll. But this is very basic stuff. I am willing to use third party api(or do some code on my own too) to extract whatever useful information I can lay my hands on from that dll. I want to know everything that makes it slow. I want to know about it's memory leaks. Anything at all that would help me find bottlenecks of the application. I'd want similar approach to find expensive db operations. But all this, under one application.
So what approach do you suggest? Which tools can I bring under my umbrella so that I can use them in my project?
I want to make a 'single' application that will take generic inputs like dlls, can also take input as source code tree(solution, projects, .cs files) and emit out results in the form of response times, identifying bottlenecks, memory leaks, etc.
Anything at all that would help me find bottlenecks of the
application.
Be careful of the universal profiling assumption that measurements unerringly lead to where the bottlenecks are, because some of them can be found that way, but only some.
Then the remaining bottlenecks sit there consuming time needlessly, but the developer is oblivious to them because profiler measurements did not isolate them.
A simple example could be some sort of dictionary lookup that appears to be optimal, except that the words being looked up are highly non-random. If certain words are looked up much more frequently, that represents an opportunity for optimization, but to detect that you need to know something about the data. Measuring profilers don't look at the program's data.
A more extreme example is any sort of interpreter, whose data is the "instruction set" for another language. The bottlenecks could easily be in that other language, but since it is data the measuring profiler would not see them.
What does see problems of this sort are not measurements but a small number of samples of the program's state, where the developer can fully examine and characterize the content of each sample (call stack and data). This leads to a much better understanding of how and why the program is spending its time than putting measurements on methods or browsing a call graph.
Is there an easy way of JIT-ing C# code up front, rather than waiting for the first time the code is invoked? I have read about NGEN but I don't think that's going to help me.
My application waits and responds to a specific external event that comes from a UDP port, and none of the critical-path code is (a) run before the event arrives, or (b) ever run again, so the cost of JIT is high in this scenario. Checking with ANTS profiler the overhead for JIT is around 40-50%, sometimes it's as high as 90%. My application is highly latency sensitive, and every millisecond counts.
My initial thought is that I could add a bool parameter to every critical path method, and call those methods before the event occurs, in order to initiate the JIT compile. However, is there a prettier and less hacky way?
Many thanks
I'd say use NGEN and if it doesn't work, you likely have deeper problems.
But, to answer your question, this article on how to pre-jit uses System.Runtime.CompilerServices.RuntimeHelpers.PrepareMethod to force a JIT. It includes sample code to use reflection to get the method handles.
What happens the second time the event arrives? Is it faster then or just as slow. If its still slow then JIT is not the problem, because the code gets "JIT"ed only once, the first time it is run.
NGEN would provide the answer for you. My suggestion is to take the bare minimum of the code you need, the critical path if you will, and put it in dummy/sandbox project. Start profiling/NGenning this code and seeing the performance.
If this bare minimum code, even after being NGEN'ed performs poorly on multiple calls, then pre-compiling isn't going to help you. Its something else in the code that is causing performance bottle necks.
How can I prove that multithreading is working in my C# programs? This is for a testing requirement. For example, I'm going to have to add some locking to a logger class (yes I know, I shouldn't have written my own logging class), and I need a test case to prove that the change will work.
If you want to test that your locking code is correctly synchronizing access to your log(s), you need to construct tests that guarantee contention. This may require you to refactor your code so that you can inject a mock log writer class that can hold the log's lock for arbitrary periods of time.
This is a broad topic, and you can find several related questions on StackOverflow, which are all worth reading:
How do I perform a Unit Test using threads?
How to write an automated test for thread safety
What are some strategies to unit test a scheduler?
Unit testing a multithreaded application?
Should I unit test for multithreading problems before writing any lock? (.NET C# TDD)
CHESS is a framework under development for identifying "assertion violations, deadlocks, livelocks, data-races, and memory-model errors." I haven't actually used this, but it looks like it might be extremely helpful.
Well, this may sound wrong but the truth is you can't prove multi-threaded behavior with unit-tests. Having said that, you can gain some confidence in the code with testing and over time it might actually present an issue.
<rant>
Multi-threaded code is the bane of my existence in many a project. Often people/developers do not have the expertise required to do a good job. Bugs often go unnoticed for long periods of time before anyone sees it in the wild, and then you can't reproduce the issue to identify whats going on. Further, attempting to 'fix' broken multi-threaded code via debugging is often not a viable approach.
</rant>
Anyway, go ahead and test it, there is no harm in doing that much and it's easy enough to do. Just fire up N number of threads, have them all wait on a ManualRestEvent, and then call your api in a tight loop a couple of hundred-thousand times :). But first I would recommend everyone on your team do a code review. Walk every line of code thinking about it executing in parallel. Ask yourself:
Do I really need this lock()?
What's the least amount of code that MUST be in the lock()?
Can I make this object/state immutable and avoid the locking?
Is there any way for a caller to have code execute inside the lock?
Review all members accessed and changed inside a lock for 'volatile'?
Are you using System.Threading.Thread.MemoryBarrier correctly?
If multiple locks are involved are they always obtained in the same order?
[wiki add here]
You just can't :) It's all depends on timing and it might blow up at any time. You have to mentally check every possible situation and that is the only way to go. That's why a lot of developers think multithreading is impossible to get right.
I have actually found Thread.Sleep() to be very useful to simulate various different race conditions. However, for obvious reasons you need to ensure that you either remove (or use configuration to disable) the Thread.Sleep before deploying to production.
In Robert C Martin's book "Clean Code", he recommends using "jiggling strategies" in your unit tests to ferret out multi-threading issues. "Jiggling" involves adding random wait times to your code so that threads run in different order at different times. You can then run your unit tests many times and your jiggling may route out some flaws. The important thing is NOT to ignore any unit test failures involving multithreading just because they pass the next time you run the test.
You actually can't. You can, however, write some debug-time code (start of routine, end of routine, special actions routine takes, ...) that writes to a console, so you can see that routines run at the same time.
Thread.Sleep. If you're suffering from race conditions with multithreading a well placed Thread.Sleep can increase the size of the race condition making it easier to reproduce.
WriteA();
// potential race condition as another bit of code might read all the state
// and only get A in their read.
WriteB();
to
WriteA();
Thread.Sleep(60000);
WriteB();
Then you can write code that reproduces the problem. Then you can write code that fixes the problem. Then you can assert that your fix works. Profit!
Another thread posted a related answer, using Microsoft's CHESS program.
If I have an existing solution containing multiple c# projects, are there any static analysis tools that can help me determine which areas of code are the most often used?
I'd like to use this information in order to determine which areas should have their test coverage ramped up first.
I've looked at some static analysis tools already, but they mostly seem to focus on things like complexity, coding conventions, code duplication etc.
Alternatively, if there aren't any analysis tools available that can do this, do you have any advice about how to determine which code I ought to focus on testing first?
Thanks!
EDIT: Just to clarify, what I'm looking for isn't code coverage. I'd like a rundown of which parts of my application are most often used, so that I can in turn focus on improving the coverage in those areas. I'm trying to avoid just writing tests for areas that don't yet have any, as they may be edge cases that aren't often executed.
Even static analysis tools which do try to figure out what happens at run-time do not usually try to estimate how often a piece of code is executed. The subject is hard enough as it is!
But dynamic analysis tools (e.g. profiling tools that either rely on transparent instrumentation of the code or use sampling) can tell you, after one or several "typical" executions (you provide the entries that you judge typical), how often this or that function was executed.
See Profiling (computer programming) on Wikipedia.
If I understood the question right, you are looking for a profiler. Give EQATEC Profiler a try. It's free.
It's originally intended to profile an application before shipping (to detect bottlenecks by measuring execution time of methods etc.) so I'm not sure if it's suitable for an application in a productive environment. At least it changes your code for profiling purposes and that might be unwanted. You should check this out.
Code coverage appears to be what you want.
NCover is a popular code coverage tool for .NET, if you can afford it.
What you're asking for is simply impossible to do accurately. The number of times something is executed can and usually will depend on the data that's entered at run-time. The best you can hope for from a static analysis tool isa direct answer when statically determinedAn O(N) style analysis otherwise
Even the latter would be quite difficult to get right overall. For example, it would need to know the complexity of essentially every function in the (huge and ever-expanding) .NET library. Some of those are hard to even characterize.
Just for example, how long does it take to allocate a block of memory? Well, usually it's usually nearly constant time -- but it's always possible that an allocation can trigger a garbage collection cycle, in which case the time taken will be (roughly) proportional to the number of objects still in use that were allocated since the last GC cycle...
"Profiler" is what you're looking for; which you choose is up to you.
I've used HP's Diagnostic Server to do this, although it'd cost money. It'll tell me what methods are called how many times, and the average and worst-case time spent in them.
As an important safety tip, running a profiler will slow down the execution of your code; it's not ideal for a long-term installation into a production environment.
If you want to see just what is used:
SD C# Test Coverage Tool
If you want to see how often it is used:
SD C# Profiler Tool
If coverage is not what you look for then you could look use two things:
As suggested a profiler and run predictive test scenarios.
Use performance counters which would end in a more permanent solution. These are quite difficult to implement but are helpful to diagnose performance deltas by analyzing the counters report. One way to implement them would be to wrap boundaries and manage counters from those wrap. Be wary that it's way much easier to integrate in a new project than in an existing one.