I've been doing quite a bit of debugging of managed applications lately using both Visual Studio and WinDbg, and as such I'm often ask to assist colleagues in debugging situations. On several occasions I have found people aho just insert break points here and there and hope for the best. In my experience that is rarely a useful technique.
My approach goes something like this.
Reproduce the problem. Ideally reduce the input as much as possible.
Examine what goes wrong and list theories for where the bug may be.
Examine one theory at a time by debugging that specific area of the code.
Repeat steps as necessary.
For complex debugging problems I often work with a colleague. For WinDbg this is especially useful.
Any other useful tips or best practices for debugging?
If there was one tip I could give to everyone about debugging it would be to break it again.
That is, when you think you've found the fix and the system seems to work. Back the fix out and see if the system breaks again.
Sometimes you can get lost in the sequence of what you've tried as potential solutions and you finish up in a totally different area of the system while you're debugging the problem. Then you forget what you've changed back in the original area where you were working.
Backing the fix out and then reproducing the problem ensures that the candidate fix isn't relying on something else that you've changed in another part of the system. That your patch for the fix is a correct standalone solution.
HTH.
cheers,
Rob
One very best practice is not diving into debugger immediately but look at the code and think hard for some time.
I'm not sure where I read about "Rubber Duck Debugging", but I think its great. The basic idea is to set a rubber duck on your desk and explain the code to it. The idea is that as you explain the code to the duck, you'll eventually find yourself saying "Now, this happens", and you'll notice that 'this' is not what you intend to be happening.
Lacking a duck, I find I just walk through the code and explain it to myself. It works, but I still think I might bring in a duck.
[EDIT]
I found where I read about the rubber duck Rubber Duck Debugging
As another poster says, with some hard thinking, it's often possible to just see the logic error if you understand what's going on.
But often we think we do, and we don't, or we're simply required to fix something that we don't understand too well, so it's back to first principles.
Reproducing the problem is certainly a vital first step. If you can't do this, then you stand no chance of finding the problem, except by accident.
The next step is to establish beyond doubt the path through the code that actually executes when the bug hits. In a WinForms application that might have many events, and more than one thread, this can be anything but a trivial exercise.
Until you know exactly where the code is going, all the theories in the world about where the bug may be are worthless. And if the code is complex, discovering that code does not stop on a breakpoint can be as informative as having it stop.
So in my experience, using breakpoints early and often can be an essential tool for discovering a how code's working.
I often find that when a problem seems particularly intractable, it's because I've made a fatal assumption about what's going on, and not actually verified it.
So my 'best practice' is not to move on until I'm sure I understand, and not guess.
Not directly related to debugging, but to make debugging easier in the future, there are a few things to consider:
Implementing unit testing, preferably in the form of TDD, forces you to stay on task and develop only with the goal of passing tests. It is harder to "wander" when you are coding to a test, instead of to a task.
Get in the practice of regularly refactoring your code. Small, on-point methods are easier to debug than monolithic "jack of all trades" methods.
Utilize your team members. Often adding an extra set of eyes can help flush something out. Chances are, if you do not find something in a relatively quick manner, you are going to continue to overlook it for a while.
You can always rollback code in your version control system to try and isolate what version of a file caused the introduction of the bug. Once you do that, you can diff between the last good and first bad and just focus on the changes between the two.
The barriers to entry for the debugger in VS.NET with a language like C# or VB.NET is just so ridiculously low that it is often easier to insert a breakpoint or two where you know the problem is and just step through.
Sometimes I find myself using Edit & Continue to write code. It's great. You can see results immediately. Often it's most useful when there's some algorithm or relatively difficult to understand loop.
This book is honestly the best I've read about debugging, especially when you are beyond the normal debugging situation. It contains many tricks and is fun to read with all the "true story". If you are working with a big amount of code that you haven't written yourself, especially if it's crappy, this book is a must!
http://www.amazon.com/Debugging-Applications-Microsoft%C2%AE-Microsoft-Pro-Developer/dp/0735615365/ref=sr_1_1?ie=UTF8&s=books&qid=1238705836&sr=1-1
alt text http://ecx.images-amazon.com/images/I/51RQ146x9VL._SS500_.jpg
Something that helps, especially when you're new to debugging, is to keep some kind of debugging journal, with solutions to problems you've solved in the past. Most bugs follow relatively common patterns (for instance, apparently random problems in non-threaded applications are usually due to undefined variables, or similar use of uninitialized memory) and by keeping track of those patterns, you'll get much better at nailing in on future problems.
After a while, you just develop the necessary intuition (and then your journal becomes a very fun memory of all the nasty enemies you've conquered)
Like Conceptual Blockbusting suggests, I like to try different ways whenever I get stuck. "printf debugging", thinking about the behavior, binary search on code, binary search on version-control commits, writing a unit test to clarify, scratch refactoring, and also firing the debugger.
IMO too much preparation is a waste of time. If you know the codebase reasonably well, you can usually right aways think of a few key places where the problem manifests. Put break points there and see if you're right. Whenever you see better key points, move your breakpoints to get closer to the problem.
When you're chasing bad data such as a null pointer, it depends on where it comes from: if it's passed as an argument, look at the call stack to find where it comes from. If it's part of some data structure or object (the easier case), put a breakpoint there to see when and how it is modified.
Conitional breakpoints can be a great help, otherwise you can simulate them by adding if statements enclosing no-ops. If you have a breakpoint in a hot spot that gets hit too often before you get to the problem, deactivate it and put another one in a place you know will be hit shortly before the problem manifests, then activte the one in the hot spot.
A good practice is to make sure you're not fixing a symptom, but the cause.
Often, one might see an odd value while debugging and fix it there, without checking what caused the value to get there in the first place. This is, of course, a very bad idea.
BTW, this is why Linus objected adding built-in support for kernel debugging.
I'll paraphrase my answer on a similar thread (which is essentially the last bullet point in joseph.ferris's answer to this thread ):
Using your version control system, isolate the file revision where the bug was introduced, using binary search tree approach.
Diff that revision of the source file against the previous revision. The diff may make the reason for the bug apparent.
This is by no means a technical tip but it often works in my case.
Just stop working hard to find a root cause or fix a bug. Relax for a while: take a walk, eat dinner, or just switch to another task (hopefully much easier) - whatever you like...
...Then think about a problem a bit later, when you're "fresh" again. Trace back all the mental process of debugging that you already went through (theories, experiments, assumptions etc. that you made). Chances are that you will instantly see some key factor that you overlooked before ;).
Programmers (or at least me) tend to gradually narrow down thier perspective of problem and wear off of creativity during a long debugging session. But wide perspective combined with creative ideas are man's most powerful weapon in the battle with bugs!
I just replayed in another post, the question was C debugging but as i stated in my replay i think that debugging techniques are language independent.
One thing I like to hammer home is that where you have one instance working and one not (say production and dev) it's about the differences and you need to clearly identify what those could be and deal with them one at a time. Environmental problems can be the hardest to trace and you'll go insane if you don't work systematically.
Incidentally this is one of the reasons I habitually run my VS webapp projects through IIS not cassini anymore.
Another thing I've started doing to all my projects is to add a TraceListener (or derived class) and using it to take key snapshots of my application.
This usually gives me a good idea of where to focus my initial debugging efforts.
Plus, I can it on/off using a config file switch, so I can even get a hint on a production system, without recompiling the code.
Related
Is there a way/system to debug/monitor code without stopping execution?
In industrial automation control programming (PLC/PAC/DCS) it is possible to connect the debugger while the program is running, and see in the code editor the value of variables and expressions, without setting breakpoints or tracepoints.
As an example, let's have a F# multithreaded application, where code is executed in a continuous loop or triggered by timers. Is there a way to attach a debugger like Visual studio Debugger and see the values of variables and expressions (in the code editor or in a watch pane) WITHOUT interrupting the execution?
It doesn't matter if it's not synchronous, it's acceptable if the debugger/monitor does not capture all the code scans.
I am tasked to create an high level controller for a process plant and I would like to use C# or F# or even C++ with a managed or native application, instead of a PAC system. But being forced to interrupt execution to debug is a huge disadvantage in this kind of application.
UPDATE
First of all thanks to all for their answer.
Based on those answers, though, I realized that probably I need to reformulate my question as follows:
Is anyone aware of any library/framework/package/extension that allows to work with a native or managed application in windows or linux (C#, F# or C++) the exact same way as a PAC development platform, specifically:
1) Put the dev platform in "status" mode, where it shows automatically the runtime value for variables and expressions present in the code exceprt currently visible, without interrupting execution?
2) Create watch windows that show the runtime value of variables and expressions, again without interrupting execution?
Also, what I am looking for is something that (like any PAC platform) offers these features OUT OF THE BOX, without requiring any change in the application code (like adding log instructions).
Thank you in advance
UPDATE 2
It looks like there is something (see http://vsdevaids.webs.com/); does anyone know whether they are still available somewhere?
UPDATE 3
For those interested, I managed to download the last available release of VSDEVAIDS. I installed it and looks working, but it's pointless without a licence and couldn't find information on how to reach the author.
http://www.mediafire.com/file/vvdk2e0g6091r4h/VSDevAidsInstaller.msi
If somebody has better luck, please let me know.
this is a normal requirement - needing instrumentation / diagnostic data from a production system. Its not really a debugger. Its usually one of the first things you should establish in your system design.
Not knowing your system at all its hard to say what you need but generally they fall into 2 categories
human readable trace - something like log4net is what I would recommend
machine readable counters etc. Say 'number of widget shaving in last pass',..... This one is harder to generalize, you could layer it onto log4net too. Or invent your own pipe
With regards to your edited question, I can almost guarantee you that what you are looking for does not exist. Consequence-free debugging/monitoring of even moderate usefulness for production code with no prior effort? I'd have heard of it. Consider that both C++ and C# are extremely cross-platform. There are a few caveats:
There are almost certainly C++ compilers built for very specific hardware that do what you require. This hardware is likely to have very limited capabilities, and the compilers are likely to otherwise be inferior to their larger counterparts, such as gcc, clang, MSVC, to name a few.
Compile-time instrumentation can do what you require, although it affects speed and memory usage, and even stability, in my experience.
There ARE also frameworks that do what you require, but not without affecting your code. For example, if you are using WPF as your UI, it's possible to monitor anything directly related to the UI of your application. But...that's hardly a better solution than log4net.
Lastly, there are tools that can monitor EVERY system call your application makes for both Windows (procmon.exe/"Process Monitor" from SysInternals) and Linux (strace). There's very little you can't find out using these. That said, the ease of use is hardly what you're looking for, and strictly internal variables are still not going to be visible. Still might be something to consider if you know you'll be making system calls with the variables you're interested in and can set up adequate filtering.
Also, you should reconsider your "No impact on the code" requirement. There are .NET frameworks that can allow you to monitor an entire class merely by making a single function call during construction, or by deriving from a class in the framework. Many modern UIs are predicated on the UIs being able to be notified of any change to the data they are monitoring. Extensive effort has gone into making this as powerful and easy as possible. But it does require you to at least consider it when writing your code.
Many years ago (think 8 bit 6502/6809 days) you could buy (or usually rent, I seem to remember a figure of £40K to purchase one in the late 80s) a processor simulator, that would allow you replace the processor in your design with a pin compatible device that had a flying lead to the simulator box. this would allow things like capturing instructions/data leading up to a processor interrupt, or some other way of stopping the processor (even a 'push button to stop code' was possible). You could even step-backwards allowing you to see why an instruction or branch happened.
In these days of multi-core, nm-technology, I doubt there is such a thing.
I have been searching for this kind of features since quite a long time with no luck, unfortunately. Submitting the question to the StackOverflow community was sort of a "last resort", so now I'm ready to conclude that it doesn't exist.
VSDevAids (as #zzxyz pointed out) is not a solution, as it requires significant support from the application itself.
Pod cpu emulators (mentioned by #Neil) aka in-circuit emulators (ICE) and their evolutions are designed to thoroughly test the interaction between firmware and hardware, not so useful in high level programming (especially if managed like .NET).
Thanks for all contributions.
I find timing my code for performance to be a bit of a headache and I have to assume this is a common problem for folks.
Adding in a bunch of code to time my loops is great but often there is more overhead in adding the timing checks than adding a parallel task.
making a loop parallel can take seconds whereas adding in timing chunks requires several lines of code and be dangerous if forgotten.
So... My question is why dont IDE's include this ability to time chunks of code like a break point?
Something like i mark a section of code and tell it to dump the timing output to a console with some options to make it more readable. averages and such.
I know break points can seriously hamper performance but there must be an easier way to time specific chunks of code then to write all my own timing routines for each chunk of code i want to check.
First, VS 2015 does offer timing indicators while debugging.
(source: s-msft.com)
But I say don't use it for serious profiling purposes. Use it to get a very rough idea of what's going on.
Timing anything in a debug build yields unreliable figures, because of many factors (disabled compiler optimizations, disabled JIT optimizations, nop instructions everywhere, breakpoints, etc). Garbage in, garbage out.
Just do your profiling properly, it's a hard thing to get right. And use a real profiler for that if you need detailed performance data.
If I have an existing solution containing multiple c# projects, are there any static analysis tools that can help me determine which areas of code are the most often used?
I'd like to use this information in order to determine which areas should have their test coverage ramped up first.
I've looked at some static analysis tools already, but they mostly seem to focus on things like complexity, coding conventions, code duplication etc.
Alternatively, if there aren't any analysis tools available that can do this, do you have any advice about how to determine which code I ought to focus on testing first?
Thanks!
EDIT: Just to clarify, what I'm looking for isn't code coverage. I'd like a rundown of which parts of my application are most often used, so that I can in turn focus on improving the coverage in those areas. I'm trying to avoid just writing tests for areas that don't yet have any, as they may be edge cases that aren't often executed.
Even static analysis tools which do try to figure out what happens at run-time do not usually try to estimate how often a piece of code is executed. The subject is hard enough as it is!
But dynamic analysis tools (e.g. profiling tools that either rely on transparent instrumentation of the code or use sampling) can tell you, after one or several "typical" executions (you provide the entries that you judge typical), how often this or that function was executed.
See Profiling (computer programming) on Wikipedia.
If I understood the question right, you are looking for a profiler. Give EQATEC Profiler a try. It's free.
It's originally intended to profile an application before shipping (to detect bottlenecks by measuring execution time of methods etc.) so I'm not sure if it's suitable for an application in a productive environment. At least it changes your code for profiling purposes and that might be unwanted. You should check this out.
Code coverage appears to be what you want.
NCover is a popular code coverage tool for .NET, if you can afford it.
What you're asking for is simply impossible to do accurately. The number of times something is executed can and usually will depend on the data that's entered at run-time. The best you can hope for from a static analysis tool isa direct answer when statically determinedAn O(N) style analysis otherwise
Even the latter would be quite difficult to get right overall. For example, it would need to know the complexity of essentially every function in the (huge and ever-expanding) .NET library. Some of those are hard to even characterize.
Just for example, how long does it take to allocate a block of memory? Well, usually it's usually nearly constant time -- but it's always possible that an allocation can trigger a garbage collection cycle, in which case the time taken will be (roughly) proportional to the number of objects still in use that were allocated since the last GC cycle...
"Profiler" is what you're looking for; which you choose is up to you.
I've used HP's Diagnostic Server to do this, although it'd cost money. It'll tell me what methods are called how many times, and the average and worst-case time spent in them.
As an important safety tip, running a profiler will slow down the execution of your code; it's not ideal for a long-term installation into a production environment.
If you want to see just what is used:
SD C# Test Coverage Tool
If you want to see how often it is used:
SD C# Profiler Tool
If coverage is not what you look for then you could look use two things:
As suggested a profiler and run predictive test scenarios.
Use performance counters which would end in a more permanent solution. These are quite difficult to implement but are helpful to diagnose performance deltas by analyzing the counters report. One way to implement them would be to wrap boundaries and manage counters from those wrap. Be wary that it's way much easier to integrate in a new project than in an existing one.
I'm currently writing a library that can be used to show the internal state of some running code (mainly fields and properties both public and private). Objects are accessed in a different thread to put their info into a window for the user to see. The problem is, there are times while I'm walking a long IList in which its structure may change. Some piece of code in the program being 'watched' may add a new item, or even worse, remove some. This of course causes the whole thing to crash.
I've come up with some ideas but I'm afraid they're not quite correct:
Locking the list being accessed while I'm walking it. I'm not sure if this would work since the IList being used may have not been locked for writing at the other side.
Let the code being watched to be aware of my existence and provide some interfaces to allow for synchronization. (I'd really like it to be totally transparent though).
As a last resort, put every read access into a try/catch block and pretend as if nothing happened when it throws. (Can't think of an uglier solution that actually works).
Thanks in advance.
The only way you're going to keep things "transparent" to the code being monitored is to make the monitoring code robust in the face of state changes.
Some suggestions
Don't walk a shared list - make a copy of the list into a local List instance as soon (and as fast) as you can. Once you have a local (non-shared) list of instances, noone can monkey with the list.
Make things as robust as you can - putting every read into a try/catch might feel nasty, but you'll probably need to do it.
Option number 3 may feel ugly, but this looks to be similar to the approach the Visual Studio watch windows use, and I would choose that approach.
In Visual Studio, you can often set a watch on some list or collection and at a later point notice the watch simply displays an exception when it can't evaluate a certain value due to user or code state changes.
This is the most robust approach when dealing with such an open ended range of possibilities. The fact is, if your watching code is designed to support as many scenarios as possible you will not be able to think of all situations in advance. Handling and presenting exceptions nicely is the next best approach.
By the way, someone else mentioned that locking your data structures will work. This is not true if the "other code" is not also using locks for synchronization. In fact both pieces of code must lock the same synchronization object, very unlikely if you don't control the other code. (I think you mention this in your question, so I agree.)
While I like Bevan's idea of copying the list for local read access, if the list is particularly large, that may not be a truly viable option.
If you really need seamless, transparent, concurrent access to these lists, you should look into the Parallel Extensions for .NET library. It is currently available for .NET 2.0 through 3.5 as a CTP. The extensions will be officially included in .NET 4.0 along with some additional collections. I think you would be interested in the BlockingCollection from the CTP, which would give you that transparent concurrent access you need. There is obviously a performance hit as with any threaded stuff that involves synchronization, however these collections are fairly well optimized.
As I understand, you don't want to have ANY dependency/requirement on the code being watched or enforce any constrains on how the code is written.
Although this is my favourite approach to code a "watcher", this causes you application to face a very broad range of code and behaviours, which can cause it to crash.
So, as said before me, my advice is to make the watcher "robust" in the first step. You should be prepared for anything going wrong anywhere in your code, because considering the "transparency", many things can potentially go wrong! (Be careful where to put your try/catch, entering and leaving the try block many times can have a visible performance impact)
When you're done making your code robust, next steps would be making it more usable and dodging the situations that can cause exceptions, like the "list" thing you mentioned. For example, you can check the watched object and see if it's a list, and it's not too long, first make a quick copy of it and then do the rest. This way you eliminate a large amount of the probability that can make your code throw.
Locking the list will work, because it is being modified, as you've observed via the crashing :)
Seems to me, though, that I'd avoid locking (because it seems that your thread is only the 'watcher' and shouldn't really interrupt).
On this basis, I would just try and handle the cases where you determine missing things. Is this not possible?
In my website I have a rather complicated business logic part encapsulated in a single DLL. Because CPU usage hits the roof when I call certain methods of it I need to measure and optimize.
I know about the performance profiler, but I don't know how to set it up for a library.
Furthermore I don't seem to find any usefull resources about it.
How do you approach this problem?
You can create a simple exe that runs the main methods of your library.
Although it requires some understanding to know which method to call it can help you focus on speciifc scenarios and check their bottlenecks.
You can also put some performance counters: look into msdn, or open a debugger and use old system: create a Stopwatch and do Debug.Writeline to see what's happening.
As Dror says, run it stand-alone under a simple exe. I would add, run THAT under the IDE, and while it's being slow, just pause it, several times, and each time look in detail at what it's doing. It's counterintuitive but very effective.