We Keep Coding

"On-the-run" Debugging/Monitoring

Is there a way/system to debug/monitor code without stopping execution?
In industrial automation control programming (PLC/PAC/DCS) it is possible to connect the debugger while the program is running, and see in the code editor the value of variables and expressions, without setting breakpoints or tracepoints.
As an example, let's have a F# multithreaded application, where code is executed in a continuous loop or triggered by timers. Is there a way to attach a debugger like Visual studio Debugger and see the values of variables and expressions (in the code editor or in a watch pane) WITHOUT interrupting the execution?
It doesn't matter if it's not synchronous, it's acceptable if the debugger/monitor does not capture all the code scans.
I am tasked to create an high level controller for a process plant and I would like to use C# or F# or even C++ with a managed or native application, instead of a PAC system. But being forced to interrupt execution to debug is a huge disadvantage in this kind of application.
UPDATE
First of all thanks to all for their answer.
Based on those answers, though, I realized that probably I need to reformulate my question as follows:
Is anyone aware of any library/framework/package/extension that allows to work with a native or managed application in windows or linux (C#, F# or C++) the exact same way as a PAC development platform, specifically:
1) Put the dev platform in "status" mode, where it shows automatically the runtime value for variables and expressions present in the code exceprt currently visible, without interrupting execution?
2) Create watch windows that show the runtime value of variables and expressions, again without interrupting execution?
Also, what I am looking for is something that (like any PAC platform) offers these features OUT OF THE BOX, without requiring any change in the application code (like adding log instructions).
Thank you in advance
UPDATE 2
It looks like there is something (see http://vsdevaids.webs.com/); does anyone know whether they are still available somewhere?
UPDATE 3
For those interested, I managed to download the last available release of VSDEVAIDS. I installed it and looks working, but it's pointless without a licence and couldn't find information on how to reach the author.
http://www.mediafire.com/file/vvdk2e0g6091r4h/VSDevAidsInstaller.msi
If somebody has better luck, please let me know.

this is a normal requirement - needing instrumentation / diagnostic data from a production system. Its not really a debugger. Its usually one of the first things you should establish in your system design.
Not knowing your system at all its hard to say what you need but generally they fall into 2 categories
human readable trace - something like log4net is what I would recommend
machine readable counters etc. Say 'number of widget shaving in last pass',..... This one is harder to generalize, you could layer it onto log4net too. Or invent your own pipe

With regards to your edited question, I can almost guarantee you that what you are looking for does not exist. Consequence-free debugging/monitoring of even moderate usefulness for production code with no prior effort? I'd have heard of it. Consider that both C++ and C# are extremely cross-platform. There are a few caveats:
There are almost certainly C++ compilers built for very specific hardware that do what you require. This hardware is likely to have very limited capabilities, and the compilers are likely to otherwise be inferior to their larger counterparts, such as gcc, clang, MSVC, to name a few.
Compile-time instrumentation can do what you require, although it affects speed and memory usage, and even stability, in my experience.
There ARE also frameworks that do what you require, but not without affecting your code. For example, if you are using WPF as your UI, it's possible to monitor anything directly related to the UI of your application. But...that's hardly a better solution than log4net.
Lastly, there are tools that can monitor EVERY system call your application makes for both Windows (procmon.exe/"Process Monitor" from SysInternals) and Linux (strace). There's very little you can't find out using these. That said, the ease of use is hardly what you're looking for, and strictly internal variables are still not going to be visible. Still might be something to consider if you know you'll be making system calls with the variables you're interested in and can set up adequate filtering.
Also, you should reconsider your "No impact on the code" requirement. There are .NET frameworks that can allow you to monitor an entire class merely by making a single function call during construction, or by deriving from a class in the framework. Many modern UIs are predicated on the UIs being able to be notified of any change to the data they are monitoring. Extensive effort has gone into making this as powerful and easy as possible. But it does require you to at least consider it when writing your code.

Many years ago (think 8 bit 6502/6809 days) you could buy (or usually rent, I seem to remember a figure of £40K to purchase one in the late 80s) a processor simulator, that would allow you replace the processor in your design with a pin compatible device that had a flying lead to the simulator box. this would allow things like capturing instructions/data leading up to a processor interrupt, or some other way of stopping the processor (even a 'push button to stop code' was possible). You could even step-backwards allowing you to see why an instruction or branch happened.
In these days of multi-core, nm-technology, I doubt there is such a thing.

I have been searching for this kind of features since quite a long time with no luck, unfortunately. Submitting the question to the StackOverflow community was sort of a "last resort", so now I'm ready to conclude that it doesn't exist.
VSDevAids (as #zzxyz pointed out) is not a solution, as it requires significant support from the application itself.
Pod cpu emulators (mentioned by #Neil) aka in-circuit emulators (ICE) and their evolutions are designed to thoroughly test the interaction between firmware and hardware, not so useful in high level programming (especially if managed like .NET).
Thanks for all contributions.

Cleanup before termination?

This question has been bugging me for a while: I've read in MSDN's DirectX article the following:
The destructor (of the application) should release any (Direct2D) interfaces stored...
DemoApp::~DemoApp()
{
SafeRelease(&m_pDirect2dFactory);
SafeRelease(&m_pRenderTarget);
SafeRelease(&m_pLightSlateGrayBrush);
SafeRelease(&m_pCornflowerBlueBrush);
}
Now, if all of the application's data is getting released/deallocated at the termination (source) why would I go through the trouble to make a function in-order-to/and release them individually? it makes no sense!
I keep seeing this more and more over the time, and it's obviously really bugging me.
The MSDN article above is the first time I've encountered this, so it made sense to mention it of all other cases.
Well, since so far I didn't actually ask my questions, here they are:
Do I need to release something before termination? (do explain why please)
Why did the author in MSDN haven chosen to do that?
Does the answer differ from native & managed code? I.E. Do I need to make sure everything's disposed at the end of the program while I'm writing a C# program? (I don't know about Java but if disposal exists there I'm sure other members would appreciate an answer for that too).
Thank you!

You don't need to worry about managed content when your application is terminating. When the entire process's memory is torn down all of that goes with it.
What matters is unmanaged resources.
If you have a lock on a file and the managed wrapper for the file handler is taken down when the application closes without you ever releasing the lock, you've now thrown away the only key that would allow access to the file.
If you have an internal buffer (say for logging errors) you may want to flush it before the application terminates. Not doing so would potentially mean the fatal error that caused the application to end isn't logged. That could be...bad.
If you have network connections open you'll want to close them. If you don't then the OS likely won't do it for you (at least not for a while; eventually it might notice the inactivity) and that's rather rude to whoever's on the other end. They may be continuing to listen for a response, or continuing to send you information, not knowing that you're not there anymore.

Now, if all of the application's data is getting released/deallocated
at the termination (source) why would I go through the trouble to make
a function in-order-to/and release them individually?
A number of reasons. One immediate reason is because not all resources are memory. Only memory gets reclaimed at process termination. If some of your resources are things like shared mutexes or file handles, not releasing those resources could mess up other programs or subsequent runs of your program.
I think there's a more important, more fundamental reason though. Not cleaning up after yourself is just lazy, sloppy programming. If you are lazy and sloppy in cleanup at termination, are you lazy and sloppy at other times? If your tendancy is to be lazy and sloppy and only override that tendancy in specific areas where you're cognizant of potential problems, then your tendancy is to be lazy and sloppy. What if there are potential problems you're not cognizant of? How can you rely on your overall philosophy of lazy, sloppy programming to write correct, robust programs?
Don't be that guy. Clean up after yourself.

How to keep memory of process protected

I have a process that will have some important values in the memory. I don't want anyone to be able to read the memory of my process and obtain those values. So I tried to create a program that would look at the list of programs running and determine if any of them were "debuggers", etc. But I realized that someone could just write a quick program to dump the memory of my process. I know several process on my system have their memory protected. How could I also obtain this? (ps: I'm using C#)

Any application that runs under an user with enough privileged (eg. local administrator) can call ReadProcessMemory and read your process at will, any time, without being attached to your process debugging port, and without your processing being able to prevent, or even detect this. And I'm not even going into what is possible for a system kernel driver to do...
Ultimately, all solutions available to do this are either snake oil, or just a way to obfuscate the problem by raising the bar to make it harder. Some do make it really hard, but none make it bullet-proof. But ultimately, one cannot hide anything from a user that has physical access to the machine and has sufficiently elevated privileges.
If you don't want users to read something, simply don't have on the user machine. Use a service model where your IP is on a server and users access it via internet (ie. web services).

First of all, there will always be a way to dump the memory image of your program. Your program can only make it harder, never impossible. That said, there may be ways to 'hide' the values. It is generally considered hard to do and not worth the trouble, but there are programs which encrypt those values in memory. However, to be able to use them, they need to decrypt them temporarily and re-encrypt (or discard) them afterwards.
While encryption is easy with the .Net framework, discarding the decrypted value is not an easy thing to do in C#. In C, you would allocate a chunk of memory to store the decrypted values and clear that (by writing zero's or random data to it) before freeing it. In C#, there is no guarantee that your data won't be stored somewhere (caching, garbage collection) and you won't be able to clear it. However, as eulerfx noted in a comment, in .Net 4.0 SecureString may be used. How safe that is, I don't know.
As you may see, there will always be a short time where the value lies in memory unencrypted, and that is the vulnerability here.

I think the best way to do it is employ a commercial solution such as in this PDF brochure, this is their website. You may be better off going down this route if you really care about protecting the application from sniffing, IP theft etc instead of rolling up your own solution...
Edit: I would not go down the route in kidding myself that the solution I shall craft up will be tamper proof, crack proof, idiot proof. Leave that to a company like Arxan I mentioned (I aint a sales rep - I'm just giving an example), sure it might be costly, but you can sleep better at night knowing it is much harder for a cracker to break than having no solution at all...

How can I retrieve information about disk volumes?

I was looking around for a tool to monitor the status of my software raid under Windows 2003 Server, but couldn't find anything suitable (i.e. not grossly oversized or needlessly complicated). So I decided to just do it myself, it's nothing spectacularly difficult.
So how can I retrieve the status of the volumes programmatically? It's been a while since I fiddled with the Windows API and I couldn't find anything right off the bat using Google. I know I can use diskpart /s and parse its output, but that gets messy fairly quickly (although it does have some advantages).
Any pointers into the right direction are highly appreciated :)

The Win32 API is the (apparently only) way to go here, Virtual Disk Service is the magic word.
Here is a good example in C++ that will get you started. The number of different COM interfaces was pretty confusing for me at first, but the How Virtual Disk Service Works article was of great help getting the big picture.
It's actually pretty easy. Despite never having done any serious C++ coding and never having even touched COM before, I was still able to get the basic functionality to work in a few hours.

Did you check WMI?
You can take a look here for a demo.

You could try monitoring the Event log for RAID events.

I don't know if RAID stuff will complicate matters, but I've used System.IO.DriveInfo.GetDrives() before and that's worked fine for my needs.

Concurrency issues while accessing data via reflection in C#

I'm currently writing a library that can be used to show the internal state of some running code (mainly fields and properties both public and private). Objects are accessed in a different thread to put their info into a window for the user to see. The problem is, there are times while I'm walking a long IList in which its structure may change. Some piece of code in the program being 'watched' may add a new item, or even worse, remove some. This of course causes the whole thing to crash.
I've come up with some ideas but I'm afraid they're not quite correct:
Locking the list being accessed while I'm walking it. I'm not sure if this would work since the IList being used may have not been locked for writing at the other side.
Let the code being watched to be aware of my existence and provide some interfaces to allow for synchronization. (I'd really like it to be totally transparent though).
As a last resort, put every read access into a try/catch block and pretend as if nothing happened when it throws. (Can't think of an uglier solution that actually works).
Thanks in advance.

The only way you're going to keep things "transparent" to the code being monitored is to make the monitoring code robust in the face of state changes.
Some suggestions
Don't walk a shared list - make a copy of the list into a local List instance as soon (and as fast) as you can. Once you have a local (non-shared) list of instances, noone can monkey with the list.
Make things as robust as you can - putting every read into a try/catch might feel nasty, but you'll probably need to do it.

Option number 3 may feel ugly, but this looks to be similar to the approach the Visual Studio watch windows use, and I would choose that approach.
In Visual Studio, you can often set a watch on some list or collection and at a later point notice the watch simply displays an exception when it can't evaluate a certain value due to user or code state changes.
This is the most robust approach when dealing with such an open ended range of possibilities. The fact is, if your watching code is designed to support as many scenarios as possible you will not be able to think of all situations in advance. Handling and presenting exceptions nicely is the next best approach.
By the way, someone else mentioned that locking your data structures will work. This is not true if the "other code" is not also using locks for synchronization. In fact both pieces of code must lock the same synchronization object, very unlikely if you don't control the other code. (I think you mention this in your question, so I agree.)

While I like Bevan's idea of copying the list for local read access, if the list is particularly large, that may not be a truly viable option.
If you really need seamless, transparent, concurrent access to these lists, you should look into the Parallel Extensions for .NET library. It is currently available for .NET 2.0 through 3.5 as a CTP. The extensions will be officially included in .NET 4.0 along with some additional collections. I think you would be interested in the BlockingCollection from the CTP, which would give you that transparent concurrent access you need. There is obviously a performance hit as with any threaded stuff that involves synchronization, however these collections are fairly well optimized.

As I understand, you don't want to have ANY dependency/requirement on the code being watched or enforce any constrains on how the code is written.
Although this is my favourite approach to code a "watcher", this causes you application to face a very broad range of code and behaviours, which can cause it to crash.
So, as said before me, my advice is to make the watcher "robust" in the first step. You should be prepared for anything going wrong anywhere in your code, because considering the "transparency", many things can potentially go wrong! (Be careful where to put your try/catch, entering and leaving the try block many times can have a visible performance impact)
When you're done making your code robust, next steps would be making it more usable and dodging the situations that can cause exceptions, like the "list" thing you mentioned. For example, you can check the watched object and see if it's a list, and it's not too long, first make a quick copy of it and then do the rest. This way you eliminate a large amount of the probability that can make your code throw.

Locking the list will work, because it is being modified, as you've observed via the crashing :)
Seems to me, though, that I'd avoid locking (because it seems that your thread is only the 'watcher' and shouldn't really interrupt).
On this basis, I would just try and handle the cases where you determine missing things. Is this not possible?