I have a ASP.NET (C#) website that uses a third party DLL to process the data that the users POST via a web form. The call is pretty straightforward:
string result = ThirdPartyLib.ProcessData(myString);
Once in a blue moon this library hangs and (according to my hosting provider logs) consumes 100% of CPU. The website is hosted on a shared hosting, so I have no access to the IIS or event logs. When this happens, my website is automatically stopped by the hosting provider performance monitor, and I have manually switch it back on.
Now, I know that the right thing to to is investigate the problem and fix (or replace) the DLL. But as it's third-party software, I am unuable to fix it, and their support is not helpful at all. Moreover, I can't reproduce the problem. Replacing the library is a pain too.
Is there a way in C# to detect when this DLL starts consuming 100%CPU and kill the process automatically from my ASP.NET code?
You cannot "detect" if the current process is hanging because as the caller of a method (third party or not) you're simply not in control until it returns.
What you can do is move the call to the third party library into a separate executable and have it output its result via the standard output (you can simply use Console.WriteLine(string) for this).
Once you've done that, you can start a separate Process that runs this executable, read the result via StandardOutput and use WaitForExit(int) to wait a certain amount of time (maybe a few seconds) for the process to finish. The return value of WaitForExit() tells you if the process actually exited. In case it didn't, you can Kill() it and move on without IIS worker process hanging as a whole.
Related
I am working on a "debug dispatcher" C# program that is a debug assistance tool. This is not a new application; it has been a part of this project and invaluable to debugging it for some time. However, it has some limitations, which I have been trying to address to enable a more complete debugging experience.
This debug dispatcher takes the place of a system service that accepts requests to launch applications, and its purpose is to permit an attached debugger to automatically interact with code that would ordinarily be launched in a child process. The child processes are themselves .NET applications.
When this tool was made (years ago), the first thing that was investigated was whether there might be any way to launch a child process with the current debugger already attached to it. None was found, and so instead the tool creates an independent AppDomain within which to launch each process, then loads the application as an assembly and calls its entry method. This is almost working perfectly, but the problem I'm running into is that if those child processes call Environment.GetCommandLineArgs, they get the debug dispatcher tool's command-line instead of the command-line intended to be passed into a child process.
I have been trying to find a way to override Environment.GetCommandLineArgs.
Based on the publicly-available source code, it looks like if my application were .NET Core, there would in fact be an internal method SetCommandLineArgs I could invoke via reflection. The fact that this is internal isn't particularly troubling to me as this tool is specifically a debug assistant; if it happens to break down the road because the implementation changed, so be it. It serves no purpose whatsoever outside of a debugging context and won't ever be on a non-dev machine. But... .NET Core and .NET 5 don't support AppDomains at all, and never will, so that's a non-starter.
I have tried using Ryder to redirect Environment.GetCommandLineArgs to my own implementation, but it doesn't seem to work, even with a .ini file specifying a [.NET Framework Debugging Control] section with AllowOptimize=0. It almost looks as though the JIT has special handling for this specific method, because even though the reference source shows it making an icall into a native method, when I request disassembly of the JIT output in the debugger, it shows no calls at all, simply loading a value directly from an inlined memory address.
I searched for ways to change the current process's command-line at the Win32 level, but that appears to be unmodifiable.
In the context of supporting multiple concurrent applications inside the same process by means of AppDomains (solely for assisting debugging), is there any way to intercept and/or override the return value of Environment.GetCommandLineArgs, so that I can support hosting applications that obtain their command-line arguments exclusively via that method?
Okay, well, I'm not sure what I did that changed it, but at some point redirecting Environment.GetCommandLineArgs using Ryder seemed to go from being unreliable (some calls would redirect, others wouldn't -- in some debug sessions, Ryder seemed to have no effect at all) to reliable (every call gets redirected). Ryder's redirection apparently doesn't automatically apply in all AppDomains, so I have to reinstall it each time I create an AppDomain, after which my experience has been that the process dies a messy death if I try to unload the AppDomain. But, for debug purposes... I think it's adequate.
So this is a weird one.
I created a WPF application using MahApps for the GUI. So far my testing indicates that the app works fine on several different machines. Of course this is not the case on the client's machine.
The client makes use of Terminal Services and Windows Server 2008R2. Several users can be logged into their own version of the server at anytime. The app starts up fine once or twice, but after a day or so, it no longer opens up.
The app doesn't show up in the Application tab of Task Manager, but its process can be seen to be running in Processes Tab of Task Manager.
To be honest, I'm completely stumped. I had a look at the event manager log and couldn't find anything indicative of a problem. (Of course I might have missed something). I saw another SO question suggesting to disable hardware acceleration, but I'm not if that would help.
Any and all ideas would be greatly appreciated.
EDIT:
I thought I might mention the only thing that helps is if we restart the client machine.
EDIT:
I think I have isolated the issue to integration with Twain (should probably have mentioned that as another possible factor). I think the Twain library (unmanaged code) somehow stalls without sending back an error. Disabling it has "fixed" the issue.
This somehow relates to Twain and multi-session setups. I'm almost sure of it.
First you can analyze the wait chain in Windows Resource Monitor to check if there are any resources the process is waiting for. (You can find more information about the wait chain here or here.)
If you don't find any viable suspects there, you can create a memory dump of the hanging process and analyze the call stacks. If you don't know how to create one, you can read about it here. If you want to use Windows Task Manager and your OS is 64-bit then please be aware that you need to use the same bitness of Task Manager as the application.
That is: If your application is 64-bit then you have to use C:\Windows\System32\taskmgr.exe and if it's 32-bit you have to use C:\Windows\SysWOW64\taskmgr.exe. If you forget this important step you'll just get an unusable dump full of gibberish.
After you got the memory dump you can either load it into WinDbg (using the same bitness as the application) or Visual Studio (best to use 2015 or later) and analyze the call stacks of all running threads.
You can download WinDbg here and read about the necessary WinDbg configuration here. For the list of all threads you need to use this SOS command.
If you need help in loading memory dumps into Visual Studio you can find more information here.
After you've looked at the call stacks you most definitely find the answer what is waiting on what resources and is thus preventing the shutdown or startup of the application. It can either be a classic deadlock or an external resource like writing/reading of a file or some other waiting without a timeout like accessing a database or an URL that can't be reached at the moment. And of course it can also be just an infinite loop - if it doesn't consume much CPU then perhaps with some kind of DoEvents in between.
And last but very not least: If you are really interested what can be analyzed if an application hangs you can read about an example analysis done by the absolutely awesome great Mark Russinovich here.
I'm fixing bugs on an application, that is kind of data consumer/worker, getting data from third party application, using supplied API and libraries for doing so. It's c++ based API and the .net application is using a bit of c++ to access the libraries. Also - the application is multi-threaded, it's windowed (Winforms), uses several third party libraries (nhibernate, mysql and others). It might be relevant to add, that our consumer thread is the only place in the code, when it accesses the c++ library.
The problem? When the producent application is closing (takes a bit more time, more than a minute), consumer application dies within seconds, without error/exception - even thought they're opened independently. No information in Event Log, no Dr. Watson action, no exceptions in Visual Studio (debug just stops).
I've tried:
Stepping throughout the code to see the moment, where it closes, but it always happened in different places, was it calling the producent's libraries code, or not.
Debugged the application with Visual Studio configured to break on any exception throwing - but it dies without a thing.
Creating crash dumps (using ADPlus.vbs) and using windbg on them (I'm new to such low-level debugging, though), but !analyze resulted with different stack traces - leaving me traceless.
What would be the good direction to find out why the consumer application dies? Is there a way, to get around the problem (like showing a prompt message to the user, like: "Producent application is closing, consumer application will do the same!")?
[EDIT]
Consumer application is multi-threaded, and it's one consumer thread as addon to UI thread. Also - the third party app we're using as producer uses COM to send information to any consumer app (aka add-on).
Me and my coworker decided to comment out some code, to find the code, that possibly makes the problem. And probably we've found it - the application dies if and only if we've registered our consumer to producer. After reading documentation for the third party app, it turned out that consumer apps have to actively query for message of closing the producer, otherwise they would be forcefully terminated by the producer app.
So: 95% that the problem is third party application which we're querying for data is sending COM message to forcefully terminate our application (I'll post info / change to wiki, if we'd test it's the only reason).
The general scenario described here is a source for a very common confusion and misunderstanding related to cases where one tries to understand 'how come my application vanished into thin air without leaving any trace?'.
The immediate assumtion would be: my application 'died' or 'crashed' or 'encountered such unexpected exception, which is even not visible to the debugger and thus did not create any dump-file. Happened to me few good times...
The real answer in most cases would be that the application did not realy crash or die and did not receive any excpetion, but was simply shutted-down gracefully, but from a flow that I did not expect.
The easiest way to debug such cases will be to put a breakpoint in kernel32!ExitProcess and to follow the stack and see how we got here.
Hope this helps
It turns out, that its the host application, that kills my application. The proper way to debug the problem was to spy on windows messages and to see, that my application is getting Process Terminate message.
I have a console application that writes on a txt files information retrieved from a database. Until now I manually executes the executable generated by the console application.
Now I need to automatize the invocation of the .exe from my web application, so that each time a specific condition happens in my code behind I can run the .exe with a logic "fire and forget".
My goals are:
1) Users must not be affected in any way by the console application execution (the SQL queries and txt file generation might take around 3 to 5 minutes), therefore the logic of "fire and forget" delegated to a separate process.
2) Since the executable will be still run manually in some cases, I would prefer having the all logic in one place, in order to avoid the risk of having a different behaviour.
Can I safely use System.Diagnostics.Process to achieve this?
System.Diagnostics.Process cmd = new System.Diagnostics.Process();
cmd.Start("Logger.exe");
Does the process automatically ends or do I have to set a timeout and explicitly close it? Is it "safe" in a web application environment with different users accessing the web application let them call the executable without the risk of concurring accesses?
Thanks.
EDIT:
Changed to use the built in class for more clarity, thanks for the hint.
As far as the mechanics, I assume CommandLineProcess wraps Process? If so, I don't see anything necessarily wrong with it, at first glance. I just have some issue with running this as an executable from a web application, as you are more likely to reduce security to get it working than rearchitect (if you follow the normal path I see in development).
If you encapsulate the actual business code in a class library, you can run the code in the web application. The main rule is the folder it saves to should be under webroot (physically or logically) so you don't have to reduce security. But, if the logic is encapsulated, you can run the "file creeator" in the web process without spinning up a Process.
Your other option is wrap the process in a service (I like a non-HTTP WCF service, but you can go windows service, if you want). I would only go this direction if it makes sense to follow a SOA path with a service endpoint. As this is likely to be isolated to a single application, in process makes more sense (unless you are saving to a directory outside of webroot).
Hope this makes sense.
Yes, it will die on it's own - provided that the .exe file will terminate on it's own. It will run with the same credentials of the web server.
Keep in mind this is considered unsafe, since you are executing code based on whatever your webapp is doing. However, the problem is with .exe files being executed this way in general and not with the actual users accessing the app.
Similar question here How do I run a command line process from a web application?
I have to create a custom download manager that will replace a standard download manager in Internet Explorer. After googling I've learned that I have to create a COM component that implements the IDownloadManager interface.
As far as I understand I have to create a dll, generate guid for it and register it using regasm.exe utility, and then add specific entry in windows registry for IE.
I have a few questions:
I want my program to be an exe and I want to be able to run it manualy and add url to it as well as run it by IE after clicking on a downloadable link.
Although I would prefer to have a single executable, I think to achieve this i have to create dll and exe, and from dll i should check whether the exe is running (by window id) and run if it isn't and communicate with it somehow. Is this correct approach?
I want to share my program with other users, and i don't want them to register COM manually. Is it possible to do it from the code? Or perhaps I should create an installer (which I would like to avoid)?
I'll start with a WARNING: Do not create a .Net components that will be loaded in IE. Ask yourself the question "What would happen if another app does the same, and it uses different version of the CLR?". IE does not guarantee any order of loading the different COM components it needs, so there's no guarantee that your version of the CLR will be loaded in the process by the time IE calls you.
Now onto your problem. There are several issue with your scenario:
.Net does not support creating out-of-proc COM components natively. Yes, it is possible to create one by doing bunch of hacks and manual registration; however, it is not a simple task and requires deep knowledge of how COM works;
with the above in mind, your option is really to create a .Net DLL and use the ComVisible attribute to expose the classes you need to COM. As you mentioned it, you will need to register it using RegAsm.exe, for IE to be able to use it;
since you want the main functionality of your download manager to be in a standalone executable, you will have to use a .Net supported cross-process communication mechanism. .Net Remoting is likely the easiest way to implement it, and should for the most part meet your requirements. The alternative is to implement the download functionality in-proc. However, beside the consideration that you now could easily hose the IE process, if you are not careful to listen to its quit notification (which require a lot more work by itself), there's also the whole enchilada with the IE7+ protected mode, which severely limits what your in-proc code can do (limited file access, registry access, Windows APIs and other limitations);
there are certain complications arising from the IE8 and IE9 process model. Besides the top frame process, IE8/9 create a pool of processes and load-balance the tabs into these. I don't know which process will try to create your COM component and wheter it's going to be one per tab or per process or for the whole IE session (which spans multiple processes), so you have to be prepared that you might have multiple instances in multiple processes running concurrently. If this is the case, you will have to figure out how to ensure that the communication between the in-proc COM component and the executable is not serialized one instance at a time, or you might affect the browsing experience for the user. (A simple scenario would be a page with multiple download links and the user right-clicking on each link and selecting Open in new tab, thus launching multiple downloads in several tabs at once);
even if there is one instance per IE session, elevated IE instances run in a separate session from the regular user IE instances for security reasons. There's the interesting complication that your .Net Remoting call from the in-proc COM component in the elevated IE session will result in a second copy of your executable being launched also elevated. Thus, your download manager will have to be prepared that there might be two processes accessing the same download queue;
starting with IE7, IE protected mode (the default) will intercept any calls that result in starting a new process and show a dialog to the user. The only way to avoid this would be to register a silent IE elevation policy for your process. The elevation policies are registered in HKEY_LOCAL_MACHINE, which means that you will need an installer, or at least a simple script for the users to run as administrator;
even if you decide against the elevation policy and to live with the bad experience of this dialog, to register your download manager with IE, you still will have to write to the HKEY_LOCAL_MACHINE registry hive, otherwise IE will not know of it and won't use it. In other words, you still need some kind of installer or a deployment script;
IE is fairly aggressive in measuring the performance of the code that runs on the UI thread and in terminating background threads when exiting the process. So whatever functionality you have in the in-proc component, you will have to balance between being as fast as possible on the UI thread (which means less work or you'll impact the user experience) and doing work on the background threads (which means be prepared you might be killed without notification at any moment);
I think this list covers the main issues you will have to solve. The biggest problem you will encounter is that a lot of the specifics around IE process model are not well documented on MSDN, and there are almost no examples of implementing this scenario in managed code (and of those that exist, most are old and are not updated for IE8/IE9, and some even won't work in IE7).