Error handler in Azure Websites does not log all exceptions - c#

I have a site, that uses StackExchange.Exceptional to log exceptions. Problem is, that it does not log all exceptions, like on my development machine.
If it would not log any exceptions, I would think that some configuration/permissions are not working on Azure Websites, may be module is not loaded/working, but for example if I misstype controller action, I get an exception logged:
A public action method 'Logi' was not found on controller 'Acme.Controllers.AccountController'.
I've tried running site using release configuration localy, it works without problem.
As I understand, StackExchange.Exceptional is http module, that is logging unhandled exceptions using this code:
protected virtual void OnError(object sender, EventArgs args)
{
var app = (HttpApplication)sender;
var ex = app.Server.GetLastError();
LogException(ex, app.Context);
}
(https://github.com/NickCraver/StackExchange.Exceptional/blob/master/StackExchange.Exceptional/ExceptionalModule.cs#L36)
May be in some cases on Windows Azure it does not receive OnError event, any ideas what to check, where to search for a clue?

I've managed to replicate the same behaviour on local machine with added this setting:
<customErrors mode="On">
As long as RemoteOnly mode is default, I wasn't seeing problem on local development machine. Not sure why, but when customErrors are enabled, some exceptions were not logged through http handler.

Related

Could not start Windows Service, Error 1064

I wrote a Windows Service to run on Win10, and it worked perfectly fine until I decided to change it a bit. I rewrote some logic, tested it in both Debug and Release configurations, and everything was fine. Then I uninstalled the current version of the service using installutil.exe /u %servicename.exe% and reinstalled it again using installutil.exe %servicename.exe%.
For some reason, this new version cannot start, and it crashes with Error 1064. This is the full error text:
Windows could not start %servicename% service on Local Computer. Error 1064: An exception occurred in the service when handling the control request.
The last time I installed this service, I ran into some difficulties, but quickly fixed them by changing the Log On properties. This time, it is not working. Please help with this issue.
Thanks.
Update 1
Here are my Main() and OnStart() service methods:
Main()
static void Main()
{
#if DEBUG
var service = new SalesforceToJiraService();
service.OnDebug();
Thread.Sleep(Timeout.Infinite);
#else
ServiceBase[] ServicesToRun;
ServicesToRun = new ServiceBase[]
{
new SalesforceToJiraService()
};
ServiceBase.Run(ServicesToRun);
#endif
}
OnStart()
protected override void OnStart(string[] args)
{
this.ConfigureServices();
this.timer.Start();
this.logger.Information("SalesforceToJira service started.");
}
Update 2
More code:
ConfigureServices()
protected void ConfigureServices()
{
this.configuration = ConfigurationHelper.LoadConfiguration(ConfigurationPath);
this.logger = ConfigurationHelper.ConfigureLogger(this.configuration.Logs.LogsPath);
this.timer = ConfigurationHelper.ConfigureTimer(this.configuration.ProcessInterval.TotalMilliseconds,
(sender, eventArgs) => this.ProcessCasesAsync(sender, eventArgs).GetAwaiter().GetResult());
this.salesforceClient = new SalesforceCliClient(this.configuration.Salesforce.CliPath);
this.jiraClient = Jira.CreateRestClient(
this.configuration.Jira.Url,
this.configuration.Jira.Username,
this.configuration.Jira.Password);
}
I'm using Newtonsoft.JSON for deserializing a JSON configuration file, Serilog for logging, System.Timers.Timer for periodic events, AtlassianSDK for the Jira API and some wrappers over Salesforce CLI for Salesforce.
Thanks to #Siderite Zackwehdex's comment, I was able to find the full stack trace of the underlying exception in EventViewer, under:
Windows Logs\Application
In my case, my service is named "HttpDispatcher", which appears in the "Source" column in the top pane.
I could see immediately it was due to a dependency issue where my .NET 4.7.2 project was not pulling across my .NET Standard references. (That ol' chestnut).
I faced the same issue. The reason was I forgot to set the Database connection properly in configurations.
I had this exact same error 1064 starting my service. For me the user I had the service registered as was not a valid user in the database. Once added, it worked great.
I also had the same error in my Windows Service.
The reason was it can't read a configuration parameter, so it crash.
Adding some validation (bugfixing), the Windows Services can start it correctly.
In my case the error was due to issues with Event log name
It got fixed after I went to RegEdit and deleted old service name from HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services
I have also faced this issue. In my case it is due to connection fail with the data base. I think it is due to code throw the exception.
My error :
Windows could not start the service1 service on local computer.
Error 1064: An exception occured in the service when handling the control request
I corrected my issue by updating the third party DLL.
I faced the same issue, here is how I resolved it after troubleshooting.
If you are running service on the Server with multiple users, make
sure to run the service as admin user. Click on the service
properties and then on Log on tab click on this account and provide
the admin user name and password.
And If your service is accessing some shared drive, then make sure
you have a general user on all servers for accessing the shared
drives and add the user as local admin as well.
For me it happened when I tried to restart a process. Turned out the process was hanging in 'Stopping' so I had to kill it manually via command line and the PID.

Unexpected exit code from a .NET console application

I have written a console application and a companion class library to export some data from a cloud service. The application is called by SQL Server Integration Services which relies on the exit code returned by the application to determine if it worked correctly or not.
Intermittently the application returns an exit code of -532462766 (0xE0434352) which is the generic error code for a .NET unhandled exception. I'm totally flummoxed as to why this is happening.
The log files generated by the applications do not show any issues and they look like everything has completed successfully.
There are no entries in the Application Event Viewer logs.
The application even has an unhandled exception handler:
AppDomain.CurrentDomain.UnhandledException += UnhandledErrorHandler;
...
private void UnhandledErrorHandler(object sender, UnhandledExceptionEventArgs e) {
logWriter.Write(e.ExceptionObject.ToString(), logLevel.Fatal);
logWriter.Write("Exiting now...", logLevel.Fatal);
Dispose();
}
I've even written a batch file to execute the application and log the exit code before passing it along to SSIS. The exit codes that SSIS are receiving are the ones that seem to be returned by the application. But I cannot see an unhandled exception happening anywhere.
The console application returns the exit code by defining Main() like so:
class Program {
static int Main(string[] args) {
...
return (Success) ? 0 : 1;
}
Because it is intermittent (and the data extraction can take a couple of hours) I can't just run it in Visual Studio and debug it. I have a suspicion it might be related to the fact that the application does run for such a long time but I can't seem to confirm that.
Is there anything else that can cause a .NET application to return that exit code? Am I missing something in my troubleshooting?
quick check: wrap your entire code inside a try catch block and save the exception in a log file.
static int Main(string[] args)
{
try
{
//your existing code....
}
catch(Exception Ex)
{
//write your log results here.
}
}
Check if you are using multiple app domains. I encountered this same issue when an exception X was thrown in AppDomain B and could not cross to AppDomain A because it was not serializable. See also best practices for exceptions (search for 'across app domains'):
When you create user-defined exceptions, you must ensure that the
metadata for the exceptions is available to code that is executing
remotely, including when exceptions occur across app domains. For
example, suppose App Domain A creates App Domain B, which executes
code that throws an exception. For App Domain A to properly catch and
handle the exception, it must be able to find the assembly that
contains the exception thrown by App Domain B. If App Domain B throws
an exception that is contained in an assembly under its application
base, but not under App Domain A's application base, App Domain A will
not be able to find the exception, and the common language runtime
will throw a FileNotFoundException exception. To avoid this situation,
you can deploy the assembly that contains the exception information in
two ways:
Put the assembly into a common application base shared by
both app domains.
or
If the domains do not share a common application base, sign the assembly that contains the exception information with a strong name
and deploy the assembly into the global assembly cache.

IIS: Unexpected behavior in case of unhandled exception in Application_Start

I'm using Windows 7, IIS 7.5.7600.16385 and currently .NET 4.6.1 is installed and we have a MVC application.
Some days ago we had some strange behavior at our application. Unfortunately a service which is called inside Application_Start was not available and an unhandled exception was thrown inside. My expected behavior was that the Application_Start() is called again with the next request or the the next request is starting directly with Application_BeginRequest() like mentioned in What happens if an unhandled exception is thrown in Application_Start?.
Unfortunately I get the following result:
In case of exception inside Application_Start() I get an error 500 at the first request. That's ok.
After this all other requests are returning the exception which is thrown at the first request. I verified it by throwing an exception with timestamp inside at my local environment. Each response contains the exception with the timestamp from first request and the HTTP answer is still 500. It has no dependency which url is called. At our code no breakpoint is hit but the IIS log show the requests. It seems that the answer is cached somewhere.
Personally I like the behavior because the application doesn't respond to the requests with undefined initialization status.
And yes I know that calling other service resources inside Application_Start() is not the best idea and we will probably remove it next time :)
My Questions:
Is it possible to configure the behavior in case of an exception is thrown at Application_Start()?
Maybe somebody know when this behavior was changed or does it exists already a long time?
Well I analyzed this scenario and search through many sites, but coould not find any info about it. However, I managed to observe so behavior:
When unhandled error is thrown inside Application_Start then IIS returns error page and web app starts to shut down.
During the shutdown (in my case that was 10 sec.) any new request are handled by IIS and the response is the same as in the first request. If you think about it its logical, because IIS knows that website is shutting down so its obvious that last error cause it.
After some time application raises Application_End event to let know that shutting down is complete. After that event the next request to the website will raise Application_Start again and new response will be generated.
I don't think you can alter this behavior, because application just need some time to restart.
Today I had some time to check the behavior again. We introduced Serilog some releases ago and it seems that the configuration has an effect to the restarting behavior.
protected void Application_Start()
{
SerilogManager.Configure(); //own class
using (LogContext.PushProperty(SerilogManager.PROPERTY_NAME_ComponentName, "xxx")){}
throw new Exception(DateTime.Now.ToString("hh:mm:ss"));
}
If I remove the PushProperty line from Application_Start then the restart will work without any problem. With this line no Application_End will be called.
Now I can reproduce it on private und business computer. Not sure why my demo application didn't call Application_end on my business machine last time.

Why does my c# Windows service stop running without any messages being written to the application event log?

I am fairly new to Windows services. I created an installer for my c# Windows service and the installation on the server (Windows Server 2003) appears to have worked. When it's started, it writes Service started successfully to the log. When it's stopped, it writes Service stopped successfully. However, sometimes the service stops running without writing anything to the log, so I start it back up manually. When I look at the log afterward, it says Service started successfully as expected. It's weird seeing that in the log twice in a row being that it's obviously missing an entry where the service had somehow stopped running.
What could be the potential causes for this? I have the service set up as automatic and installed it to run for all users. I was under the impression that this means the service starts automatically whenever the machine boots up. How can I find out why it stopped? Do services that crash automatically write to the event log or do I have to handle exceptions in such a way that they log their own reason for the crash?
Edit: Some additional info:
I have it set up to log on as Local System Account
Under Recovery options, I have it set up to restart on first failure. I don't have anything for second or subsequent failures.
Update: An answerer recommended a global exception handler. While I won't implement this as a permanent fix, it will at least help me figure out where the problem is occurring. I actually tested this with my installed service and it works. I found out that unhandled exceptions actually do crash the service without writing anything to the log at all. I thought it'd at least report some application error, but it doesn't.
static void Main()
{
AppDomain.CurrentDomain.UnhandledException += new UnhandledExceptionEventHandler(CurrentDomain_UnhandledException);
//other code here
}
static void CurrentDomain_UnhandledException(object sender, UnhandledExceptionEventArgs e)
{
Utilities.WriteIt(e.ExceptionObject as Exception);
}
It's always best to handle the exceptions. At least use a global exception handler and write it to a logfile
It sounds like your service is failing unexpectedly without doing any form of exception-handling and/or logging. Windows services do not automatically write exceptions to the Event Log - it's up to you to handle exceptions and (if they're fatal) write them out somewhere so that you can diagnose the problem.
At the very least, I'd recommend a logfile somewhere (perhaps in the service executable folder, or preferably somewhere else that's easy to get to and won't run afoul of permissioning issues) and a standard logging method that all your exception-handlers call to write their messages to.
If a service quits unexpectedly because of some exception, I am not sure it would end up in the Event Log automatically.
I would highly recommend a logging suite like log4net for more thorough logging. You'll be able to provide a multitude of logging 'levels' (debug traces to see if you reached some code, info traces for important events, error traces to log exceptions).
You can look here for an example of a EventLogAppender. However, I would suggest starting with getting a FileAppender, one of the easiest logs to create, working first and then add a second appender for the Event Log.

Should the custom error set to off to be able to simply log errors?

I implemented a simple error logging as a test in a MVC website using the following code:
protected void Application_Error(object sender, EventArgs e)
{
Exception ex = Server.GetLastError();
using (StreamWriter sw = File.AppendText(Server.MapPath("\\log.txt")))
{
sw.WriteLine(string.Format("Time:{0}\n{1} \n\n\n", DateTime.Now, ex.ToString()));
}
}
The problem i am facing is that logging is not writing to the file if the customerror in the website web.config is not Off <customErrors mode="Off"></customErrors>
But I think this will let the website users see the yellow screen of death if an error happened, is there a way to log the errors the way I am doing but without setting the custom errors to off?
You will get these Yellow Screen of Dead because you are not handling the exceptions correctly. If you are logging all of the exceptions, then you have (most likely) one method that will receive all of the exceptions. Check this out so that you could better trap errors at Page Level, Application Level or in the Web.Config file.
Take a look at this if you want to see Microsoft's Best Practices for Handling Exceptions.
Good Luck!

Categories