Azure Application Insights - Custom alerts - c#

We run an hybrid application that runs on different Azure Roles (2 Web Roles + 2 Worker Roles). Last weekend something went wrong and the service went into "Unresponsive" state and stayed into that state for two days (!) without being rebooted.
We decided than to integrate Azure Application Insights because we cannot stand a 2 days down without even know.
What I'd like to have is kinda of a heartbeat of my application. One of our worker roles has different services running concurrently. I'd like to monitor if this services are running and what are their performance (on a metric defined by me, let say "number of messages processed in a minute").
I would like to receive an alert if this metric, let say, goes down (or up) a threshold. I tried with a small demo application but I couldn't do that.
What I did, with Azure Application Insights API on my C# demo app:
1. inside an infinite loop with a 10 seconds wait after each loop, tracked a StartOperation
2. inside this StartOperation, tracked a TrackMetric passing a random value from 0 to 10
3. checked if everything was working on Azure (and it was)
4. defined an alert saying that an email has to be sent if that metric was less or equal to 1 in five minutes
Nothing arrived, but everything was correctly running. Than I stopped my service, I saw events dropping in Azure, but no alert raised. Is that normal?
How do you check a case like mine?
Thanks
Marco

You might be able to use Application Insights Web Tests functionality to check if the endpoint is available from the different geo regions and alert when it's not.
If all endpoints are authenticated you may expose simple "/ping" endpoint and run web tests against this.
However, it won't work for Worker Roles out of the box unless you register it to accept "/ping" over web protocols (doable for Worker roles, e.g. one can implement a WCF service that way).

The problem is that Application Insights custom alerts are currently triggered only upon data arrival.
A strategy we’ve been using when faced with the same problem is having a separate service send out periodically the same metric but with a “zero meaning” value.
In our specific case we use an availability metric in which “1” means healthy, whereas “0” means nothing, it is just used to elicit an alert in case there was no “1” sent for the defined duration.
You can use any wide set of possible mechanisms to send out the “0” metric, as long as it is independent of the service you actually want to monitor. You need to make sure they can’t fail at the same time.
Hope this helps,
Maxim

Don't think that App Insights will let you send an alert on lack of metrics, which is what happens when your instance becomes unresponsive.
If you have the budget for external tools, look into CloudMonix. It'll do exactly what you need with using default configuration (no need for agents, custom code, etc). Disclaimer: I'm affiliated with the product

I think you should look at this.
https://azure.microsoft.com/en-us/documentation/articles/app-insights-alerts/
Thanks

Related

Windows 2016: ServiceController.DisplayName returns ServiceController.ServiceName on Windows Start

I have a service (Main), which is responsible for taking care of other Windows Services. It decides which services need to be stopped, started, etc.
The Main service controls other services by DisplayNames.
On the Windows start it reads DisplayNames from config and threats other services.
On Windows 2016 it takes some time to start services, and ServiceControl.DisplayName returns ServiceController.ServiceName, so the Main service is not able to find a service from config.
Does some workaround exist to fix this?
I think, might you can try to wait for specific status?
For example:
ServiceController.WaitForStatus(ServiceControllerStatus.Running); is going to wait until its status is "Running". It's suspended an application's processing until the service has reached the required status.
Here is a documentation.
And also, you can check out this article - ServiceController in C#.
Let me know if it worked.
Display Names, as the name suggests, are intended to be displayed. They are not meant to be compared to anything, and that used to determine a match.
For program to program interaction, you should be using the service names.
So your question is based on a bad premise - that display names are stable and may be relied upon. However distasteful it may seem, you need to re-architect your system to not rely on display names.

Web application + SQL server triggers / thresholds

We have a web application that is hosted in IIS. In our database that serves the application we have all kinds of different data values. We are trying to figure out a way to have an email sent to a client if a certain data value exists or exceeds a threshold value.
Generic Example:
Say we have a table that lists widgets and their 'in inventory' quantity. Every time someone sells a widget, this quantity value would be depleted. We want to send an email to the manager when the widget quantity gets below 5 and tell him to reorder more widgets.
We don't want to have sql triggers that check the quantity any time a 'depletion' transaction takes place. Instead, we want some type of background monitoring process that checks the level of the widgets on a timed basis. How can we accomplish this? Windows Service / WinForm application? Something built into IIS that will run ASP.net C# code?
Polling based monitoring should be your last resort. It uses too many resources for a simple task and most of the time it will only see that it's not the case to do anything. And it doesn't even scale when your data grows.
Instead, you should focus on the code that changes those values and act then, on in spot. And the check will also be lighter: only one item being checked not all, and only once, not every x seconds/minutes/hours/...
Apart from the architectural considerations, to answer your question, just as Jonathan said: anything that can read a database and send emails will do, but I'd consider a Windows Service for this job because that's what they were made for: background jobs running all the time, unrelated to the host users. You also get some extra benefits like automatic startup and recovery options.
Anything that can read the database and send an email could accomplish this - console app, winforms app, web app -- it doesn't really matter.
It may be more efficient to monitor when the values are changed (what changes them? A web application?) and have that application also send notifications

How to do something periodically in ASP.NET/WCF?

First of all sorry of asking such a dumb question, I am quite a newbie in asp.net.
So, I am supposed to do something periodically, say I am owner of site heartpatients.com (hypothetically) and I want that for each of my site user who visits the site, a message to be shown after 2hrs "Take your pills". so, basically this is all my question, how am I supposed to show this message after every 2hr (or 4, 6 whatever time)after, also how can I customize time.
One more thing, say if I have this method in a WCF service, that shows this message, how can I call that service at a particular time, and that even configured by user (say someone is taking pills after 10hrs?) So how to call that service (that particular method in service) after the time specified by user passes periodically?
I hope I made my question quite clear.
Any help is appreciated.
ASP.NET generally isn't suited as a task scheduler. The nature of the web is as a request/response system. So a web application should just sit and wait for a request, generate a response, and be done.
For any kind of back-end scheduled task, I'd recommend either:
A Windows Service
A console application scheduled to run (I think Windows comes with a task scheduler)
There are pros and cons either way. For example, a Windows Service will run from boot time and has no console UI, and is generally very manageable from a server perspective. While a console application is traditionally simpler to write and debug.
These can still share code from your web application. If your business logic and data access and all that good stuff are in their own projects/assemblies then these other applications can use those assemblies just as well. (Of course, if everything in your web application is UI-bound, that's another question entirely.)
What concerns me the most is... How do you plan to show this message to a user? Is the user just sitting on your website for hours at a time and you need to remind them to take their meds? Or do you plan to send an email or something? Maybe the example you gave doesn't really explain what you're trying to do? I'm not sure.
Running tasks in the background is one thing, but it seems to me that an entire half of your overall equation (displaying a message to the user) is sort of glazed over and not really thought through.
Check out Quartz.Net: http://quartznet.sourceforge.net/
It enables you to schedule tasks to run using cron expressions:
http://en.wikipedia.org/wiki/Cron

How should I complete this type of notification?

I am basically creating a site for recruiters. One of the functionality in my application requires posting to Facebook periodically. The posting frequency can be from 0(Never) to 4(High)
For Eg. If a recruiter has 4 open jobs and he has posting frequency set to 4, each job should be posted as per it's turn: 1st job on 1st day, 2nd job on 2nd, 3rd job on 3rd etc, on 5th day again 1st job (round robin fashion).
Had he set the posting frequency to 2, two jobs would be posted daily (thus each job would be posted every 2 days)
My only question is what type of threading should I create for this since this is all dynamic!! Also, any guidelines on what type of information should I store in database?
I need just a general strategy to solve this problem. No code..
I think you need to seperate it from your website, I mean its better to run the logic for posting jobs in a service hosted on IIS ( I am not sure such a thing exists or not, but I guess there is).
Also you need to have table for job queue to remember which jobs need to be posted, then your service would pick them up and post them one by one.
To decide if this is the time for posting a job you can define a timer with a configurable interval to check if there is any job to post or not.
Make sure that you keep the verbose log details if posting fails. It is important because it is possible that Facebook changes its API or your API key becomes invalid or anything else then you need to know what happened.
Also I strongly suggest to have a webpage for reporting the status of jobs-to-post queue, if they failed what was the causes of problem.
If you program runs non-stop, you can just use one of the Timer classes available in .NET framework, without the need to go for full-blown concurrency (e.g. via Task Parallel Library).
I suspect, though, that you'll need more than that - some kind of mechanism to detect which jobs were successfully posted and which were "missed" due program not running (or network problems etc.), so they can be posted the next time the program is started (or network becomes available). A small local database (such as SQLite or MS SQL Server Compact) should serve this purpose nicely.
If the requirements are as simple as you described, then I wouldn't use threading at all. It wouldn't even need to be a long-running app. I'd create a simple app that would just try to post a job and then exit immediately. However, I would scheduled it to run once every given period (via Windows Task Scheduler).
This app would check first if it hasn't posted any job yet for the given posting frequency. Maybe put a "Last-Successful-Post-Time" setting in your datastore. If it's allowed to post, the app would just query the highest priority job and then post it to Facebook. Once it successfully posts to Facebook, that job would then be downgraded to the lowest priority.
The job priority could just be a simple integer column in your data store. Lower values mean higher priorities.
Edit:
I guess what I'm suggesting is if you have clear boundaries in your requirements, I would suggest breaking your project into multiple applications. This way there is a separation of concerns. You wouldn't then need to worry how to spawn your Facebook notification process inside your web site code.

Windows Service Vs Simple Program

Let me give a back ground for everybody before I go to my problem. My company hosts website for many clients, my company also contracts some of the work to another company.
So when we first set up a website with all the informations to our clients, we pass that information to the other company we contracted and three of us have the same data. Problem is once the site is up and running, our clients will change some data and when ever they do that we should be able to update our contracted company.
The way we transfer data to the contracted company is by using a web service (httppost, xml data). Now my question is what it the best way to write a program which sends updated data to the contracted company everytime our clients change some data.
1) Write a windows service having a timer inside my code where every 30min or so connects to the database and find all changes and send it to the contracted company
2) Write the same code as #1 (with out the timer in it) but this time make it a simple program and let windows scheduler wake it every 30min
3) Any other suggestion you may have
Techenologies available for me are VS 2008, SQLServer 2005
Scheduled task is the way to go. Jon wrote up a good summary of why services are not well suited for this sort of thing: http://weblogs.asp.net/jgalloway/archive/2005/10/24/428303.aspx
A service is easy to create and install and is more "professional" feeling so why not go that way? Using a non-service EXE would also work of course and would be slightly easier to get running (permissions, etc.) but I think the difference in setup between the two is nearly negligible.
One possible solution would be to add a timestamp column to your data tables.
Once this is done, you can have one entry in each table that has the last collected time by your contracted company. They can pull all records since that last time and update their records accordingly.
A Windows Service is more self contained, and you can easily configure it to start up automatically when the OS is starting up. You might also need to create additional configuration options, as well as some way to trigger the synchronization immediately.
It will also give you more room to grow your functionality for the service in the future.
A standalone app should be easier to develop though, however you are reliant on the windows scheduler to execute the task always. My experience has been that it is easier to mess up things with the windows scheduler and have it not run, for example in cases where you reboot the OS but no user has logged in.
If you want a more professional approach go with the service, even though it might mean a little bit more work.
A windows service makes more sense in this case. Think about what happens after your server is restarted:
With a Windows Application you need to have someone restart the application, or manually copy a shortcut to the startup folder to make sure the application gets launched
OR,
With a Windows Service you set it to start automatically and forget about it. When the machine reboots your service starts up and continues processing.
One more consideration, what happens when there is an error? A Windows application would likely show an error dialog and wait for input before continuing; whereas a service would log the error in the event log and carry on.

Categories