We Keep Coding

How to exclude Exception and Error logging from sampling in Application Insights in .NET Core 2.1?

I'm running a web api with these specs:
ASP.NET Core 2.1
Microsoft.ApplicationInsights.AspNetCore 2.13.1
Hosted in Azure App Service.
In Startup.cs ConfigureServices I have added:
services.AddApplicationInsightsTelemetry();
_loggerFactory.AddAzureWebAppDiagnostics();
I've setup custom exception handler in the Startup.cs:
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
else
{
app.UseCustomExceptionHandler(telemetryClient, _loggerFactory);
}
Inside this CustomExceptionHandler I try to log the Exception like:
var logger = loggerFactory.CreateLogger("Unhandled Exception");
logger.LogError(ex, errorId);
var telemetryProperties = new Dictionary<string, string>();
telemetryProperties.Add("errorId", errorId);
telemetryProperties.Add("traceIdentifier", context.TraceIdentifier);
telemetryClient.TrackException(ex, properties: telemetryProperties);
With this configuration in place, not all Exceptions or Error Logs reached the Log Analytics buckets. So I have found this configuration for Application Insights:
var builder = aiTelemetryConfiguration.DefaultTelemetrySink.TelemetryProcessorChainBuilder;
builder.UseAdaptiveSampling(excludedTypes: "Trace;Exception");
builder.Build();
Here I exclude Trace and Exception from the Adaptive Sampling.
Currently this configuration is on production. It's handling +/- 50k requests a minute. But the exception bucket stays empty.
I noticed these messages between the Traces:
AI (Internal): [Microsoft-ApplicationInsights-Core] [msg=Log Error];[msg=Exception while initializing Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.AspNetCoreEnvironmentTelemetryInitializer, exception message - System.ArgumentException: The key already existed in the dictionary.
at System.Collections.Concurrent.ConcurrentDictionary`2.System.Collections.Generic.IDictionary.Add(TKey key, TValue value)
at Microsoft.ApplicationInsights.AspNetCore.TelemetryInitializers.AspNetCoreEnvironmentTelemetryInitializer.Initialize(ITelemetry telemetry)
at Microsoft.ApplicationInsights.TelemetryClient.Initialize(ITelemetry telemetry)]
and
AI: A Metric Extractor detected a telemetry item with SamplingPercentage < 100. Metrics Extractors should be used before Sampling Processors or any other Telemetry Processors that might filter out Telemetry Items. Otherwise, extracted metrics may be incorrect.
To be clear, I'm looking at these locations:
Azure Application Insights -> Search -> Trace|Custom Event|Exception
Log Analytics with this query:
exceptions
| order by timestamp desc
Is this at least the correct way to disable sampling?
Many thanks in advance.

I'm now adding this TelemetryInitializer:
public class ExceptionTelemetrySamplingFilter : ITelemetryInitializer
{
public void Initialize(ITelemetry telemetry)
{
if (telemetry is ExceptionTelemetry)
{
((ISupportSampling)telemetry).SamplingPercentage = 100;
}
}
}
And registered it:
services.AddSingleton<ITelemetryInitializer, ExceptionTelemetrySamplingFilter>();
It's from the official documentation about sampling. And I removed the added configurations from startup.
I'll keep you posted on improvements.

Why is Application Insights TrackException not logging anything?

I'm currently trying to log exceptions in Application Insights for later investigation.
However, any exception I try to log with the TrackException method does not appear in Application Insights.
The docs for Application Insights event filtering state the following:
Exception - Uncaught exceptions in the server, and those that you log by using TrackException().
I'm using the following code to log my exception:
private static readonly TelemetryClient Client = new TelemetryClient{InstrumentationKey = ApplicationInsightsSettings.InstrumentationKey};
public static void TrackException(Exception exception, string traceReference)
{
var properties = new Dictionary<string, string> {{"Trace Reference", traceReference}};
Client.TrackException(exception);
Client.TrackException(exception, properties);
Client.TrackEvent(traceReference);
}
Whenever I run this code only an event is logged but none of the exceptions are.

Scope Logging in Application Insight

We have a .net core web api and we have used inbuild logger available in Microsoft.Extensions.Logging namespace.
We have integrated this logger with Application Insight.
I can see all the logs getting logged correctly. However, I am not able to see the logging information of scope
Below is my code:
var taskId = Guid.NewGuid();
using (logger.BeginScope("Assigning Task {taskId}.",taskId))
{
logger.LogInformation("{taskId} is assigned",taskId);
}
I can see this output where scope log information is matained in '{Original Format}'. However, structured logging is not working for that field :
I have below questions:
How does scope logging works in Application Insight?
Is this the only way to see the scope information?
Is there any way to see all the logs under one scope?
Why structured logging is not working for {Original Format}?

Scopes logging works nowadays in Application Insights (I'm using version 2.19.0). Instead of passing a format string + params to BeginScope() you should pass a dictionary of name-value pairs, e.g.
using (logger.BeginScope(new Dictionary<string, object>
{ { "TaskId" = taskId }, { "Action", "AssigningTask" } }))
{
..
logger.LogInformation("{taskId} is assigned", taskId);
...
}
Here putting the taskId in the inner log message is a bit redundant, of course.

ASP.NET Core EventLog Not Registering

I've been trying to add an EventLog Provider for logging in my ASP.NET Application. I was able to add it to the 'Application' source, but there's a lot of information in that log and I'd like for it to be under it's own source. I've actually gotten 3 logs to be written to it's own source, but I have no idea how or why. This is what my code looks like right now:
CreateWebHostBuilder(args)
.ConfigureLogging( logging =>
{
logging.AddEventLog(
new Microsoft.Extensions.Logging.EventLog.EventLogSettings
{
SourceName = "MySweetApp",
MachineName = ".",
LogName = "MySweetApp",
Filter = (category, level) =>
{
if (category.EndsWith("Controller") || level >= LogLevel.Error)
{
return true;
}
return false;
},
});
})
.Build()
.Run();
I've already added a registry key there for this app (since the app doesn't have admin rights and can't add a source) along with the EventMessageFile of C:\Windows\Microsoft.NET\Framework64\v4.0.30319\EventLogMessages.dll. I've messed with different values of this including the non-64 framework folder and other v2 versions. There are no errors or warnings, and I can't find any logs that have anything related in them. I've also added a debug logger with .AddDebug() and it prints just fine the debug window.
I'm guessing it's something on the server side of things and not my code. Thanks in advance.
UPDATE: I can now re-create my app creating error logs in the eventlog. If I start my program by double-clicking the .exe, there is an authentication error that is thrown and gets sent to the log.

Found it. I still had a "Logging" section of my appsettings.json that was set default to warning. Deleted that section and everything started printing!

Application insight log exceptions only

I want to use Application Insights to log exceptions only. How can I do that?
I tried searching for ways turning other settings off such as this and it says that there is no way to turn it off.
I tried ITelemetryProcessor and encountered the same problem as this question. I tried both config and code ways of registering ITelemetryProcessor but it is not hit even if I explicitly throw an exception in my Web API controller.
I am using VS 2017 and created a new .Net Framework 4.6.2 Web API. I also have an InstrumentationKey and can see the exception logged in Azure portal.

First of all, the first link you referenced is nothing to do with your issue.
You want to only log the exceptions, but that link means that remove the old telemetry data like Trace in repository(where the telemetry data is stored after upload to app insights).
You can take use of ITelemetryProcessor to log exceptions only. Please follow my steps as below:
1.Add Application insights to your web api project by right clicking your project name -> select Configure Application Insights:
After SDK added, do not select the Enable trace collection:
2.Add a .cs file in your project, then implement your custom ITelemetryProcessor class, code is as below:
using Microsoft.ApplicationInsights.Channel;
using Microsoft.ApplicationInsights.Extensibility;
namespace WebApplicationWebApi
{
public class ExceptionsFilter:ITelemetryProcessor
{
private ITelemetryProcessor Next { get; set; }
public ExceptionsFilter(ITelemetryProcessor next)
{
this.Next = next;
}
public void Process(ITelemetry item)
{
string s = item.GetType().Name;
//if it's not exception telemetry, just return without log it to app insights.
if (s != "ExceptionTelemetry")
{
return;
}
this.Next.Process(item);
}
}
}
3.Register your custom ITelemetryProcessor in the ApplicationInsights.config. In the node, add <Add Type="WebApplicationWebApi.ExceptionsFilter,WebApplicationWebApi"/> :
4.Then run your code. To make sure the custom ITelemetryProcessor class is called, you can set a breakpoint in that class to see if it's hit when running.
And for the testing purpose, I add some telemetry data in the HomeController.cs:
public class HomeController : Controller
{
TelemetryClient client = new TelemetryClient();
public ActionResult Index()
{
RequestTelemetry r1 = new RequestTelemetry();
r1.Name = "request message for testing";
client.TrackRequest(r1);
client.TrackTrace("trace message for testing wwwww.");
client.TrackException(new Exception("exception message for testing wwwww."));
ViewBag.Title = "Home Page";
return View();
}
}
5.In your visual studio output window, you should see these messages:
6.Then in visual studio, nav to Application Insights Search (in vs -> view -> other windows -> Application Insights Search), then check if there are some values here(if it has values like "4" in screenshot below, click on it):
7.If it has values in step 6, please click the update button, then check All:
8.Then you can see that only the Exceptions are logged: