When an exception is unhandled in our web API, a response code 500 (Internal Server Error) will return to the client.
Although Application Insights doesn't record this as a 500, but rather a 200. Successful request is false, but the response code is still wrong.
How can I get the corrrect response codes in my telemetry?
Startup's Configure:
public void Configure(IApplicationBuilder app, IHostingEnvironment environment)
{
if (!TelemetryConfiguration.Active.DisableTelemetry)
{
// Add Application Insights to the beginning of the request pipeline to track HTTP request telemetry.
app.UseApplicationInsightsRequestTelemetry();
// Add Application Insights exceptions handling to the request pipeline. Should be
// configured after all error handling middleware in the request pipeline.
app.UseApplicationInsightsExceptionTelemetry();
}
app.UseRequireHttps(environment.IsLocal());
app.UseMiddleware<NoCacheMiddleware>();
app.UseJwtBearerAuthentication(...);
app.UseCors("CorsPolicy");
app.UseStaticFiles();
app.UseCompression();
app.UseMvc();
}
Although Application Insights doesn't record this as a 500, but rather a 200. Successful request is false, but the response code is still wrong.
As far as I know, if application has no error handling middleware Application Insights will report response status code 200 when unhandled exception is thrown. We could find detailed info from this article. I am using the following code in the method Configure and I could get the correct response status code.
public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
if (!Microsoft.ApplicationInsights.Extensibility.TelemetryConfiguration.Active.DisableTelemetry)
{
app.UseApplicationInsightsRequestTelemetry();
app.UseApplicationInsightsExceptionTelemetry();
}
loggerFactory.AddConsole(Configuration.GetSection("Logging"));
loggerFactory.AddDebug();
app.UseIISPlatformHandler();
app.UseExceptionHandler(options => {
options.Run(
async context =>
{
context.Response.StatusCode = (int)HttpStatusCode.InternalServerError;
context.Response.ContentType = "text/html";
var ex = context.Features.Get<IExceptionHandlerFeature>();
if (ex != null)
{
var err = $"<h1>Error: {ex.Error.Message}</h1>{ex.Error.StackTrace }";
await context.Response.WriteAsync(err).ConfigureAwait(false);
}
});
});
app.UseStaticFiles();
app.UseMvc();
}
ApplicationInsights tracks the interaction but does not report an error without the client logging the error as a Custom Event or the Server having Middleware code added (as per #fei-han)
If you want Analytics alerts then the Azure Data Lake query I use is:
let sitename = "localhost:26047/api";
requests
| where timestamp > ago(1d)
| where url contains sitename
| where resultCode contains "40"
| order by timestamp desc
For alerts you could create a new 'Rule' from the Analytics blade for about $1.50 a month.
Related
I have a .Net Core 3.1 application that I use as an API but it also serves my SPA (Angular). As of recently I am having some issues with SEO so I would like to serve a static version of my Angular application when Googlebot comes around.
Is here any way to dynamically set the SPA source path?
app.UseStaticFiles();
app.UseSpaStaticFiles();
app.UseSpa(spa =>
{
spa.Options.SourcePath = "ClientApp";
});
Currently I have a small middleware file where I detect Googlebot based on the UserAgent string. Then I pass that through the request.
public async Task InvokeAsync(HttpContext context)
{
var userAgent = context.Request.Headers["User-Agent"];
context.Items["isCrawler"] = userAgent.Contains("Googlebot");
await _next(context);
}
But I cannot access the Request in the Configure() method in the Startup.cs file. Is there anyway how I can make this work? I really want to be able to dynamically set the SourcePath.
Thanks a lot!
Regards
You can try the following codes , putting the middleware in Configure（）method .
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
app.Use(async (context, next) =>
{
var userAgent = context.Request.Headers["User-Agent"];
context.Items["isCrawler"] = userAgent.Contains("Googlebot");
if ((bool)context.Items["isCrawler"])
{
app.UseSpa(spa =>
{
spa.Options.SourcePath = "ClientApp";
});
}
else {
//do your stufff....
}
// Call the next delegate/middleware in the pipeline
await next();
});
…}
ASP.NET Core 5 Razor Pages using Serilog
UseStatusCodePagesWithReExecute works as expected and re-executes a page after it goes to my /CustomError page.
How to suppress Serilog logging of the 2nd call to the re-executed page?
password-postgres full sample
// program.cs
public static void Main(string[] args)
{
Log.Logger = new LoggerConfiguration()
// if only do warning, then will get duplicate error messages when an exception is thrown, then again when re-executed
// we do get 2 error message per single error, but only 1 stack trace
.MinimumLevel.Override("Microsoft.AspNetCore", LogEventLevel.Fatal)
.Enrich.FromLogContext()
.WriteTo.Console()
.CreateLogger();
try
{
Log.Information("Starting up");
CreateHostBuilder(args).Build().Run();
}
catch (Exception ex)
{
Log.Fatal(ex, "Application start-up failed");
}
finally
{
Log.CloseAndFlush();
}
}
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.UseSerilog() // <- Add this line
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
});
}
and then
public class Startup
{
public void ConfigureServices(IServiceCollection services)
{
// snip
}
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
else
{
app.UseExceptionHandler("/CustomError");
}
app.UseStaticFiles();
// https://khalidabuhakmeh.com/handle-http-status-codes-with-razor-pages
// https://andrewlock.net/retrieving-the-path-that-generated-an-error-with-the-statuscodepages-middleware/
app.UseStatusCodePagesWithReExecute("/CustomError", "?statusCode={0}");
app.UseRouting();
// don't want request logging for static files so put this serilog middleware here in the pipeline
app.UseSerilogRequestLogging(); // <- add this
app.UseAuthentication();
app.UseAuthorization();
app.UseCookiePolicy(new CookiePolicyOptions { MinimumSameSitePolicy = SameSiteMode.Strict });
app.UseEndpoints(endpoints =>
{
endpoints.MapRazorPages();
});
}
}
and then
// CustomError.cshtml.cs
[ResponseCache(Duration = 0, Location = ResponseCacheLocation.None, NoStore = true)]
[IgnoreAntiforgeryToken]
public class CustomErrorModel : PageModel
{
public int? CustomStatusCode { get; set; }
public void OnGet(int? statusCode = null)
{
var feature = HttpContext.Features.Get<IStatusCodeReExecuteFeature>();
// a non 500 eg 404
// this can be non page requests eg /js/site-chart.js
// feature can be null when a 500 is thrown
if (feature != null)
{
//Log.Warning($"Http Status code {statusCode} on {feature.OriginalPath}");
CustomStatusCode = statusCode;
return;
}
// a 500
// relying on serilog to output the error
//var exceptionHandlerPathFeature = HttpContext.Features.Get<IExceptionHandlerPathFeature>();
// integration tests can call a page where the exceptionHandlerPathFeature can be null
CustomStatusCode = 500;
// somewhere else is emitting the Log.Error stacktracke
//Log.Error($"Exception is {exceptionHandlerPathFeature.Error}");
//OriginalPath = exceptionHandlerPathFeature.Path;
//Exception exception = exceptionHandlerPathFeature.Error;
}
public void OnPost()
{
Log.Warning( "ASP.NET failure - maybe antiforgery. Caught by OnPost Custom Error. Sending a 400 to the user which is probable");
Log.Warning("Need to take off minimumlevel override in Program.cs for more information");
CustomStatusCode = 400;
}
}
Duplicate log entries for errors eg 404 - ideally only want 1
Update
Thanks to Alan's answer below I've put the SerilogRequestLogging at the start of configure.
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
app.UseSerilogRequestLogging();
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
else
{
app.UseExceptionHandler("/CustomError");
}
app.UseStaticFiles();
app.UseStatusCodePagesWithReExecute("/CustomError", "?statusCode={0}");
// snip..
}
This gives 2 ERR messages in the log:
Which I'm fine with.
There is probably a way to merge the 2 ERR entries, but this is simple. Also the 2 entries are for different concepts. Requests and Exceptions.
It may be possible to give each log entry a RequestId as the boilerplate Error.cshtml.cs gives.
RequestId = Activity.Current?.Id ?? HttpContext.TraceIdentifier;
But hey, this solution is good enough for me. Thanks Alan!
Short answer:
To prevent duplicate logging in this case, you can place UseSerilogRequestLogging() before UseStatusCodePagesWithReExecute() in your Configure method:
app.UseSerilogRequestLogging();
app.UseStatusCodePagesWithReExecute("/CustomError", "?statusCode={0}");
Long answer:
According to the ASP.NET documentation, the order used to build middlewares matters:
The order that middleware components are added in the Startup.Configure method defines the order in which the middleware components are invoked on requests and the reverse order for the response. The order is critical for security, performance, and functionality.
Now, according to Serilog documentation, UseSerilogRequestLogging will only process components that appear after in the middleware pipeline.
With that in mind, I noticed that on the Startup class, you added your UseSerilogRequestLogging middleware after UseStatusCodePagesWithReExecute.
UseStatusCodePagesWithReExecute documentation says it does two things:
Returns the original status code to the client.
Generates the response body by re-executing the request pipeline using an alternate path.
In other words, when you get an error, it seems Serilog is unaware that the second request was internally generated by a previous middleware, so it will log both:
the original request
the second one, created by the execution of the alternate path (/CustomError)
Step-by-step, this is what would happen in the request pipeline when a GET request hits the StatusCode400 endpoint:
Request flow:
UseStatusCodePagesWithReExecute -> Serilog -> StatusCode400 endpoint (error happens here)
Response flow:
UseStatusCodePagesWithReExecute (error status code, so re-execution kicks in) <- Serilog (logs request) <- StatusCode400 endpoint
Re-execution request flow:
UseStatusCodePagesWithReExecute -> Serilog -> CustomError endpoint (no error now, but re-execution preserves the HTTP status code)
Re-execution response flow:
UseStatusCodePagesWithReExecute <- Serilog (logs request) <- CustomError endpoint
Therefore, if you don't want to have duplicate log messages, you can place the Serilog middleware before the re-execution one, so it will process only one request coming from it (the second one):
Request flow:
Serilog -> UseStatusCodePagesWithReExecute -> StatusCode400 endpoint (error happens here)
Response flow:
Serilog <- UseStatusCodePagesWithReExecute (error status code, so re-execution kicks in) <- StatusCode400 endpoint
Re-execution request flow:
Serilog -> UseStatusCodePagesWithReExecute -> CustomError endpoint (no error now, but re-execution preserves the HTTP status code)
Re-execution response flow:
Serilog (logs request) <- UseStatusCodePagesWithReExecute <- CustomError endpoint
You can use this solution to avoid double logging. Obviously this snippet code is a solid block and must be inserted at the beginning of the Configure method.
app.Use(async (context, next) =>
{
try
{
await next.Invoke();
}
catch
{ //avoid double logging by Microsoft.AspNetCore.Server.IIS.Core.IISHttpServer or others middleware
context.Response.StatusCode = StatusCodes.Status500InternalServerError;
}
});
app.UseSerilogRequestLogging();
I am having the issue when running the API under IIS and getting the following error on the client
"Status(StatusCode=\"Cancelled\", Detail=\"No grpc-status found on response.\")"
Client code
static async Task Main(string[] args)
{
// This switch must be set before creating the GrpcChannel/HttpClient.
AppContext.SetSwitch(
"System.Net.Http.SocketsHttpHandler.Http2UnencryptedSupport", true);
var httpHandler = new HttpClientHandler();
// Return `true` to allow certificates that are untrusted/invalid
httpHandler.ServerCertificateCustomValidationCallback =
HttpClientHandler.DangerousAcceptAnyServerCertificateValidator;
var channel = GrpcChannel.ForAddress("https://localhost:5001/",
new GrpcChannelOptions { HttpHandler = httpHandler });
var client = new Greeter.GreeterClient(channel);
var reply = await client.SayHelloAsync(new HelloRequest { Name = "GreeterClient" });
Console.WriteLine("Greeting: " + reply.Message);
Console.WriteLine("Press any key to exit...");
Console.ReadKey();
}
Server code
public void ConfigureServices(IServiceCollection services)
{
services.AddGrpc();
services.AddGrpcReflection();
}
// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
else
{
app.UseHsts();
}
app.UseHttpsRedirection();
app.UseRouting();
app.UseEndpoints(endpoints =>
{
endpoints.MapGrpcService<GreeterService>();
endpoints.MapGet("/", async context =>
{
await context.Response.WriteAsync("Communication with gRPC endpoints must be made through a gRPC client. To learn how to create a client, visit: https://go.microsoft.com/fwlink/?linkid=2086909");
});
});
}
from https://learn.microsoft.com/en-us/aspnet/core/host-and-deploy/iis/protocols?view=aspnetcore-5.0
IIS do support HTTP/2 features to support gRPC after Windows 10, OS Build 20300.1000
Update i found error in event view
Category: Grpc.AspNetCore.Server.ServerCallHandler
EventId: 6
SpanId: cff724fbe3176247
TraceId: 288c25acb18dd646ad3eaabfdd53175f
ParentId: 0000000000000000
RequestId: 8000000a-0004-fc00-b63f-84710c7967bb
RequestPath: /greet.Greeter/SayHello
Error when executing service method 'SayHello'.
Exception:
System.InvalidOperationException: Trailers are not supported for this response. The server may not support gRPC.
at Grpc.AspNetCore.Server.Internal.GrpcProtocolHelpers.GetTrailersDestination(HttpResponse response)
at Grpc.AspNetCore.Server.Internal.HttpResponseExtensions.ConsolidateTrailers(HttpResponse httpResponse, HttpContextServerCallContext context)
at Grpc.AspNetCore.Server.Internal.HttpContextServerCallContext.EndCallCore()
at Grpc.AspNetCore.Server.Internal.HttpContextServerCallContext.EndCallAsync()
at Grpc.AspNetCore.Server.Internal.CallHandlers.ServerCallHandlerBase`3.HandleCallAsync(HttpContext httpContext
In a .NET Core Web Application I am using middleware (app.UseMyMiddleware) to add an Intranet Header to the response. The restriction is that the Handler which is formatting the response (MyResponseHandler.FormatResponse) can not be changed as it is used on legacy apps and returns a string value. I have got it working as such but want to limit the processing to Page requests only, rather than every single request that goes through (i.e css/js/images etc).
The FormatResponse basically just takes in the current response as a string, finds the "body" tag and appends a load of html which creates a nice menu bar to be able to access all the other applications.
Firstly, is this a good way to implement a common header in .NET Core?
Everything I've found so far just says to use the Layout page which is fine for a single application, but not for a larger number of applications written in different frameworks.
Secondly, how do I go about filtering ONLY page requests?
I can filter all pages that don't contain '.' but what happens if there's a '.' in a query string for example? I'm also not sure if MVC or razor pages ever have a '.' in the url (not from what I've seen so far).
Thirdly, is there a more efficient way to do this? Going through all this code for every single request seems like it may slow things down a little. Can I filter earlier on to save code execution or is there an alternative way?
Startup.cs
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
else
{
app.UseExceptionHandler(MiddlewareExtensions.GenericExceptionHandler);
app.UseHsts();
}
app.UseHttpsRedirection();
app.UseStaticFiles();
app.UseMyMiddleware();
app.UseMvc(routes =>
{
routes.MapRoute(
name: "default",
template: "{controller=Home}/{action=Index}/{id?}");
});
}
MiddlewareExtensions.cs
public static void UseMyMiddleware(this IApplicationBuilder app)
{
app.UseMiddleware<LoadIntranetHeader>();
}
LoadIntranetHeader.cs
public class LoadIntranetHeader
{
private readonly RequestDelegate _next;
public LoadIntranetHeader(RequestDelegate next)
{
_next = next;
}
public async Task Invoke(HttpContext context)
{
var originalBodyStream = context.Response.Body;
var config = (IConfiguration)context.RequestServices.GetService(typeof(IConfiguration));
using (var responseBody = new MemoryStream())
{
context.Response.Body = responseBody;
await _next(context);
context.Response.Body = originalBodyStream;
responseBody.Seek(0, SeekOrigin.Begin);
string response = new StreamReader(responseBody).ReadToEnd();
if (!context.Request.Path.ToString().Contains(".")
&& !context.Request.Path.ToString().Contains("/api/", StringComparison.CurrentCultureIgnoreCase))
{
response = MyResponseHandler.FormatResponse(config, response);
}
await context.Response.WriteAsync(response);
}
}
}
I'm trying to understand how to delve into the automagic I get when I configure the hell out of asp.net. I'm currently translating a small api from asp.net web-api 2 to asp.net core. I'm not sure where the 403 is coming from in this configuration or how to fix it. Right now the majority of the api endpoint just need a valid token and do not need to check for any specific claim in the token. So for all my authenticated controllers I get a 403 response that should be a 200, when using a valid bearer token. Also right now I use asymmetric keys with Auth0 as the provider.
Startup.cs configure method I'm using to validate the JWT bearer tokens.
public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
loggerFactory.AddConsole(Configuration.GetSection("Logging"));
loggerFactory.AddDebug();
//Middleware added here order matters
//TODO formatter settings https://docs.asp.net/en/latest/mvc/models/formatting.html
//samples to check
//https://auth0.com/docs/server-apis/webapi-owin
//https://github.com/auth0-samples/auth0-aspnetcore-webapi-rs256
var options = new JwtBearerOptions
{
Audience = Configuration["auth0:clientId"]
,Authority = $"https://{Configuration["auth0:domain"]}/"
,Events = new JwtBearerEvents() // just a pass through to log events
};
app.UseJwtBearerAuthentication(options);
// Very hacky to catch invaild tokens https://github.com/aspnet-contrib/AspNet.Security.OpenIdConnect.Server/issues/191
// issue says the need for the required hack is fixed but it's been still happening. Issue about the fix https://github.com/aspnet/Security/issues/411
app.Use(next => async context => {
try
{
await next(context);
}
catch
{
// If the headers have already been sent, you can't replace the status code.
// In this case, throw an exception to close the connection.
if (context.Response.HasStarted)
{
throw;
}
context.Response.StatusCode = 401;
}
});
app.UseMvc();
// TODO global exception handling https://github.com/dotnet/corefx/issues/6398
app.UseSwaggerGen();
app.UseSwaggerUi();
}
}
It seems your token middleware is not executed to validate incoming requests.
Try setting the token middleware to run automaticly.
var options = new JwtBearerOptions
{
//other configurations..
AutomaticAuthenticate = true;
};
You can also use attributes to specify the authentication scheme in the controllers.
[Authorize(AuthenticationSchemes = "MyAuthenticationScheme")]
Read more about it here: Limiting identity by scheme
The problem was with the policy in the ConfigureServices section. The simplest policy is all I need at the moment.
public void ConfigureServices(IServiceCollection services)
{
// Add framework services.
services.AddMvc(c =>
{
// TODO implement this abstract class c.Filters.Add(typeof(ExceptionFilterAttribute));
var policy = new AuthorizationPolicyBuilder()
.RequireAuthenticatedUser()
.Build();
c.Filters.Add(new AuthorizeFilter(policy));
c.Filters.Add(typeof(ValidateModelFilter));
});