I wrote a class that get the html from a site, this is the code:
public class NetworkHelper {
static Lazy<HttpClient> httpClient = new Lazy<HttpClient>(() => {
var handler = CreateHandler();
return new HttpClient(handler) {
Timeout = TimeSpan.FromSeconds(3)
};
});
static HttpMessageHandler CreateHandler() {
var handler = new HttpClientHandler();
// if the framework supports redirect configuration
// set max redirect to the desired amount the default is 50
if (handler.SupportsRedirectConfiguration) {
handler.AllowAutoRedirect = true;
handler.MaxAutomaticRedirections = 5;
}
// if the framework supports automatic decompression
// set automatic decompression
if (handler.SupportsAutomaticDecompression) {
handler.AutomaticDecompression = System.Net.DecompressionMethods.GZip |
System.Net.DecompressionMethods.Deflate;
}
return handler;
}
/// <summary>
/// Get the html structure of a site.
/// </summary>
/// <param name="url">Represents the URL of the page where to download the data.</param>
/// <returns>Return a string that contains the html of the site.</returns>
public async Task<string> GetHtmlAsync(Uri url, CancellationToken cancellationToken = default(CancellationToken)) {
var response = await httpClient.Value.GetAsync(url, cancellationToken);
var content = await response.Content.ReadAsStringAsync();
return content;
}
}
the problem's that when I try to download the data from this url:
string html = await new NetworkHelper().GetHtmlAsync(new Uri("https://int.soccerway.com/charts/statsplus/2139109"));
I'll get this content:
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved here.</p>
</body></html>
so I guess AllowAutoRedirect isn't working as expected?
I discovered that changing:
return new HttpClient(handler) {
Timeout = TimeSpan.FromSeconds(3)
};
to:
return new HttpClient(handler) {
//Timeout = TimeSpan.FromSeconds(3)
};
all works well. Seems that there is a limited number of HTTP requests you can do per second, others get queued.
Related
I want to have a Non Browser Client talk to an Application Load Balancer and send authenticated requests on behalf of a user in a Cognito pool. While my proof of concept is currently working, the Non Browser Client (a C# program) is manually constructing HTTP requests as though it is a browser and it just doesn't seem to be the right way of doing it.
Is there some way to do this via the SDK or some proper API I can consume?
I have included diagrams and my proof of concept code to illustrate my scenario.
If this is in fact the correct way of going about it, I'll just refactor and move forward.
[High Level
Sequence Diagram
/// <summary>
/// Proof of Concept Code.
/// </summary>
/// <returns></returns>
public static async Task ManualAuthAppLb2()
{
using (var handler = new HttpClientHandler
{
ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator,
AllowAutoRedirect = false
})
using (var client = new HttpClient(handler))
{
//--------------------------------------------------------//
//Go to /secure_page and get redirected //
var url = new Uri("https://<app_lb_id>.elb.amazonaws.com/secure_page");
var req = new HttpRequestMessage
{
Method = HttpMethod.Get,
RequestUri = url
};
var response = await client.SendAsync(req);
//--------------------------------------------------------//
//--------------------------------------------------------//
//Follow First Redirect to cognito /authorize page
var headers = response.Headers;
var rediruri = headers.First(h => h.Key.ToLower() == "location").Value.First();
req = new HttpRequestMessage
{
Method = HttpMethod.Get,
RequestUri = new Uri(rediruri)
};
response = await client.SendAsync(req);
//--------------------------------------------------------//
//--------------------------------------------------------//
//Follow next redirect to cognito /login page
headers = response.Headers;
var cookies = headers.First(h => h.Key.ToLower() == "set-cookie").Value.ToList();
var csrfToken = cookies.First().Split('=')[1]
.Split(';')[0]; ;
rediruri = headers.First(h => h.Key.ToLower() == "location").Value.First();
req = new HttpRequestMessage
{
Method = HttpMethod.Get,
RequestUri = new Uri(rediruri)
};
response = await client.SendAsync(req);
//--------------------------------------------------------//
//--------------------------------------------------------//
//Post Login Data to cognito
headers = response.Headers;
var data = new Dictionary<string, string>()
{
["_csrf"] = csrfToken,
["username"] = "someuser",
["password"] = "somepassword"
};
req = new HttpRequestMessage
{
Method = HttpMethod.Post,
RequestUri = new Uri(rediruri),
Content = new FormUrlEncodedContent(data)
};
response = await client.SendAsync(req);
//--------------------------------------------------------//
//--------------------------------------------------------//
//Follow Redirect Back to Application Load Balancer
headers = response.Headers;
rediruri = headers.First(h => h.Key.ToLower() == "location").Value.First();
req = new HttpRequestMessage
{
Method = HttpMethod.Get,
RequestUri = new Uri(rediruri)
};
response = await client.SendAsync(req);
headers = response.Headers;
rediruri = headers.First(h => h.Key.ToLower() == "location").Value.First();
//--------------------------------------------------------//
//--------------------------------------------------------//
//Follow redirect back to /secure_page
req = new HttpRequestMessage
{
Method = HttpMethod.Get,
RequestUri = new Uri(rediruri)
};
response = await client.SendAsync(req);
headers = response.Headers;
Console.WriteLine(await response.Content.ReadAsStringAsync());
//--------------------------------------------------------//
}
}
There is an amazon SDK for C# : the image shows the packages used to develop such application.
ASP.Net Core Web API Call Thirds party API fails intermittently.
The following exception raises intermittently when load test with postman.
"Call failed with status code 500 (Internal Server Error): POST https://sample.com/apiendpoint."
I tried the Named/Typed with HttpClient/IHttpClientFactory approach and the problem continues.
How to make sure it uses connection pooling and not create new on one.
what is the right value for SetHandlerLifetime to keep the connection in the pool for future call to use.
services.AddHttpClient<IRestService, RestServiceOne>()
.SetHandlerLifetime(TimeSpan.FromMinutes(5)) //Set lifetime to five minutes
.AddPolicyHandler(GetRetryPolicy());
The following code is in RestServiceOne.cs
public class RestServiceOne : IRestService
{
private readonly IHttpClientFactory _httpClientFactory;
public RestServiceOne(IHttpClientFactory httpClientFactory)
{
_httpClientFactory = httpClientFactory;
}
public async Task<HttpResponseMessage> GetDataAsync(string destinationUrl, string user,
string password, string requestXml, string orderNumber, CancellationToken cancellationToken)
{
var endpoint = $"{destinationUrl}";
var authToken = Encoding.ASCII.GetBytes($"{user}:{password}");
var data = new StringContent(requestXml, Encoding.UTF8, "text/xml");
var httpRequestMessage = new HttpRequestMessage(
HttpMethod.Post,
endpoint)
{
Headers =
{
{ "Accept", "application/vnd.github.v3+json" },
{ "User-Agent", "HttpRequestsConsoleSample" }
}
};
httpRequestMessage.Content = data;
var httpClient = _httpClientFactory.CreateClient();
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic",
Convert.ToBase64String(authToken));
var httpResponseMessage = await httpClient.SendAsync(httpRequestMessage);
return httpResponseMessage;
}
}
I also tried HttpClient injection given in Microsoft type example.
public class RestService
{
private readonly HttpClient _httpClient;
public RestService(HttpClient httpClient)
{
_httpClient = httpClient;
try
{
_httpClient.BaseAddress = new Uri("https://testxx.com/test");
// GitHub API versioning
_httpClient.DefaultRequestHeaders.Add("Accept",
"application/vnd.github.v3+json");
// GitHub requires a user-agent
_httpClient.DefaultRequestHeaders.Add("User-Agent",
"HttpClientFactory-Sample");
}
catch(Exception ex)
{
}
}
public async Task<HttpResponseMessage> GetDataAsync(string destinationUrl, string user,
string password, string requestXml, string orderNumber, CancellationToken cancellationToken)
{
var endpoint = $"{destinationUrl}";
var authToken = Encoding.ASCII.GetBytes($"{user}:{password}");
_httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic",
Convert.ToBase64String(authToken));
var data = new StringContent(requestXml, Encoding.UTF8, "text/xml");
try
{
var response = await _httpClient.PostAsync(endpoint, data);
return response;
}
catch(Exception ex)
{
throw;
}
}
}
I tried with Flurl directly in service layer
var endpoint = $"{destinationUrl}";
var authToken = Encoding.ASCII.GetBytes($"{user}:{password}");
var data = new StringContent(requestXml, Encoding.UTF8, "text/xml");
try
{
var response = await endpoint
.WithHeader("Content-Type", "text/xml")
.WithHeader("app-bu-id", "SANIDERMMEDICAL")
.WithHeader("Authorization", new AuthenticationHeaderValue("Basic", Convert.ToBase64String(authToken)))
.PostAsync(data);
The above all 3 approach failed.
what is right value for .SetHandlerLifetime() to keep the connection in the pool and avoid creating new.
I saw some other example using the following approach but how to use this with IHttpClientFactory / Flurl.
var socketsHandler = new SocketsHttpHandler
{
PooledConnectionLifetime = TimeSpan.FromMinutes(10),
PooledConnectionIdleTimeout = TimeSpan.FromMinutes(5),
MaxConnectionsPerServer = 10
};
var client = new HttpClient(socketsHandler);
How can I ensure it use connection pooling and avoid the 500 error when calling 3rd party API from Azure.
I found solution when I use httpclient as given below.
private static readonly HttpClient httpClient = new HttpClient(new SocketsHttpHandler
{
PooledConnectionLifetime = TimeSpan.FromSeconds(60),
PooledConnectionIdleTimeout = TimeSpan.FromSeconds(20),
MaxConnectionsPerServer = 10
});
This article helped me
Tried couple of value for SocketsHttpHandler but finally choose this since no error.
This answer to the question on how to make HttpClient not follow redirects gives a solution to be set upon creating the actual client:
var handler = new HttpClientHandler { AllowAutoRedirect = false };
var client = new HttpClient(handler);
The comment below the answer is my actual question:
Is it possible to do this on a per-request basis without needing two separate HttpClient instances (i.e. one that allows redirects and one that does not)?
I have a specific reason too for now wanting separate clients: I want the client to retain its cookies from earlier requests. I'm trying to do a few requests first that include valid redirects, but only the last one in the chain I don't want to be a redirect.
I've searched, and looked through the overloads of .GetAsync(url, ...), and looked through the properties and methods of HttpClient, but found no solution yet.
Is this possible?
Yes, you can set the properties of the HttpClientHandler per each request, like so:
using (var handler = new HttpClientHandler())
using (var client = new HttpClient(handler))
{
handler.AllowAutoRedirect = false;
// do your job
handler.AllowAutoRedirect = true;
}
Just make sure that only one thread consumes the HttpClient at a time, if the client handler settings are different.
Example (note: only works in test environment)
Dummy remote server with Node.js runnin on localhost:
const express = require('express')
const app = express()
const cookieParser = require('cookie-parser')
const session = require('express-session')
const port = 3000
app.use(cookieParser());
app.use(session({secret: "super secret"}))
app.get('/set-cookie/:cookieName', (req, res) => {
const cookie = Math.random().toString()
req.session[req.params.cookieName] = cookie
res.send(cookie)
});
app.get('/ok', (req, res) => res.send('OK!'))
app.get('/redirect-301', (req, res) => {
res.writeHead(301, {'Location': '/ok'})
res.end();
})
app.get('/get-cookie/:cookieName', (req, res) => res.send(req.session[req.params.cookieName]))
app.listen(port, () => console.log(`App listening on port ${port}!`))
Tests
using System.Net;
using System.Net.Http;
using System.Threading.Tasks;
using NUnit.Framework;
public class Tests
{
private HttpClientHandler handler;
private HttpClient client;
private CookieContainer cookieJar = new CookieContainer();
private string cookieName = "myCookie";
private string cookieValue;
[SetUp]
public void Setup()
{
handler = new HttpClientHandler()
{
AllowAutoRedirect = true,
CookieContainer = cookieJar
};
client = new HttpClient(handler);
}
[Test]
public async Task Test0()
{
using (var response = await client.GetAsync($"http://localhost:3000/set-cookie/{cookieName}"))
{
Assert.AreEqual(HttpStatusCode.OK, response.StatusCode);
cookieValue = await response.Content.ReadAsStringAsync();
}
}
[Test]
public async Task Test1()
{
handler.AllowAutoRedirect = true;
using (var response = await client.GetAsync("http://localhost:3000/redirect-301"))
{
Assert.AreEqual(HttpStatusCode.OK, response.StatusCode);
Assert.AreEqual(await response.Content.ReadAsStringAsync(), "OK!");
}
}
[Test]
public async Task Test2()
{
handler.AllowAutoRedirect = false;
using (var response = await client.GetAsync("http://localhost:3000/redirect-301"))
{
Assert.AreEqual(HttpStatusCode.MovedPermanently, response.StatusCode);
}
}
[Test]
public async Task Test3()
{
using (var response = await client.GetAsync($"http://localhost:3000/get-cookie/{cookieName}"))
{
Assert.AreEqual(await response.Content.ReadAsStringAsync(), cookieValue);
}
}
}
Output via dotnet test:
Test Run Successful.
Total tests: 4
Passed: 4
Total time: 0.9352 Seconds
The question asks whether following redirects can be done on a case-by-case basis. While certainly useful for many common cases, I found the existing answers lacking in that regard.
The following implementation allows the decision on whether to follow a redirect or not to be configured on a true case-by-case basis via a predicate.
The solution is to override the SendAsync() method of HttpClientHandler.
using System;
using System.Net;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
namespace HttpClientCustomRedirectBehavior
{
static class Program
{
private const string REDIRECTING_URL = "http://stackoverflow.com/";
static async Task Main(string[] args)
{
HttpMessageHandler followRedirectAlwaysHandler = new RestrictedRedirectFollowingHttpClientHandler(
response => true);
HttpMessageHandler followRedirectOnlyToSpecificHostHandler = new RestrictedRedirectFollowingHttpClientHandler(
response => response.Headers.Location.Host == "example.com");
HttpResponseMessage response;
using (HttpClient followRedirectAlwaysHttpClient = new HttpClient(followRedirectAlwaysHandler))
{
response = await followRedirectAlwaysHttpClient.GetAsync(REDIRECTING_URL);
Console.WriteLine(response.StatusCode); // OK
}
using (HttpClient followRedirectOnlyToSpecificHostHttpClient = new HttpClient(followRedirectOnlyToSpecificHostHandler))
{
response = await followRedirectOnlyToSpecificHostHttpClient.GetAsync(REDIRECTING_URL);
Console.WriteLine(response.StatusCode); // Moved
}
followRedirectOnlyToSpecificHostHandler = new RestrictedRedirectFollowingHttpClientHandler(
response => response.Headers.Location.Host == "stackoverflow.com");
using (HttpClient followRedirectOnlyToSpecificHostHttpClient = new HttpClient(followRedirectOnlyToSpecificHostHandler))
{
response = await followRedirectOnlyToSpecificHostHttpClient.GetAsync(REDIRECTING_URL);
Console.WriteLine(response.StatusCode); // OK
}
}
}
public class RestrictedRedirectFollowingHttpClientHandler : HttpClientHandler
{
private static readonly HttpStatusCode[] redirectStatusCodes = new[] {
HttpStatusCode.Moved,
HttpStatusCode.Redirect,
HttpStatusCode.RedirectMethod,
HttpStatusCode.TemporaryRedirect,
HttpStatusCode.PermanentRedirect
};
private readonly Predicate<HttpResponseMessage> isRedirectAllowed;
public override bool SupportsRedirectConfiguration { get; }
public RestrictedRedirectFollowingHttpClientHandler(Predicate<HttpResponseMessage> isRedirectAllowed)
{
AllowAutoRedirect = false;
SupportsRedirectConfiguration = false;
this.isRedirectAllowed = response => {
return Array.BinarySearch(redirectStatusCodes, response.StatusCode) >= 0
&& isRedirectAllowed.Invoke(response);
};
}
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
int redirectCount = 0;
HttpResponseMessage response = await base.SendAsync(request, cancellationToken).ConfigureAwait(false);
while (isRedirectAllowed.Invoke(response)
&& (response.Headers.Location != request.RequestUri || response.StatusCode == HttpStatusCode.RedirectMethod && request.Method != HttpMethod.Get)
&& redirectCount < this.MaxAutomaticRedirections)
{
if (response.StatusCode == HttpStatusCode.RedirectMethod)
{
request.Method = HttpMethod.Get;
}
request.RequestUri = response.Headers.Location;
response = await base.SendAsync(request, cancellationToken).ConfigureAwait(false);
++redirectCount;
}
return response;
}
}
}
The Main method shows three example requests to http://stackoverflow.com (which is a URI that redirects to https://stackoverflow.com):
The first GET request will follow the redirect and therefore we see the status code OK of the response to the redirected request, because the handler is configured to follow all redirects.
The second GET request will not follow the redirect and therefore we see the status code Moved, because the handler is configured to follow redirects to the host example.com exclusively.
The third GET request will follow the redirect and therefore we see the status code OK of the response to the redirected request, because the handler is configured to follow redirects to the host stackoverflow.com exclusively.
Of course, you can substitute any custom logic for the predicate.
As you've probably discovered, you're not allowed to change the HttpClientHandler configuration after a request has been made.
Because your motivation for wanting to do this is to maintain the cookies between requests, then I propose something more like this (no exception/null reference handling included):
static CookieContainer cookieJar = new CookieContainer();
static async Task<HttpResponseMessage> GetAsync(string url, bool autoRedirect)
{
HttpResponseMessage result = null;
using (var handler = new HttpClientHandler())
using (var client = new HttpClient(handler))
{
handler.AllowAutoRedirect = autoRedirect;
handler.CookieContainer = cookieJar;
result = await client.GetAsync(url);
cookieJar = handler.CookieContainer;
}
return result;
}
Test:
static async Task Main(string[] args)
{
string url = #"http://stackoverflow.com";
using (var response = await GetAsync(url, autoRedirect: false))
{
Console.WriteLine($"HTTP {(int)response.StatusCode} {response.StatusCode}");
Console.WriteLine($"{response.Headers}");
Console.WriteLine("Cookies:");
Console.WriteLine($"{cookieJar.GetCookieHeader(new Uri(url))}\r\n");
}
Console.WriteLine(new string('-', 30));
using (var response = await GetAsync(url, autoRedirect: true))
{
Console.WriteLine($"HTTP {(int)response.StatusCode} {response.StatusCode}");
Console.WriteLine($"{response.Headers}");
Console.WriteLine("Cookies:");
Console.WriteLine($"{cookieJar.GetCookieHeader(new Uri(url))}\r\n");
}
Console.ReadLine();
}
I created a WebAPI but now I want to secure it with Basic Authorization.
// POST the data to the API
using (var client = new WebClient())
{
client.Headers.Add("Content-Type", "application/json");
client.Headers.Add(HttpRequestHeader.Authorization, "Basic" + Convert.ToBase64String(Encoding.ASCII.GetBytes(credentials)));
string json = JsonConvert.SerializeObject(ex);
string content = client.UploadString("http://myURL/v1/endpoint", json);
}
Below, how I post the data. Now, I would like to create a function that I can add to my controller or my Application_Start(). It will check:
if the request.Headers.Authorization is != null
if the request.Headers.Authorization.Scheme is != "Basic"
if there are some parameters
get the parameter and decode it to create a pair (SecretId/SecretKey)
call a service to check in the DB if there is a client with this pair
create an identity with IPrincipal
The thing is I don't know the best way is to create a customAttribute or a filter or something else. There is plenty of different way to do this but I would like to understand the difference.
Create the below-mentioned Filter in your project and use it at top of your web API method as :
**[BasicAuth]**
/// <summary>
/// Basic Authentication Filter Class
/// </summary>
public class BasicAuthAttribute : ActionFilterAttribute
{
/// <summary>
/// Called when [action executing].
/// </summary>
/// <param name="filterContext">The filter context.</param>
public override void OnActionExecuting(HttpActionContext filterContext)
{
try
{
if (filterContext.Request.Headers.Authorization == null)
{
// Client authentication failed due to invalid request.
filterContext.Response = new System.Net.Http.HttpResponseMessage()
{
StatusCode = HttpStatusCode.Unauthorized,
Content = new StringContent("{\"error\":\"invalid_client\"}", Encoding.UTF8, "application/json")
};
filterContext.Response.Headers.WwwAuthenticate.Add(new AuthenticationHeaderValue("Basic", "realm=xxxx"));
}
else if (filterContext.Request.Headers.Authorization.Scheme != "Basic" ||
string.IsNullOrEmpty(filterContext.Request.Headers.Authorization.Parameter))
{
// Client authentication failed due to invalid request.
filterContext.Response = new System.Net.Http.HttpResponseMessage()
{
StatusCode = HttpStatusCode.BadRequest,
Content = new StringContent("{\"error\":\"invalid_request\"}", Encoding.UTF8, "application/json")
};
}
else
{
var authToken = filterContext.Request.Headers.Authorization.Parameter;
Encoding encoding = Encoding.GetEncoding("iso-8859-1");
string usernamePassword = encoding.GetString(Convert.FromBase64String(authToken));
int seperatorIndex = usernamePassword.IndexOf(':');
string clientId = usernamePassword.Substring(0, seperatorIndex);
string clientSecret = usernamePassword.Substring(seperatorIndex + 1);
if (!ValidateApiKey(clientId, clientSecret))
{
// Client authentication failed due to invalid credentials
filterContext.Response = new System.Net.Http.HttpResponseMessage()
{
StatusCode = HttpStatusCode.Unauthorized,
Content = new StringContent("{\"error\":\"invalid_client\"}", Encoding.UTF8, "application/json")
};
}
// Successfully finished HTTP basic authentication
}
}
catch (Exception ex)
{
// Client authentication failed due to internal server error
filterContext.Response = new System.Net.Http.HttpResponseMessage()
{
StatusCode = HttpStatusCode.BadRequest,
Content = new StringContent("{\"error\":\"invalid_request\"}", Encoding.UTF8, "application/json")
};
}
}
/// <summary>
/// Validates the API key.
/// </summary>
/// <param name="recievedKey">The recieved key.</param>
/// <returns></returns>
private bool ValidateApiKey(string clientId, string clientSecret)
{
if (your condition satisfies)
{
return true;
}
return false;
}
}
I found few interesting articles about handlers/filter and attribute. I don't want to override [Authorize] so I will probably do an Authentication Filter.
Below some good links:
filter with attribute
filter
handler
#Nkosi: Cheers to confirm. I'm going to change the code a little bit because I don't want to use an Attribute but rather an filter that I put in the WebApiConfig
I'm trying to write a Owin midleware component that would LOG every incoming request and response to the database.
Here's how far I managed to get.
I got stuck on reading the response.body. Says:
Stream does not support reading.
How can I read the Response.Body ?
public class LoggingMiddleware : OwinMiddleware
{
private static Logger log = LogManager.GetLogger("WebApi");
public LoggingMiddleware(OwinMiddleware next, IAppBuilder app)
: base(next)
{
}
public override async Task Invoke(IOwinContext context)
{
using (var db = new HermesEntities())
{
var sw = new Stopwatch();
sw.Start();
var logRequest = new log_Request
{
Body = new StreamReader(context.Request.Body).ReadToEndAsync().Result,
Headers = Json.Encode(context.Request.Headers),
IPTo = context.Request.LocalIpAddress,
IpFrom = context.Request.RemoteIpAddress,
Method = context.Request.Method,
Service = "Api",
Uri = context.Request.Uri.ToString(),
UserName = context.Request.User.Identity.Name
};
db.log_Request.Add(logRequest);
context.Request.Body.Position = 0;
await Next.Invoke(context);
var mem2 = new MemoryStream();
await context.Response.Body.CopyToAsync(mem2);
var logResponse = new log_Response
{
Headers = Json.Encode(context.Response.Headers),
Body = new StreamReader(mem2).ReadToEndAsync().Result,
ProcessingTime = sw.Elapsed,
ResultCode = context.Response.StatusCode,
log_Request = logRequest
};
db.log_Response.Add(logResponse);
await db.SaveChangesAsync();
}
}
}
Response Body can be logged in this manner:
public class LoggingMiddleware : OwinMiddleware
{
private static Logger log = LogManager.GetLogger("WebApi");
public LoggingMiddleware(OwinMiddleware next, IAppBuilder app)
: base(next)
{
}
public override async Task Invoke(IOwinContext context)
{
using (var db = new HermesEntities())
{
var sw = new Stopwatch();
sw.Start();
var logRequest = new log_Request
{
Body = new StreamReader(context.Request.Body).ReadToEndAsync().Result,
Headers = Json.Encode(context.Request.Headers),
IPTo = context.Request.LocalIpAddress,
IpFrom = context.Request.RemoteIpAddress,
Method = context.Request.Method,
Service = "Api",
Uri = context.Request.Uri.ToString(),
UserName = context.Request.User.Identity.Name
};
db.log_Request.Add(logRequest);
context.Request.Body.Position = 0;
Stream stream = context.Response.Body;
MemoryStream responseBuffer = new MemoryStream();
context.Response.Body = responseBuffer;
await Next.Invoke(context);
responseBuffer.Seek(0, SeekOrigin.Begin);
var responseBody = new StreamReader(responseBuffer).ReadToEnd();
//do logging
var logResponse = new log_Response
{
Headers = Json.Encode(context.Response.Headers),
Body = responseBody,
ProcessingTime = sw.Elapsed,
ResultCode = context.Response.StatusCode,
log_Request = logRequest
};
db.log_Response.Add(logResponse);
responseBuffer.Seek(0, SeekOrigin.Begin);
await responseBuffer.CopyToAsync(stream);
await db.SaveChangesAsync();
}
}
}
Response body is a write-only network stream by default for Katana hosts. You will need to replace it with a MemoryStream, read the stream, log the content and then copy the memory stream content back into the original network stream. BTW, if your middleware reads the request body, downstream components cannot, unless the request body is buffered. So, you might need to consider buffering the request body as well. If you want to look at some code, http://lbadri.wordpress.com/2013/08/03/owin-authentication-middleware-for-hawk-in-thinktecture-identitymodel-45/ could be a starting point. Look at the class HawkAuthenticationHandler.
I've solved the problem by applying an action attribute writing the request body to OWIN environment dictionary. After that, the logging middleware can access it by a key.
public class LogResponseBodyInterceptorAttribute : ActionFilterAttribute
{
public override async Task OnActionExecutedAsync(HttpActionExecutedContext actionExecutedContext, CancellationToken cancellationToken)
{
if (actionExecutedContext?.Response?.Content is ObjectContent)
{
actionExecutedContext.Request.GetOwinContext().Environment["log-responseBody"] =
await actionExecutedContext.Response.Content.ReadAsStringAsync();
}
}
}
And then in the middleware:
public class RequestLoggingMiddleware
{
...
private void LogResponse(IOwinContext owinContext)
{
var message = new StringBuilder()
.AppendLine($"{owinContext.Response.StatusCode}")
.AppendLine(string.Join(Environment.NewLine, owinContext.Response.Headers.Select(x => $"{x.Key}: {string.Join("; ", x.Value)}")));
if (owinContext.Environment.ContainsKey("log-responseBody"))
{
var responseBody = (string)owinContext.Environment["log-responseBody"];
message.AppendLine()
.AppendLine(responseBody);
}
var logEvent = new LogEventInfo
{
Level = LogLevel.Trace,
Properties =
{
{"correlationId", owinContext.Environment["correlation-id"]},
{"entryType", "Response"}
},
Message = message.ToString()
};
_logger.Log(logEvent);
}
}
If you're facing the issue where the Stream does not support reading error occurs when trying to read the request body more than once, you can try the following workaround.
In your Startup.cs file, add the following middleware to enable buffering of the request body, which allows you to re-read the request body for logging purposes:
app.Use(async (context, next) =>
{
using (var streamCopy = new MemoryStream())
{
await context.Request.Body.CopyToAsync(streamCopy);
streamCopy.Position = 0;
string body = new StreamReader(streamCopy).ReadToEnd();
streamCopy.Position = 0;
context.Request.Body = streamCopy;
await next();
}
});
This middleware creates a copy of the request body stream, reads the entire stream into a string, sets the stream position back to the beginning, sets the request body to the copied stream, and then calls the next middleware.
After this middleware, you can now use context.Request.Body.Position = 0; to set the position of the request body stream back to the beginning so you can re-read the request body.
I hope this helps!