There is exception being occurred at client side saying Error while copying the stream content and below that the response ended prematurely.
I cant figure out the solution. In the server side I have asp.net core web api which modify response stream. It actually reads what the controller send and encrypts it to a string and then writes to the response stream.
Also when when the content type is text/plain the response is shown on the Postman but when the content-type is application/json the content is not shown but in the header I can see the content length having some numbers. And for client side, the above exception occurs for both the content-type.
So what i am missing in my middleware code?? I know this is causing issue because when i comment out MyMiddleware in startup.cs, the normal flow works.
Below is the Invoke function in middleware on the server side
public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
string reponseContent = string.Empty;
// Store the "pre-modified" response stream.
var existingBody = context.Response.Body;
using (var newBody = new MemoryStream())
{
// We set the response body to our stream so we can read after the chain of middlewares have been called.
context.Response.Body = newBody;
await next(context);
// Set the stream back to the original.
context.Response.Body = existingBody;
newBody.Seek(0, SeekOrigin.Begin);
//reading the content
var contentReader = new StreamReader(newBody);
reponseContent = await contentReader.ReadToEndAsync();
string encryptedData = _cryptoService.Encrypt(reponseContent);
// Send our modified content to the response body.
await context.Response.WriteAsync(encryptedData);
}
I am not sure how you are calling your MiddleWare. I have successfully reproduce the issue and get the response accordingly.
Middle Ware Class:
public class CustomMiddleware
{
private readonly RequestDelegate next;
public CustomMiddleware(RequestDelegate next)
{
this.next = next;
}
public async Task InvokeAsync(HttpContext context)
{
string reponseContent = string.Empty;
// Store the "pre-modified" response stream.
var existingBody = context.Response.Body;
using (var newBody = new MemoryStream())
{
// We set the response body to our stream so we can read after the chain of middlewares have been called.
context.Response.Body = newBody;
await next(context);
// Set the stream back to the original.
context.Response.Body = existingBody;
newBody.Seek(0, SeekOrigin.Begin);
//reading the content
var contentReader = new StreamReader(newBody);
reponseContent = await contentReader.ReadToEndAsync();
// string encryptedData = _cryptoService.Encrypt(reponseContent);
// Send our modified content to the response body.
await context.Response.WriteAsync(reponseContent);
}
}
}
Note: You should use constructor to invoke your RequestDelegate like this way. But you have designed this with two parameter, not sure how you are passing the argument while calling.
Startup.cs:
Calling Middleware In Startup.cs under Configure like this way
app.UseMiddleware<CustomMiddleware>();
Request From Postman:
I have tested with simple plain text and application/json type. Sent request to my controller and modify the argument on controller body, and the changed implemented on the middleware.
Controller:
[HttpPost]
public IActionResult MiddlewareReStream([FromBody] Plans plan)
{
plan.PlanName = "New Text";
return Ok(plan);
}
MiddlWare Output:
Note: Notice that I have invoke the request with "PlanName":"Test Plan" and modified the parameter which middleware successfully invoked the changes.
PostMan:
Note: Make sure you have called or implemented the InvokeAsync Middleware accordingly. Because I got the response as you are expecting.
Im thinking that the "Content-Length" header of the response represents the size of the data before you encrypt it and that you need to recalculate the size based on the new encrypted data and reset the header. Im also thinking that different servers are reacting to this missmatch differently, That it might work on IIS but not Kestrel.
In short make sure your "Content-Length" and "Content-Type" headers are matching what you are actually sending.
Related
I'm using AddOpenIdConnect and need to modify the response in case the OnRedirectToIdentityProvider event is raised. Within this event the response status is modified to 401 and I would like to set a custom message. To write this custom message, I've created the SetResponseBody method.
The solution of this post is used to set the response status, but I need to modify the Body as well.
I'm calling the SetResponseBody method (a custom method which I implemented) in order to modify the response body as soon as the OnRedirectToIdentityProvider event is raised from AddOpenIdConnect.'
As mentioned in one of the comments by #Panagiotis Kanavos in the post, the SetResponseBody method doesn't seem to be a correct solution (despite the response actually contains valid json). Could you provide an alternative?
Summarized: I would like to return a custom response besides the status code 401.
OnRedirectToIdentityProvider = async e =>
{
// e is of type RedirectContext
if (e.Request.Path.StartsWithSegments("/api")))
{
if (e.Response.StatusCode == (int)HttpStatusCode.OK)
{
e.Response.StatusCode = (int)HttpStatusCode.Unauthorized;
// TestMessage is a const
// e.Response is readonly (get) so it's not possible to set it directly.
await ResponseRewriter.SetResponseBody(e.Response, TestMessage);
}
e.HandleResponse();
}
await Task.CompletedTask;
}
with ResponseRewriter.SetResponseBody defined as follows
public static async Task SetResponseBody(HttpResponse response, string message)
{
var originalResponseBody = response.Body;
var responseBody = new MemoryStream();
response.Body = responseBody;
response.ContentType = "application/json";
var body = new { Message = message };
string json = JsonSerializer.Serialize(body);
await response.WriteAsync(json);
response.Body.Seek(0, SeekOrigin.Begin);
await responseBody.CopyToAsync(originalResponseBody);
}
The last two lines are implemented as written in this post.
Good day,
I have an issue with a custom response in API Gateway Ocelot with Middleware.
inside FormatResponse(context.Response) I change response for specific endpoint and I see the new response on debug but I receive the original Response in final result on postman.
ex :
original Response
{
"name":"mo"
}
after a change will be
{
"name":"mo123"
}
my code
// .NET Core 3.1
public async Task InvokeAsync(HttpContext context)
{
context.Request.EnableBuffering();
var builder = new StringBuilder();
var request = await FormatRequest(context.Request);
builder.Append("Request: ").AppendLine(request);
builder.AppendLine("Request headers:");
foreach (var header in context.Request.Headers)
{
builder.Append(header.Key).Append(':').AppendLine(header.Value);
}
//Copy a pointer to the original response body stream
var originalBodyStream = context.Response.Body;
//Create a new memory stream...
using var responseBody = new MemoryStream();
//...and use that for the temporary response body
context.Response.Body = responseBody;
//Continue down the Middleware pipeline, eventually returning to this class
await _next(context);
//Format the response from the server
var response = await FormatResponse(context.Response); // here ,i see new respose
builder.Append("Response: ").AppendLine(response);
builder.AppendLine("Response headers: ");
foreach (var header in context.Response.Headers)
{
builder.Append(header.Key).Append(':').AppendLine(header.Value);
}
//Save log to chosen datastore
_logger.LogInformation(builder.ToString());
//Copy the contents of the new memory stream (which contains the response) to the original
// stream, which is then returned to the client.
await responseBody.CopyToAsync(originalBodyStream);
}
private async Task<string> FormatResponse(HttpResponse response)
{
//We need to read the response stream from the beginning...
response.Body.Seek(0, SeekOrigin.Begin);
//...and copy it into a string
string text = await new StreamReader(response.Body).ReadToEndAsync();
text = CustomRes(text); //************************ here, I change response
//We need to reset the reader for the response so that the client can read it.
response.Body.Seek(0, SeekOrigin.Begin);
//Return the string for the response, including the status code (e.g. 200, 404, 401, etc.)
return $"{response.StatusCode}: {text}";
}
Reference: https://www.abhith.net/blog/dotnet-core-api-gateway-ocelot-logging-http-requests-response-including-headers-body/
The best answer from Richard Deeming:
https://www.codeproject.com/Questions/5294847/Fix-issue-with-custom-response-NET-core-API-gatewa
I upgraded my app to .net Core 3.1
And when I try to send a Post request by a http client to my web App
The body of the message arrives empty at the destination.
when i was checking the Request in my web App, i saw that the Request.ContentLenght value is completely correct.
I realized that there are problems with this in .net core 3.1,
Does anyone have a solution to this problem?
i thought it may be a Buffering issue, but i didn't find implementaion of EnableBuffering() method.
Here is my code:
Http Client who makes a post request for a web app:
StringContent httpContent = new StringContent("test Data");
httpContent.Headers.ContentType = new MediaTypeWithQualityHeaderValue("application/json");
var response = await myHttpClient.PostAsync(destUrl,httpContent);
Implementing a Post Request in My Web App:
[HttpPost]
public async Task<IActionResult> Post()
{
// Request.Body is empty.
}
If you are receiving an empty string, it means that something else already read it. Since Request.Body is a stream, you have to seek to the beginning to read it again.
But first you need to call EnableBufferring(), because by default Request.Body can only be read only once. You can do it Startup.cs file:
public void Configure(IApplicationBuilder app)
{
// be sure to add it before the other middleware
app.Use((context, next) =>
{
context.Request.EnableBuffering();
return next();
});
// everything else
}
Then you can read your Request.Body:
[HttpPost]
public async Task<IActionResult> Post()
{
using var reader = new StreamReader(Request.Body);
reader.BaseStream.Seek(0, SeekOrigin.Begin);
var rawMessage = await reader.ReadToEndAsync();
}
I am developing a .net core middle-ware (api) and thinking to use pipes with following flow, Can someone tell me is this is a good approach and comply best practices or should i use different strategy.
Request comes to api
Authorization pipe validates the request.
Request pipe logs the request into db.
Request goes to api and perform action and return a result.
Response pipe gets the response and logs into db and return the result to client.
I know that we can read stream only time (point 3) but i figured this out already and after reading i have attach it to request stream again.
So, confusion is where to write the response? In api? or in separate pipe.
If i do it in separate pipe then i am handling my response two time (one is creating response in api, second is reading response in separate pipe) which is a performance hit.
Can i pass the data from point number 4 to 5 like from api to my pipe and from there that response should added into response stream and if it is correct then how can i pass the data from api to pipe?
Yes, response stream can only be read once. You can use the MemoryStream to load the response , reference article :
First, read the request and format it into a string.
Next, create a dummy MemoryStream to load the new response into.
Then, wait for the server to return a response.
Finally, copy the dummy MemoryStream (containing the actual response) into the original stream, which gets returned to the client.
Code sample :
public class RequestResponseLoggingMiddleware
{
private readonly RequestDelegate _next;
public RequestResponseLoggingMiddleware(RequestDelegate next)
{
_next = next;
}
public async Task Invoke(HttpContext context)
{
//First, get the incoming request
var request = await FormatRequest(context.Request);
//Copy a pointer to the original response body stream
var originalBodyStream = context.Response.Body;
//Create a new memory stream...
using (var responseBody = new MemoryStream())
{
//...and use that for the temporary response body
context.Response.Body = responseBody;
//Continue down the Middleware pipeline, eventually returning to this class
await _next(context);
//Format the response from the server
var response = await FormatResponse(context.Response);
//TODO: Save log to chosen datastore
//Copy the contents of the new memory stream (which contains the response) to the original stream, which is then returned to the client.
await responseBody.CopyToAsync(originalBodyStream);
}
}
private async Task<string> FormatRequest(HttpRequest request)
{
var body = request.Body;
//This line allows us to set the reader for the request back at the beginning of its stream.
request.EnableRewind();
//We now need to read the request stream. First, we create a new byte[] with the same length as the request stream...
var buffer = new byte[Convert.ToInt32(request.ContentLength)];
//...Then we copy the entire request stream into the new buffer.
await request.Body.ReadAsync(buffer, 0, buffer.Length);
//We convert the byte[] into a string using UTF8 encoding...
var bodyAsText = Encoding.UTF8.GetString(buffer);
//..and finally, assign the read body back to the request body, which is allowed because of EnableRewind()
request.Body = body;
return $"{request.Scheme} {request.Host}{request.Path} {request.QueryString} {bodyAsText}";
}
private async Task<string> FormatResponse(HttpResponse response)
{
//We need to read the response stream from the beginning...
response.Body.Seek(0, SeekOrigin.Begin);
//...and copy it into a string
string text = await new StreamReader(response.Body).ReadToEndAsync();
//We need to reset the reader for the response so that the client can read it.
response.Body.Seek(0, SeekOrigin.Begin);
//Return the string for the response, including the status code (e.g. 200, 404, 401, etc.)
return $"{response.StatusCode}: {text}";
}
}
And register the middleware :
app.UseMiddleware<RequestResponseLoggingMiddleware>();
We are trying to implement user-determined (on a settings screen) optional gzip compression in our client which uses HttpClient, so we can log and compare performance across a number of different calls over a period of time. Our first attempt was to simply conditionally add the header as follows:
HttpRequestMessage request = new HttpRequestMessage(Method, Uri);
if (AcceptGzipEncoding)
{
_client.DefaultRequestHeaders.AcceptEncoding.Add(new System.Net.Http.Headers.StringWithQualityHeaderValue("gzip"));
}
//Send to the server
result = await _client.SendAsync(request);
//Read the content of the result response from the server
content = await result.Content.ReadAsStringAsync();
This created the correct request, but the gzipped response was not decompressed on return, resulting in a garbled response. I found that we had to include the HttpClientHandler when constructing the HttpClient:
HttpClient _client = new HttpClient(new HttpClientHandler
{
AutomaticDecompression = DecompressionMethods.GZip
});
This all works well, but we'd like to change whether the client sends the Accept-Encoding: gzip header at runtime, and there doesn't appear to be any way to access or change the HttpClientHandler after it's passed to the HttpClient constructor. In addition, altering the headers of the HttpRequestMessage object doesn't have any effect on the headers of the request if they are defined by the HttpClientHandler.
Is there any way to do this without recreating the HttpClient each time this changes?
Edit: I've also tried to modify a reference to the HttpClientHandler to change AutomaticDecompression at runtime, but that's throwing this exception:
This instance has already started one or more requests. Properties can only be modified before sending the first request.
You're almost there with the first example, you just need to deflate the stream yourself. MS's GZipSteam will help with this:
HttpRequestMessage request = new HttpRequestMessage(Method, Uri);
if (AcceptGzipEncoding)
{
_client.DefaultRequestHeaders.AcceptEncoding.Add(new System.Net.Http.Headers.StringWithQualityHeaderValue("gzip"));
}
//Send to the server
result = await _client.SendAsync(request);
//Read the content of the result response from the server
using (Stream stream = await result.Content.ReadAsStreamAsync())
using (Stream decompressed = new GZipStream(stream, CompressionMode.Decompress))
using (StreamReader reader = new StreamReader(decompressed))
{
content = reader.ReadToEnd();
}
If you want to use the same HttpClient and only want to enable compression for some requests, you are not able to use automatic decompression. When automatic decompression is enabled, the framework also resets the Content-Encoding header of the response. This means that you are unable to find out whether the response was really compressed or not. By the way, also the Content-Length header of the response matches the size of the decompressed content if you turn on automatic decompression.
So you need to decompress the content manually. The following sample shows an implementation for gzip-compressed content (as also shown in #ToddMenier's response):
private async Task<string> ReadContentAsString(HttpResponseMessage response)
{
// Check whether response is compressed
if (response.Content.Headers.ContentEncoding.Any(x => x == "gzip"))
{
// Decompress manually
using (var s = await response.Content.ReadAsStreamAsync())
{
using (var decompressed = new GZipStream(s, CompressionMode.Decompress))
{
using (var rdr As New IO.StreamReader(decompressed))
{
return await rdr.ReadToEndAsync();
}
}
}
else
// Use standard implementation if not compressed
return await response.Content.ReadAsStringAsync();
}
As per the comments above, recreating the HttpClient is really the only (robust) way to do this. Manual decompression can be achieved but it seems to be very difficult to reliably/efficiently determine whether the content has been encoded or not, to determine whether to apply decoding.