I upgraded my app to .net Core 3.1
And when I try to send a Post request by a http client to my web App
The body of the message arrives empty at the destination.
when i was checking the Request in my web App, i saw that the Request.ContentLenght value is completely correct.
I realized that there are problems with this in .net core 3.1,
Does anyone have a solution to this problem?
i thought it may be a Buffering issue, but i didn't find implementaion of EnableBuffering() method.
Here is my code:
Http Client who makes a post request for a web app:
StringContent httpContent = new StringContent("test Data");
httpContent.Headers.ContentType = new MediaTypeWithQualityHeaderValue("application/json");
var response = await myHttpClient.PostAsync(destUrl,httpContent);
Implementing a Post Request in My Web App:
[HttpPost]
public async Task<IActionResult> Post()
{
// Request.Body is empty.
}
If you are receiving an empty string, it means that something else already read it. Since Request.Body is a stream, you have to seek to the beginning to read it again.
But first you need to call EnableBufferring(), because by default Request.Body can only be read only once. You can do it Startup.cs file:
public void Configure(IApplicationBuilder app)
{
// be sure to add it before the other middleware
app.Use((context, next) =>
{
context.Request.EnableBuffering();
return next();
});
// everything else
}
Then you can read your Request.Body:
[HttpPost]
public async Task<IActionResult> Post()
{
using var reader = new StreamReader(Request.Body);
reader.BaseStream.Seek(0, SeekOrigin.Begin);
var rawMessage = await reader.ReadToEndAsync();
}
Related
I have an external endpoint which I call to get some Json response.
This endpoint will initiate a session to a POS device, so the device will show the request details and ask the customer to enter his credit card to complete the payment, then when the customer finishes; the POS will call the endpoint and it will return the result back to my application.
The problem here is that I need the operation to complete as described in this scenario (synchronously).
When I do the call to this endpoint from postman; it waits a lot of time (until the POS receives the request and customer do his entries then returns the results back to endpoint and endpoint returns the results back to Postman) ... this is all works fine.
The problem is when I do this from an ASP.NET Core app, the request is not waited for endpoint and the response is returned with null directly.
I need something to wait for it.
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Add("x-API-Key", "ApiKey");
client.DefaultRequestHeaders.Add("Connection", "keep-alive");
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
var postTask = client.PostAsJsonAsync(new Uri("terminalEndpoint here"), dto);//dto is the request payload
postTask.Wait();
var result = postTask.Result;
if (result.IsSuccessStatusCode)
{
//Should hang after this line to wait for POS
var terminalPaymentResponseDto = result.Content.ReadAsAsync<InitiateTerminalPaymentResponseDto>().Result;
//Should hit this line after customer finishes with POS device
return terminalPaymentResponseDto;
}
}
First of all, there's no need to block. In fact, in an ASP.NET Core application you should avoid blocking as much as possible. Use async and await instead. This allows ASP.NET Core to use the freed threadpool thread for other work.
Second, HttpClient is thread-safe and meant to be reused. Creating a new one every time in a using block leaks sockets. You could use a static instance but a better solution is to use IHttpClientFactory as Make HTTP requests using IHttpClientFactory in ASP.NET Core shows, to both reuse and recycle HttpClient instances automatically.
Finally, there's no reason to add these headers on every call. The Content-Type is set by PostAsJsonAsync anyway. I also suspect the API key doesn't change when calling the same server either.
In your Startup.cs or Program.cs you can use AddHttpClient to configure the API Key :
builder.Services.AddHttpClient(client=>{
client.DefaultRequestHeaders.Add("x-API-Key", "ApiKey");
});
After that you can inject IHttpClientFactory into your controllers or pages and call it asynchronously in asynchronous actions or handlers :
public class MyController:ControllerBase
{
private readonly IHttpClientFactory _httpClientFactory;
public MyController:ControllerBase(IHttpClientFactory httpClientFactory) =>
_httpClientFactory = httpClientFactory;
public async Task<InitiateTerminalPaymentResponseDto> PostAsync(MyDTO dto)
{
var client=_httpClientFactory.CreateClient();
var uri=new Uri("terminalEndpoint here");
var result = client.PostAsJsonAsync(uri, dto);payload
if (result.IsSuccessStatusCode)
{
//Should hang after this line to wait for POS
var paymentDto= await result.Content.ReadAsAsync<InitiateTerminalPaymentResponseDto>();
//Should hit this line after customer finishes with POS device
return paymentDto;
}
else {
//Do whatever is needed in case of error
}
}
}
Using HttpClientFactory allows adding retry strategies using Polly eg, to recover from a temporary network disconnection.
Why not use the await like below? And make sure to change the function to async
var postTask = await client.PostAsJsonAsync(new Uri("terminalEndpoint here"), dto);
I'm experiencing a problem when trying to use MultipartFormDataContent with HttpClient with a stream of data.
Context
I'm trying to upload a large file to ASP.NET Core Web API. A client should send the file via POST request form-data to a front-end API, which in turn should forward the file to a back-end API.
Because the file can be large, I followed the Microsoft example, i.e. I don't want to use IFormFile type but instead read the Request.Body using MultipartReader. This is to avoid loading the entire file into memory on the server, or saving it in a temporary file on server's hard drive.
Problem
The back-end API controller action looks as follows (this is almost directly copied from the ASP.NET Core 5.0 sample app with just minor simplifications):
[HttpPost]
[DisableRequestSizeLimit]
public async Task<IActionResult> ReceiveLargeFile()
{
var request = HttpContext.Request;
if (!request.HasFormContentType
|| !MediaTypeHeaderValue.TryParse(request.ContentType, out var mediaTypeHeader)
|| string.IsNullOrEmpty(mediaTypeHeader.Boundary.Value))
{
return new UnsupportedMediaTypeResult();
}
var reader = new MultipartReader(mediaTypeHeader.Boundary.Value, request.Body);
/* This throws an IOException: Unexpected end of Stream, the content may have already been read by another component. */
var section = await reader.ReadNextSectionAsync();
while (section != null)
{
var hasContentDispositionHeader = ContentDispositionHeaderValue.TryParse(section.ContentDisposition,
out var contentDisposition);
if (hasContentDispositionHeader
&& contentDisposition!.DispositionType.Equals("form-data")
&& !string.IsNullOrEmpty(contentDisposition.FileName.Value))
{
/* Fake copy to nothing since it doesn't even get here */
await section.Body.CopyToAsync(Stream.Null);
return Ok();
}
section = await reader.ReadNextSectionAsync();
}
return BadRequest("No files data in the request.");
}
I managed to reduce the problem slightly by making an integration test using Microsoft.AspNetCore.Mvc.Testing NuGet package. The following test replaces the front-end API, so instead of reading Request.Body stream in a Web API, the test just tries to add StreamContent to MultipartFormDataContent and post it via HttpClient to the back-end API:
[Fact]
public async Task Client_posting_to_Api_returns_Ok()
{
/* Arrange */
await using var stream = new MemoryStream();
await using var writer = new StreamWriter(stream);
await writer.WriteLineAsync("FILE CONTENTS");
await writer.FlushAsync();
stream.Position = 0;
using var client = _factory.CreateDefaultClient();
/* Act */
using var response =
await client.PostAsync(
"Receive",
new MultipartFormDataContent
{
{
new StreamContent(stream),
"file",
"fileName"
}
});
/* Assert */
Assert.Equal(HttpStatusCode.OK, response.StatusCode);
}
The back-end API controller then throws an IOException at await reader.ReadNextSectionAsync(), saying "Unexpected end of Stream, the content may have already been read by another component".
GitHub Repository (Complete Example)
I uploaded a complete example of the problem (including back-end API and the test) a GitHub repo.
Question
I must be doing something wrong. How can I forward a file received in a request with form-data content type in one service (front-end API) to another service (back-end API) without loading the entire file into memory or hard-drive in the front-end API, i.e. to just forward the stream of data to the back-end API?
Thanks in advance for any help.
I expected the same issue as you and it turned out that the MediaTypeHeaderValue.TryParse method parses the boundary value wrong as it wraps the string with '"' characters, because HttpClient sends the content type header like this:
multipart/form-data; boundary="blablabla"
So for me the solution was to add a Trim() method to boundary like this and pass that to the MultipartReader
var boundary = mediaTypeHeader.Boundary.Value.Trim('"');
var reader = new MultipartReader(boundary, request.Body);
There is exception being occurred at client side saying Error while copying the stream content and below that the response ended prematurely.
I cant figure out the solution. In the server side I have asp.net core web api which modify response stream. It actually reads what the controller send and encrypts it to a string and then writes to the response stream.
Also when when the content type is text/plain the response is shown on the Postman but when the content-type is application/json the content is not shown but in the header I can see the content length having some numbers. And for client side, the above exception occurs for both the content-type.
So what i am missing in my middleware code?? I know this is causing issue because when i comment out MyMiddleware in startup.cs, the normal flow works.
Below is the Invoke function in middleware on the server side
public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
string reponseContent = string.Empty;
// Store the "pre-modified" response stream.
var existingBody = context.Response.Body;
using (var newBody = new MemoryStream())
{
// We set the response body to our stream so we can read after the chain of middlewares have been called.
context.Response.Body = newBody;
await next(context);
// Set the stream back to the original.
context.Response.Body = existingBody;
newBody.Seek(0, SeekOrigin.Begin);
//reading the content
var contentReader = new StreamReader(newBody);
reponseContent = await contentReader.ReadToEndAsync();
string encryptedData = _cryptoService.Encrypt(reponseContent);
// Send our modified content to the response body.
await context.Response.WriteAsync(encryptedData);
}
I am not sure how you are calling your MiddleWare. I have successfully reproduce the issue and get the response accordingly.
Middle Ware Class:
public class CustomMiddleware
{
private readonly RequestDelegate next;
public CustomMiddleware(RequestDelegate next)
{
this.next = next;
}
public async Task InvokeAsync(HttpContext context)
{
string reponseContent = string.Empty;
// Store the "pre-modified" response stream.
var existingBody = context.Response.Body;
using (var newBody = new MemoryStream())
{
// We set the response body to our stream so we can read after the chain of middlewares have been called.
context.Response.Body = newBody;
await next(context);
// Set the stream back to the original.
context.Response.Body = existingBody;
newBody.Seek(0, SeekOrigin.Begin);
//reading the content
var contentReader = new StreamReader(newBody);
reponseContent = await contentReader.ReadToEndAsync();
// string encryptedData = _cryptoService.Encrypt(reponseContent);
// Send our modified content to the response body.
await context.Response.WriteAsync(reponseContent);
}
}
}
Note: You should use constructor to invoke your RequestDelegate like this way. But you have designed this with two parameter, not sure how you are passing the argument while calling.
Startup.cs:
Calling Middleware In Startup.cs under Configure like this way
app.UseMiddleware<CustomMiddleware>();
Request From Postman:
I have tested with simple plain text and application/json type. Sent request to my controller and modify the argument on controller body, and the changed implemented on the middleware.
Controller:
[HttpPost]
public IActionResult MiddlewareReStream([FromBody] Plans plan)
{
plan.PlanName = "New Text";
return Ok(plan);
}
MiddlWare Output:
Note: Notice that I have invoke the request with "PlanName":"Test Plan" and modified the parameter which middleware successfully invoked the changes.
PostMan:
Note: Make sure you have called or implemented the InvokeAsync Middleware accordingly. Because I got the response as you are expecting.
Im thinking that the "Content-Length" header of the response represents the size of the data before you encrypt it and that you need to recalculate the size based on the new encrypted data and reset the header. Im also thinking that different servers are reacting to this missmatch differently, That it might work on IIS but not Kestrel.
In short make sure your "Content-Length" and "Content-Type" headers are matching what you are actually sending.
I am sending a huge number (30 000 calls one by one) of JSONs over HTTP to Web API endpoint hosted on Azure. While most of them are processed just fine, about 1%-2% of JSONs received by WebAPI is simply corrupted and cannot be deserialized.
The code that sends messages looks following:
private async Task Send(string messageBody)
{
try
{
HttpRequestMessage requestMessage = new HttpRequestMessage(HttpMethod.Post, _configurationService.ServiceUrl);
requestMessage.Content = new StringContent(messageBody, Encoding.UTF8);
requestMessage.Content.Headers.ContentType = new MediaTypeHeaderValue("application/json");
HttpResponseMessage responseMessage = await _httpClient.SendAsync(requestMessage);
... // do rest of the stuff - check http codes, handle exceptions, etc
}
The messageBody string is a json array of 100 items serialized with Json.net. Sample:
[{"ContentSnippet":"Body142370","ContentImageUrl":null,"ContentNotFullySupported":false,"Title":"142370","ContentType":9,"ContentId":"98fdb109-6548-40b9-900b-6914f16a6481","ActionType":14,"ActionDate":"2016-08-31T07:42:45.3161067Z","ActionByUserFirstName":"W","ActionByUserLastName":"J","ActionByUserId":"92caa514-7168-4085-bc17-85ef2fd2a682","UserId":"96e243eb-97be-4d36-9881-74e761def5aa","CommunityId":"10d44cf3-44ea-451e-8bfb-5275ba2f28a5","NotificationType":0}, {...next item in array...}]
The backend code I am using for diagnosing it looks following:
[HttpPost]
[Route("api/feed")]
public HttpResponseMessage Insert(IEnumerable<FeedItem> feedItems)
{
Stream s = Request.Content.ReadAsStreamAsync().Result;
s.Seek(0, System.IO.SeekOrigin.Begin);
using (StreamReader reader = new StreamReader(s))
{
string reqestBody = reader.ReadToEnd();
}
What I get on backend side (in requestBody variable) is:
[""ContentSnippet"":""Body142370"",""ContentImageUrl"":null,""ContentNotFullySupported"":false,""Title"":""142370"",""ContentType"":9,""ContentId"":""98fdb109-6548-40b9-900b-6914f16a6481"",""ActionType"":14,""ActionDate"":""2016-08-31T07:42:45. e����3�� v��tionByUse��E<stName""��K��UCctionByUserLast��0���d�v���r�P�gByUserId"":""92UŇ���U�sH/ia�y��""L)?8:�ԙN�JY�:,���X�1;�%�x!; `��,�981-74
Interestingly - it happens only for 1% - 2% of calls. More interestingly - when I try to simply resend the same JSON, everything works fine. So its a phantom issue (i dont see any patterns indicating when) - but still it happens.
I tried changing HttpClient to RestSharp but the result were event worse.
What is going on here?
I have problem understanding sending Json from WebApi to Android. I know how to make requests from Json, but don't know how to make Task that waits for request and return Json. I have json made from Mongo DB query with JsonConvert.SerializeObject.
I have been searching and found this that I have for now, but this isn't what I need I think, I don't have url in PostAsync, I just want to wait for request and respond with Json, where I need to look for:
public async Task<HttpStatusCode> SendAsync()
{
var content = new StringContent(
jsonString,
Encoding.UTF8,
"application/x-www-form-urlencoded");
HttpResponseMessage response = await _httpClient.PostAsync(_url, content);
return response.StatusCode;
}
You need to return an IHttpActionResult implementation. For example:
public async Task<IHttpActionResult> SendAsync()
{
// Object passed to Ok will be automatically serialized to JSON
// if response content type is JSON (and, by default, this is the serialization
// format in ASP.NET WebAPI)
return Ok(new { text = "hello world" });
}