We Keep Coding

Parse HTTP request body to JSON string in .net core 3.0

I have implemented the following method format request body
private async Task<string> FormatRequest(HttpRequest request)
{
request.EnableBuffering();
//Create a new byte[] with the same length as the request stream
var buffer = new byte[Convert.ToInt32(request.ContentLength)];
//Copy the entire request stream into the new buffer
await request.Body.ReadAsync(buffer, 0, buffer.Length).ConfigureAwait(false);
//Convert the byte[] into a string using UTF8 encoding
var bodyAsText = Encoding.UTF8.GetString(buffer);
request.Body.Position = 0;
return bodyAsText;
}
I got the following result
------WebKitFormBoundaryY8OPXY2MlrKMjBRe
Content-Disposition: form-data; name="RoleId"
2
------WebKitFormBoundaryY8OPXY2MlrKMjBRe
Content-Disposition: form-data; name="AuthenticationSettingsId"
3
.....
Expected result
"{\"fields\":[\"RoleId\",\"2\",\"AuthenticationSettingsId\",\"1\",\"recommendation\",\"reviewerId\"],\"size\":100,\"filter\":[{\"id\":\"ApplicationId\",\"operator\":\"and\",\"parent\":\"\",\"nested\":false,\"type\":\"integer\",\"value\":[360]}],\"aggregate\":[],\"sort\":[]}"
Note: Previously we used request.EnableRewind() it was returning the above result and later upgraded to .net core 3.0

Here is a high level of how I handle JSON queries. If you really want to get fancy you can implement all this into an abstract class and inherit direct to your data model.
There are plenty of different ways to get where you want to be, hopefully this helps you get there.
I've put comments in the code, but feel free to ask away if something doesn't make sense.
class SomeHttpJsonUtility
{
//If you want to parse your return data
//directly into a data model
class DataModel{
class ReturnData
{
[JsonPropertyName("fields")]
public Field[] Fields { get; set; }
}
class Field
{
[JsonPropertyName("RoleId")]
public int RoleId { get; set; }
//...you get the idea
}
}
//Some data if your sending a post request
private Dictionary<string, string> postParameters = new Dictionary<string, string>();
//Creates a HTTP Client With Specified Parameters
//You can do this any number of ways depending on the
//source you are querying
private HttpClient GetClient()
{
HttpClient _client = new HttpClient();
_client.DefaultRequestHeaders.Clear();
_client.DefaultRequestHeaders.Add(
"UserAgent",
new string[] { "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:103.0) Gecko/20100101 Firefox/103.0" });
_client.DefaultRequestHeaders.Add(
"AcceptLanguage",
new string[] { "en-US" });
_client.DefaultRequestHeaders.Add(
"AcceptEncoding",
new string[] { "gzip", "deflate", "br" });
_client.DefaultRequestHeaders.Add(
"Accept",
new string[] { "*/*" });
_client.DefaultRequestHeaders.Add(
"Connection",
new string[] { "keep-alive" });
return _client;
}
private void GetJson(Uri from_uri)
{
//Get the HttpClient With Proper Request Headers
HttpClient _client =
GetClient();
Task.Run(async () =>
{
//If your data comes from a get request
HttpResponseMessage _httpResponse =
await _client.GetAsync(
requestUri:from_uri);
//Or if your response comes from a post
_httpResponse =
await _client.PostAsync(
requestUri: from_uri,
content: new FormUrlEncodedContent(postParameters)
);
//since your initial post used a stream, we can
//keep going in that direction
//Initilize a memory stream to process the data
using(MemoryStream _ms = new MemoryStream())
{
//Send the http response content
////into the memory stream
await _httpResponse.Content.CopyToAsync(
stream: _ms);
//Goto the start of the memory stream
_ms.Seek(
offset: 0,
loc: SeekOrigin.Begin);
//Option 1:
//Send direct to data model
// This is utilizing the Microsoft Library:
// System.Text.Json.Serialization;
DataModel dataModel =
JsonSerializer.Deserialize<DataModel>(
utf8Json: _ms);
//Option 2:
//Send to a string
using(StreamReader _sr = new StreamReader(_ms))
{
string dataAsSting = _sr.ReadToEnd();
}
}
}).Wait();
}
}
If your query is only a Get request, then it's pretty easy get get the exact headers you need.
Using Firefox hit F12 and goto the web address.
Click the Network Tab, then Headers and view the request data.
You really only need a few of these:
Accept
Accept-Encoding
Accept-Language
Connection
User-Agent
Mozilla has some nice resources regarding the different header objects.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept
Host should be taken care of by the HttpClient.
Cookies should be handled by the HttpClient (if you need them)
If you are actually getting the data back as Gzip you'll have to implement a reader, unless the HttpClient you are using will automatically decode it.
And at the end, victory :-)

I think you need to set the content-type on the request when you send it to application/json

Could you try to read this way?
var reader = new System.IO.StreamReader(request.Body);
var body = reader.ReadToEndAsync().Result;
Then you can use Newtonsoft or a similar library on body.

You need to to tokenize / encode your string with some JSON encoder.
Here you have two choices:
the internal (Microsoft) JsonConverter
the Newtonsoft.Json JsonConverter

Karthik, it appears you are sending a multipart request from a webkit browser.
If you would be able to just change it on the client side from multipart to application/json your problem would be fixed.
If this is not possible, you can just use:
private async Task<string> FormatRequest(HttpRequest request)
{
var form = request.Form.ToDictionary(x => x.Key, x => x.Value);
return JsonSerializer.Serialize(form);
}
This could parses your form into a dictionary and returns it as a Json.
(This code is written in dotnet 6, which has System.Text.Json. If you need to stick in .net 3.1, you would need to use a JsonSerializer like Newtonsoft.)

Disable ModelBinder in .NetCore

HeyGuys
I'm working on a WebApi project that receives requests from clients and redirects these requests to other services that are not open for direct access.
By default, .Net serializes and deserializes the Json request parameters automatically, so I need to re-serialize them before calling the appropriate service. The same problem occurs when receiving the service response. I need to deserialize it before sending the response to the user; otherwise .Net framework will serialize it one more time, resulting in a "Json of Json" response.
I found this answer but it does not seem to work with .NetCore; so I tried to create my own ModelBinder that just reads the Json object and returns it.
class JsonUnformatterBinderProvider : Microsoft.AspNetCore.Mvc.ModelBinding.IModelBinderProvider
{
public IModelBinder GetBinder(ModelBinderProviderContext context)
{
return new JsonUnformatterBinder(new SimpleTypeModelBinder(context.Metadata.ModelType));
}
}
and
class JsonUnformatterBinder : IModelBinder
{
private readonly IModelBinder _fallbackBinder;
public JsonUnformatterBinder(IModelBinder fallbackBinder)
{
_fallbackBinder = fallbackBinder;
}
public Task BindModelAsync(ModelBindingContext bindingContext)
{
string currMethod = bindingContext.ActionContext.HttpContext.Request.Method;
if ("POST".Equals(currMethod) || "PUT".Equals(currMethod))
{
string strData = new StreamReader(bindingContext.ActionContext.HttpContext.Request.Body).ReadToEnd();
bindingContext.Result = ModelBindingResult.Success(strData);
return Task.CompletedTask;
}
return _fallbackBinder.BindModelAsync(bindingContext);
}
}
This code is very simple, it was my first attempt and it worked well for my purposes. However, I still get the "Json of Json" problem when I take the second service answer and returns back to the user.
I basically have no idea what I can do to overcome this, so any workaround is welcome here.

If you need just redirect a request without modification, you could read it from input stream directly and send it to inner service. You could also use such approach to read responce from inner service.
//1. Set empty parameter list in action then neither serializator nor model binder are not invoked.
public async Task<ContentResult> ProxyAction(/*empty parameter list*/)
{
var newUrl = #"https://stackoverflow.com";
var data = this.Request.Body;
using (var client = new HttpClient())
{
//2. Read request body from input stream.
var reader = new StreamReader(data);
var json = reader.ReadToEnd();
using (var content = new StringContent(json))
{
//3. Set correct content type
content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue(this.Request.ContentType);
//4. Post request to inner service
var response = await client.PostAsync(newUrl, content);
//5. Read response without deserialization
var innerResponse = await response.Content.ReadAsStringAsync();
var contentType = response.Content.Headers.ContentType.ToString();
var statusCode = response.StatusCode;
//6. Return inner response without serialization
var outerResponse = this.Content(innerResponse, contentType);
outerResponse.StatusCode = (int)statusCode;
return outerResponse;
}
}
}

How do I get the raw request body from the Request.Content object using .net 4 api endpoint

I'm trying to capture the raw request data for accountability and want to pull the request body content out of the Request object.
I've seen suggestions doing a Request.InputStream, but this method is not available on the Request object.
Any idea of how to get a string representation of the Request.Content body?

In your comment on #Kenneth's answer you're saying that ReadAsStringAsync() is returning empty string.
That's because you (or something - like model binder) already read the content, so position of internal stream in Request.Content is on the end.
What you can do is this:
public static string GetRequestBody()
{
var bodyStream = new StreamReader(HttpContext.Current.Request.InputStream);
bodyStream.BaseStream.Seek(0, SeekOrigin.Begin);
var bodyText = bodyStream.ReadToEnd();
return bodyText;
}

You can get the raw data by calling ReadAsStringAsAsync on the Request.Content property.
string result = await Request.Content.ReadAsStringAsync();
There are various overloads if you want it in a byte or in a stream. Since these are async-methods you need to make sure your controller is async:
public async Task<IHttpActionResult> GetSomething()
{
var rawMessage = await Request.Content.ReadAsStringAsync();
// ...
return Ok();
}
EDIT: if you're receiving an empty string from this method, it means something else has already read it. When it does that, it leaves the pointer at the end. An alternative method of doing this is as follows:
public IHttpActionResult GetSomething()
{
var reader = new StreamReader(Request.Body);
reader.BaseStream.Seek(0, SeekOrigin.Begin);
var rawMessage = reader.ReadToEnd();
return Ok();
}
In this case, your endpoint doesn't need to be async (unless you have other async-methods)

For other future users who do not want to make their controllers asynchronous, or cannot access the HttpContext, or are using dotnet core (this answer is the first I found on Google trying to do this), the following worked for me:
[HttpPut("{pathId}/{subPathId}"),
public IActionResult Put(int pathId, int subPathId, [FromBody] myViewModel viewModel)
{
var body = new StreamReader(Request.Body);
//The modelbinder has already read the stream and need to reset the stream index
body.BaseStream.Seek(0, SeekOrigin.Begin);
var requestBody = body.ReadToEnd();
//etc, we use this for an audit trail
}

If you need to both get the raw content from the request, but also need to use a bound model version of it in the controller, you will likely get this exception.
NotSupportedException: Specified method is not supported.
For example, your controller might look like this, leaving you wondering why the solution above doesn't work for you:
public async Task<IActionResult> Index(WebhookRequest request)
{
using var reader = new StreamReader(HttpContext.Request.Body);
// this won't fix your string empty problems
// because exception will be thrown
reader.BaseStream.Seek(0, SeekOrigin.Begin);
var body = await reader.ReadToEndAsync();
// Do stuff
}
You'll need to take your model binding out of the method parameters, and manually bind yourself:
public async Task<IActionResult> Index()
{
using var reader = new StreamReader(HttpContext.Request.Body);
// You shouldn't need this line anymore.
// reader.BaseStream.Seek(0, SeekOrigin.Begin);
// You now have the body string raw
var body = await reader.ReadToEndAsync();
// As well as a bound model
var request = JsonConvert.DeserializeObject<WebhookRequest>(body);
}
It's easy to forget this, and I've solved this issue before in the past, but just now had to relearn the solution. Hopefully my answer here will be a good reminder for myself...

Here's this answer as an extension method:
using System.IO;
using System.Text;
namespace System.Web.Http
{
public static class ApiControllerExtensions
{
public static string GetRequestBody(this ApiController controller)
{
using (var stream = new MemoryStream())
{
var context = (HttpContextBase)controller.Request.Properties["MS_HttpContext"];
context.Request.InputStream.Seek(0, SeekOrigin.Begin);
context.Request.InputStream.CopyTo(stream);
var requestBody = Encoding.UTF8.GetString(stream.ToArray());
return requestBody;
}
}
}
}

ASP.NET Web API: PushStreamContent flush does not flush

I'm using PushStreamContent in ASP.NET Web API to push events from server to client (using Server-Sent Events). After each sent event, I call Flush on the Stream to push the buffered data to the client. However, I noticed that the flushing does not (always) happen. Sometimes, part of the data is sent to the client, and the rest is sent when the next event is written (which could happen seconds later).
Here's a code sample:
public class MyController : ApiController
{
private static readonly string[] LineSeparators
= new[] { Environment.NewLine };
public HttpResponseMessage GetData(string id)
{
var response = Request.CreateResponse();
response.Content = new PushStreamContent(
new Func<Stream, HttpContent, TransportContext, Task>(StartStream),
new MediaTypeHeaderValue("text/event-stream") { CharSet = "UTF-8" });
return response;
}
private async Task StartStream(Stream outputStream, HttpContent content, TransportContext context)
{
using (outputStream)
using (var writer = new StreamWriter(outputStream, new UTF8Encoding(false)))
{
writer.NewLine = "\n";
while (true)
{
WriteEvent(writer, "ping", DateTime.UtcNow.ToString("yyyy-MM-dd HH:mm:ss.fff", CultureInfo.InvariantCulture));
await Task.Delay(TimeSpan.FromSeconds(1));
}
}
}
private static void WriteEvent(TextWriter writer, string eventType, string data)
{
writer.WriteLine("event:" + eventType);
writer.WriteLine("data:" + data);
writer.WriteLine();
writer.Flush(); // StreamWriter.Flush calls Flush on underlying Stream
}
}
How can I disable the buffering of the data or force the flushing of the data?

I got it working.
In my case buffering was an isssue. I had to
1) disable gzip for my responses <urlCompression doStaticCompression="true" doDynamicCompression="false" />
2) Make sure that the proxy on Prod (Nginx) wasn't buffering either

After spending an entire day trying to figure out where the problem is, and going as far as to (desperately) giving out a bounty, I found out that the problem lied in the fact that I was using HttpSelfHostServer, and needed to configure TransferMode = TransferMode.Streamed on the HttpSelfHostConfiguration. That's all.

The source of the issue is the Stream being flushed.
At your code sample you warp the original stream with StreamWriter and then flush the StreamWriter .
You need to flush the original stream as well:
private async Task StartStream(Stream outputStream, HttpContent content, TransportContext context)
{
using (outputStream)
using (var writer = new StreamWriter(outputStream, new UTF8Encoding(false)))
{
writer.NewLine = "\n";
while (true)
{
WriteEvent(writer, "ping", DateTime.UtcNow.ToString("yyyy-MM-dd HH:mm:ss.fff", CultureInfo.InvariantCulture));
outputStream.Flush();
await Task.Delay(TimeSpan.FromSeconds(1));
}
}
}

Change WCF WebApi HttpContent response

Using the WCF Web API how would I go about changing a response's content body after the application logic has been run but before it's returned to the user. The goal is if suppressstatuscontent is true we:
Add a statuscode field to the content body
Change the statuscode on the response to 200
I have overridden a DelegatingChannel and in the SendAsnyc have some code that looks like this:
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
return base.SendAsync(request, cancellationToken).ContinueWith<HttpResponseMessage>(task =>
{
var response = task.Result;
if (CheckIfRequestHadSuppressStatusCode(request) == true)
{
string newResponse = (response.Content == null) ? "" : response.Content.ReadAsString();
newResponse = "<body>" +newResponse + "</body><statuscode>" + response.StatusCode + "</statuscode>";
response.StatusCode = HttpStatusCode.OK;
}
return response;
});
A major problem is this doesn't handle BOTH, xml and Json. I feel like there must be a much better way to go about the problem as this feels very hacky.

I'm not sure of the right approach but I would try something like this:
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
return base.SendAsync(request, cancellationToken)
.ContinueWith<HttpResponseMessage>(task =>
{
var response = task.Result;
if (CheckIfRequestHadSuppressStatusCode(request) == true)
{
switch(response.Content.Headers.ContentType.MediaType)
{
case "application/xml":
response.Content = new XmlWithStatusContent(response.Content)
break;
case "application/json":
response.Content = new JsonWithStatusContent(response.Content)
break;
}
response.StatusCode = HttpStatusCode.OK;
}
return response;
});
}
You can encapsulate the code that adds the extra status code markup in specialized versions of HttpContent (e.g. XmlWithStatusContent and JsonWithStatusContent).

You could parse the content as either XML or JSON (you can encapsulate that functionality in it's own class) which then gives you the ability to add the field independent of the format (independent in the sense that the SendAsync doesn't need to know).
var newResponse = ContentDecoder.Parse(response.Content).AddField("statuscode", response.StatusCode).ToString();
assuming Parse would return come kind of content object you can modify without having to know what the format is.
It's not a really a nice solution but it encapsulates the hackiness away a bit.
Update: Assuming you can dereive from HttpResponseMessage and are in full control of generating the response then you could have specialised subclasses dealing with it:
interface IHttpResponseContent
{
void AddField(string name, string value);
}
class XmlHttpResponseMessage : HttpResponseMessage, IHttpResponseContent
{
}
When generating the response you create Xml/JsonHttpResponseMessage objects instead. Then you can do:
var newResponse = response as IHttpResponseContent;
if (newResponse != null)
{
newResponse.AddField("statuscode", response.StatusCode);
}