Why is Refit serializing URL parameter boolean values using capital letters? - c#

I have an ASP .NET 6 Rest service in which I have a pretty basic Refit API interface defined for a GET operation with a request object that has several properties on it. Here's the request definition:
[Get("/users")]
Task<Auth0GetUsersResponse> GetUsers(Auth0GetUsersRequest request, [Header("Authorization")] string bearerToken);
The problem is that when Refit serializes the request object and constructs the URL parameters, it generates Boolean values with capital first letters, e.g. True or False. Unfortunately, URLs are case-sensitive and these values are not valid for Boolean inputs, so the API I am calling returns a 400. Here's the GET that is being generated:
GET /api/v2/users?include_totals=True
That value "True" is where I'm dying. Here's the property definition on the Auth0GetUsersRequest object:
public bool include_totals { get; set; }
And here's how I am registering the service in Startup.cs:
services.AddRefitClient<IAuth0ManagementApi>();
This is all really basic stuff. I'm not doing any extra configuring of any kind that I know of.
I've spent most of today trying to figure out ways to make that be "true" instead of "True", switching between System.Text.Json and Newtonsoft frameworks, changing property case convention in the client settings (which shouldn't have any impact but I was grasping at straws), even using a custom JsonConverter. Nothing affects it - Refit just insists on making bool values start with uppercase letters!
This only seems to be happening when serializing URL bool parameters in a GET. When serializing a POST body it works as expected, with properly lowercase values for bools.

I came up with a workaround for this issue, although I would still like to find out if it's possible to use the type I really want to use and have it generate the desired output. Until then though, this is what I did.
I created this Enum:
public class Auth0Enums
{
public enum UrlBoolean
{
[EnumMember(Value = "true")]
True,
[EnumMember(Value = "false")]
False,
[EnumMember(Value = null)]
Null
}
}
And am using it as a stand-in for the type bool. Here is the property definition on my request object now:
public Auth0Enums.UrlBoolean include_totals { get; set; }
Refit uses the EnumMember attribute to output the desired text:
GET /api/v2/users?include_totals=true

Related

How can I map JSON body properties as dynamic parameters of a .NET Core web API action method with validation?

I have to build a .NET Core REST API and I have about two dozen endpoints that take in simple JSON objects like:-
{
"foo": 23,
"bar": "bar_value"
}
and
{
"foo": 12,
"baz": true
}
etc.
Some properties, such as foo above, are common among several endpoints but have different validation requirements. In some endpoints they are required, in others, they are not and so on. I can't change these JSON payloads as they're generated by a third party I don't have any control over.
How can I map these parameters to endpoints in a .NET Core API method directly, without a class?
I can, of course, create a class for each endpoint, such as
public class SomeObject
{
[Required]
[Range(0, 100)]
public int? Foo { get; set; }
public string bar { get; set; }
}
public class SomeOtherObject
{
public int? Foo { get; set; }
[Required]
public bool Baz { get; set; }
}
...
Note the different validation rules.
But I don't feel like creating some two dozen classes. I'd much rather just specify them directly in the endpoint method:
[HttpPut]
[Route("/some-route")]
public IActionResult SomeAction([Required, Range(0, 100)] int? foo, byte? bar)
{
...
}
[HttpPut]
[Route("/some-other-route")]
public IActionResult SomeOtherAction(int? foo, [Required] baz)
{
...
}
It would be much easier to read and figure out which property is required and when by just looking at the methods instead of opening one of two dozen similarly named class files or opening one single file with two dozen similarly named classes with properties of the same name.
So how can I get .NET Core to parse the JSON and assign the property values to the action method parameters?
I'm not aware of a direct answer to this question as specified, so I'll answer this with an alternative approach as an XY problem based on your statement "It would be much easier to read and figure out which property is required and when by just looking at the methods".
This assumes there's not an easy way document your own API surface area if you're using classes. In your example, you're already writing a large amount of logic in the method signature itself, not to mention potential behaviors for default values, etc., that can make those signatures progressively harder to read and understand, and that's exactly what input model classes and model validation are designed to help encapsulate. Furthermore, now that you've decomposed the model into its parts, it becomes increasingly complex to handle validation issues as a cohesive model, regardless of whether it could be done. By accepting the entire object at once, you can run a ModelState.IsValid check, aggregate errors, or add your own and quickly return that from the controller.
By adding XML documentation to your endpoint methods and input model classes, you also open up the easy path of adding a Swagger page with Swashbuckle, which will provide a simple way for you to inspect what the model value types are and which ones are required, etc., as well as example JSON bodies in the Swagger page itself with full documentation as to the purpose of all the parameters.
While you do end up with a bunch of model classes, it's just a button press away from Visual Studio to hop to your class and see your validation requirements and input types while "in code". If class generation is frustrating, you can quickly drop your JSON samples into a class generator online and get a "pretty good" starting point for the input models: https://json2csharp.com/

REST in ASP.NET Core: validate a missing value type field on a PUT object

For example you have a class that you intend to expose via your public RESTful API:
public class MyExposedType
{
[Required]
public bool MyExposedValueType { get; set; }
}
Consider a PUT operation with the body {}. In this situation I would expect the API to return a BadRequest, but instead the object is accepted.
Since MyExposedValueType is a value type, it cannot be null and defaults to false. However, typically for a PUT operation, you must provide values for all the fields in the object to replace.
I know I could expose a DTO with a [Required] nullable bool on it. Is there a better way to handle this situation?
The problem here is that the exported model is protocol-agnostic. In some cases it can be explicitly stated, that the default values can be omitted in the request. For the bool the default value is false, so it may be even expected that in case of false there is nothing in the request.
To specifically require the value in the request, you need to either change the default value to null (by changing the property to nullable), or to adjust the model specifically for some protocol. For example, newtonsoft.json has a JsonRequiredAttribute (https://www.newtonsoft.com/json/help/html/T_Newtonsoft_Json_JsonRequiredAttribute.htm). But it will only work in case of JSON deserialization, and only if you use newtonsoft.json.
Making the type nullable as you suggest should be fine? It's only a DTO after all - not a domain object which is more likely to enforce 'proper' types, if appropriate.

FromQueryAttribute enum parameters throw validation_error invalid value, but parameters with FromBodyAttribute do not?

In ASP.NET Core, if an action accepts directly an enum type, and that enum for example has defined 1 to something, if we pass a value different than 1, we'll get a validation error.
This is good! But it doesn't work for when the enum is inside a complex object type, when they are built from the body of a request ([FromBody] attribute).
Why is this happening? I know that anything coming from the body, is being handled by JSON Converters. Why can't they handle this for us, when the Query binder (what/where is it?) does it for us?
Example Enum:
public Enum Example
{
One = 1
}
Example Action:
public object ExampleAction(Enum hello)
{
return Ok();
}
If you hit the action with a HTTP request, passing in the hello parameter in a query string with a value different than 1, you will get a validation error.
Now, if you annotate the hello parameter with the [FromBody] attribute and make a new request (passing in this time the data via the body instead of the query string), that behavior is lost.

Getting error unexpected character encountered while parsing value: . Path '', line 1, position 1

I have an ASP.Net Core 2.1 app. I am getting the same error as mentioned in the below SO link.
Unexpected character encountered while parsing value: . Path '', line 1, position 1
The solution suggested in the answer section seems to solve the issue as well.
But creating a class for one property is overhead & doesn't appeal to me.
In the past, I did the same without creating such overhead class with .Net Framework 4.5(+)
// POST api/values
[HttpPost]
public void Post([FromBody]string email)
{
}
Through Postman, I POST
{
"email": "myemail#email.com"
}
How can I pass the email value to API without creating any class or
such?
I would prefer to stick [FromBody], as all the APIs in my application is accepting the params FromBody in Post method.
If you want to pass the email value without creating class, just pass
"myemail#email.com"
without the {} brackets and the email key. When you pass
{
"email": "myemail#email.com"
}
the parser correctly suggests that this value represents object with property email, that is why wrapper class is needed.
If you know you're only ever going to need one argument, then go ahead and make it a string, as others have suggested. But please remember that you will not be able to receive two arguments from the body in ASP.NET Core: you will have to define a type that contains two (or more) properties, and the single object in the request body will be mapped to that type. So, if you're not sure, just save yourself the future refactoring and define a type for your request.
Don't be afraid of creating one more class in your project. It is a very small overhead for the machine, and a minuscule annoyance for you to spend three minutes writing it, but it improves the quality of your code, makes it more expressive, and it can even help you by making the compiler find mistakes you may not have noticed. Just give it a meaningful name, and you'll be fine.
You can simply put in a string without the key, the api will still accept it:
"myemail#email.com"
You might need to call JSON.stringify(email) to pass the email to the api, and also the content type as a header. This is how I solved the issue:
post(id: number, email: string): Observable<any> {
const httpOptions = {
headers: new HttpHeaders({'Content-Type': 'application/json'})
};
const url = this.baseUrl + `api/Portfolio/InviteUser/${id}`;
return this.http.post<any>(url, JSON.stringify(email), httpOptions);
}
My Web Api also contained an Id in the URI:
[HttpPost("[action]/{id}")]
public async Task<ActionResult<Response>> InviteUser(int id, [FromBody] string email)

web-api POST body object always null

I'm still learning web API, so pardon me if my question sounds stupid.
I have this in my StudentController:
public HttpResponseMessage PostStudent([FromBody]Models.Student student)
{
if (DBManager.createStudent(student) != null)
return Request.CreateResponse(HttpStatusCode.Created, student);
else
return Request.CreateResponse(HttpStatusCode.BadRequest, student);
}
In order to test if this is working, I'm using Google Chrome's extension "Postman" to construct the HTTP POST request to test it out.
This is my raw POST request:
POST /api/Student HTTP/1.1
Host: localhost:1118
Content-Type: application/json
Cache-Control: no-cache
{"student": [{"name":"John Doe", "age":18, "country":"United States of America"}]}
student is supposed to be an object, but when I debug the application, the API receives the student object but the content is always null.
FromBody is a strange attribute in that the input POST values need to be in a specific format for the parameter to be non-null, when it is not a primitive type. (student here)
Try your request with {"name":"John Doe", "age":18, "country":"United States of America"} as the json.
Remove the [FromBody] attribute and try the solution. It should work for non-primitive types. (student)
With the [FromBody] attribute, the other option is to send the values in =Value format, rather than key=value format. This would mean your key value of student should be an empty string...
There are also other options to write a custom model binder for the student class and attribute the parameter with your custom binder.
I was looking for a solution to my problem for some minutes now, so I'll share my solution.
When you have a custom constructor within your model, your model also needs to have an empty/default constructor. Otherwise the model can't be created, obviously.
Be careful while refactoring.
I spend several hours with this issue... :( Getters and setters are REQUIRED in POST parameters object declaration. I do not recommend using simple data objects (string,int, ...) as they require special request format.
[HttpPost]
public HttpResponseMessage PostProcedure(EdiconLogFilter filter){
...
}
Does not work when:
public class EdiconLogFilter
{
public string fClientName;
public string fUserName;
public string fMinutes;
public string fLogDate;
}
Works fine when:
public class EdiconLogFilter
{
public string fClientName { get; set; }
public string fUserName { get; set; }
public string fMinutes { get; set; }
public string fLogDate { get; set; }
}
If the any of values of the request's JSON object are not the same type as expected by the service then the [FromBody] argument will be null.
For example, if the age property in the json had a float value:
"age":18.0
but the API service expects it to be an int
"age":18
then student will be null. (No error messages will be sent in the response unless no null reference check).
This is a little old one and my answer will go down to the last place but even so I would like to share my experience.
Tried every suggestion but still having the same "null" value in a PUT [FromBody].
Finally found it was all about Date format while JSON serializing the EndDate property of my Angular Object.
No error was thrown, just received an empty FromBody object....
If using Postman, make sure that:
You have set a "Content-Type" header to "application/json"
You are sending the body as "raw"
You don't need to specify the parameter name anywhere if you are using [FromBody]
I was stupidly trying to send my JSON as form data, duh...
UPDATE: A practical solution is writing a custom JSON formatter. For a general description of the problem (but no practical solution), read on here.
TL;DR: Don't use [FromBody], but roll your own version of it with better error handling. Reasons given below.
Other answers describe many possible causes of this problem. However, the root cause is that [FromBody] simply has terrible error handling, which makes it almost useless in production code.
For example, one of the most typical reasons for the parameter to be null is that the request body has invalid syntax (e.g., invalid JSON). In this case, a reasonable API would return 400 BAD REQUEST, and a reasonable web framework would do this automatically. However, ASP.NET Web API is not reasonable in this regard. It simply sets the parameter to null, and the request handler then needs "manual" code to check if the parameter is null.
Many of the answers given here are therefore incomplete with regards to error handling, and a buggy or malicious client may cause unexpected behavior on the server side by sending an invalid request, which will (in the best case) throw a NullReferenceException somewhere and return an incorrect status of 500 INTERNAL SERVER ERROR or, worse, do something unexpected or crash or expose a security vulnerability.
A proper solution would be to write a custom "[FromBody]" attribute which does proper error handling and returns proper status codes, ideally with some diagnostic information to aid client developers.
A solution that might help (not tested yet) is to make parameters required, as follows: https://stackoverflow.com/a/19322688/2279059
The following clumsy solution also works:
// BAD EXAMPLE, but may work reasonably well for "internal" APIs...
public HttpResponseMessage MyAction([FromBody] JObject json)
{
// Check if JSON from request body has been parsed correctly
if (json == null) {
var response = new HttpResponseMessage(HttpStatusCode.BadRequest) {
ReasonPhrase = "Invalid JSON"
};
throw new HttpResponseException(response);
}
MyParameterModel param;
try {
param = json.ToObject<MyParameterModel>();
}
catch (JsonException e) {
var response = new HttpResponseMessage(HttpStatusCode.BadRequest) {
ReasonPhrase = String.Format("Invalid parameter: {0}", e.Message)
};
throw new HttpResponseException(response);
}
// ... Request handling goes here ...
}
This does (hopefully) proper error handling, but is less declarative. If, for example, you use Swagger to document your API, it will not know the parameter type, which means you need to find some manual workaround to document your parameters. This is just to illustrate what [FromBody] should be doing.
EDIT: A less clumsy solution is to check ModelState: https://stackoverflow.com/a/38515689/2279059
EDIT: It appears that ModelState.IsValid is not, as one would expect, set to false if using JsonProperty with Required = Required.Always and a parameter is missing. So this is also useless.
However, in my opinion, any solution that requires writing additional code in every request handler is unacceptable. In a language like .NET, with powerful serialization capabilities, and in a framework like ASP.NET Web API, request validation should be automatic and built-in, and it is totally doable, even though Microsoft does not provide the necessary built-in tools.
It can be helpful to add TRACING to the json serializer so you can see what's up when things go wrong.
Define an ITraceWriter implementation to show their debug output like:
class TraceWriter : Newtonsoft.Json.Serialization.ITraceWriter
{
public TraceLevel LevelFilter {
get {
return TraceLevel.Error;
}
}
public void Trace(TraceLevel level, string message, Exception ex)
{
Console.WriteLine("JSON {0} {1}: {2}", level, message, ex);
}
}
Then in your WebApiConfig do:
config.Formatters.JsonFormatter.SerializerSettings.TraceWriter = new TraceWriter();
(maybe wrap it in an #if DEBUG)
I was also trying to use the [FromBody], however, I was trying to populate a string variable because the input will be changing and I just need to pass it along to a backend service but this was always null
Post([FromBody]string Input])
So I changed the method signature to use a dynamic class and then convert that to string
Post(dynamic DynamicClass)
{
string Input = JsonConvert.SerializeObject(DynamicClass);
This works well.
After Three days of searching and none of above solutions worked for me , I found another approach to this problem in this Link:
HttpRequestMessage
I used one of the solutions in this site
[HttpPost]
public async System.Threading.Tasks.Task<string> Post(HttpRequestMessage request)
{
string body = await request.Content.ReadAsStringAsync();
return body;
}
Just to add my history to this thread.
My model:
public class UserProfileResource
{
public Guid Id { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
public string Phone { get; set; }
public UserProfileResource()
{
}
}
The above object couldn't be serialized in my API Controller and would always return null. The issue was with Id of type Guid: everytime I passed empty string as an Id (being naive that it will automatically be converted to Guid.Empty) from my frontend I received null object as [FromBody] paramether.
Solution was either to
pass valid Guid value
or change Guid to String
In my case the problem was the DateTime object I was sending. I created a DateTime with "yyyy-MM-dd", and the DateTime that was required by the object I was mapping to needed "HH-mm-ss" aswell. So appending "00-00" solved the problem (the full item was null because of this).
I've hit this problem so many times, but actually, it's quite straightforward to track down the cause.
Here's today's example. I was calling my POST service with an AccountRequest object, but when I put a breakpoint at the start of this function, the parameter value was always null. But why ?!
[ProducesResponseType(typeof(DocumentInfo[]), 201)]
[HttpPost]
public async Task<IActionResult> Post([FromBody] AccountRequest accountRequest)
{
// At this point... accountRequest is null... but why ?!
// ... other code ...
}
To identify the problem, change the parameter type to string, add a line to get JSON.Net to deserialize the object into the type you were expecting, and put a breakpoint on this line:
[ProducesResponseType(typeof(DocumentInfo[]), 201)]
[HttpPost]
public async Task<IActionResult> Post([FromBody] string ar)
{
// Put a breakpoint on the following line... what is the value of "ar" ?
AccountRequest accountRequest = JsonConvert.DeserializeObject<AccountRequest>(ar);
// ... other code ...
}
Now, when you try this, if the parameter is still blank or null, then you simply aren't calling the service properly.
However, if the string does contain a value, then the DeserializeObject should point you towards the cause of the problem, and should also fail to convert your string into your desired format. But with the raw (string) data which it's trying to deserialize, you should now be able to see what's wrong with your parameter value.
(In my case, we were calling the service with an AccountRequest object which had been accidentally serialized twice !)
This is another issue related to invalid property values in an Angular Typescript request.
This is was related to the conversion between a Typescript number to an int(Int32) in C#. I was using Ticks (UTC milliseconds) which is larger than the signed, Int32 range (int in C#). Changed the C# model from int to long and everything worked fine.
I had the same problem.
In my case, the problem was in public int? CreditLimitBasedOn { get; set; } property I had.
my JSON had the value "CreditLimitBasedOn":true when It should contain an integer. This property prevented the whole object being deserialized on my api method.
Maybe for someone it will be helpful: check the access modifiers for your DTO/Model class' properties, they should be public. In my case during refactoring domain object internals were moved to DTO like this:
// Domain object
public class MyDomainObject {
public string Name { get; internal set; }
public string Info { get; internal set; }
}
// DTO
public class MyDomainObjectDto {
public Name { get; internal set; } // <-- The problem is in setter access modifier (and carelessly performed refactoring).
public string Info { get; internal set; }
}
DTO is being finely passed to client, but when the time comes to pass the object back to the server it had only empty fields (null/default value). Removing "internal" puts things in order, allowing deserialization mechanizm to write object's properties.
public class MyDomainObjectDto {
public Name { get; set; }
public string Info { get; set; }
}
Check if JsonProperty attribute is set on the fields that come as null - it could be that they are mapped to different json property-names.
If this is because Web API 2 ran into a deserialization problem due to mismatched data types, it's possible to find out where it failed by inspecting the content stream. It will read up until it hits an error, so if you read the content as a string, you should have the back half of the data you posted:
string json = await Request.Content.ReadAsStringAsync();
Fix that parameter, and it should make it further next time (or succeed if you're lucky!)...
I used HttpRequestMessage and my problem got solved after doing so much research
[HttpPost]
public HttpResponseMessage PostProcedure(HttpRequestMessage req){
...
}
In my case, using postman I was sending a DateTime with invalid separators (%) so the parse failed silently.
Be sure you are passing valid params to your class constructor.
None of the above was my solution: in my case the issue is that [ApiController] was not added to the controller so it is giving Null value
[Produces("application/json")]
[Route("api/[controller]")]
[ApiController] // This was my problem, make sure that it is there!
public class OrderController : Controller
...
I just ran into this and was frustrating.
My setup:
The header was set to
Content-Type: application/JSON
and was passing the info from the body with JSON format, and was reading [FromBody] on the controller.
Everything was set up fine and I expect it to work, but the problem was with the JSON sent over. Since it was a complex structure, one of my classes which was defined 'Abstract' was not getting initialized and hence the values weren't assigned to the model properly. I removed the abstract keyword and it just worked..!!!
One tip, the way I could figure this out was to send data in parts to my controller and check when it becomes null... since it was a complex model I was appending one model at a time to my request params. Hope it helps someone who runs into this stupid issue.
Seems like there can be many different causes of this problem...
I found that adding an OnDeserialized callback to the model class caused the parameter to always be null. Exact reason unknown.
using System.Runtime.Serialization;
// Validate request
[OnDeserialized] // TODO: Causes parameter to be null
public void DoAdditionalValidatation() {...}
I had this problem in my .NET Framework Web API, because my model existed in a .NET Standard project that referenced a different version of data annotations.
Adding the ReadAsAsync line below highlighted the cause for me:
public async Task<HttpResponseMessage> Register(RegistrationDetails registrationDetails)
{
var regDetails = await Request.Content.ReadAsAsync<RegistrationDetails>();
In my case (.NET Core 3.0) I had to configure JSON serialization to resolve camelCase properties using AddNewtonsoftJson():
services.AddMvc(options =>
{
// (Irrelevant for the answer)
})
.AddNewtonsoftJson(options =>
{
options.SerializerSettings.ContractResolver = new CamelCasePropertyNamesContractResolver();
});
Do this in your Startup / Dependency Injection setup.
I was struggling with this for hours today. I could see that there was data in the response body because of the length, but any time I tried to read the data, I got an empty string, or the arguments in the parameter list for the method returned null. I compared this controller to another that was already working and found that I was missing the ApiController attribute for the class declaration. I also removed the FromBody attribute from my parameter declaration. I am not sure when that was added, but I am using .Net 5.0.
As detailed in my other answer, the problem is with error handling in the [FromBody] attribute, and you cannot do much about that without writing your own version of it.
However, a general solution that will improve error handling in your entire API without making changes in any controller or action is to write a custom JSON formatter (derived from FotoWareApiJsonFormatter) which handles serialization errors properly.
I will not present the entire solution here, but the important part is to catch JsonSerializationException and JsonReaderException in the formatter and make sure the endpoint will return 400 Bad Request as a result.
This ensures that if the request contains invalid JSON, or the JSON does not fulfill model constraints (such as missing required properties, type errors, etc.), the API will automatically return 400 Bad Request before your controller action is called, so you do not need to write extra error handling in the controller, and your parameter using [FromBody] will never be null.
// in JSON formatter class
private object Deserialize(Stream readStream, Type type)
{
try
{
var streamReader = new StreamReader(readStream);
return GetSerializer().Deserialize(streamReader, type);
}
catch (JsonSerializationException e)
{
// throw another exception which results in returning 400 Bad Request
}
catch (JsonReaderException e)
{
// throw another exception which results in returning 400 Bad Request
}
}
You also have to ensure that your custom JSON formatter is the only formatter, e.g., by adding this code to Application_Start():
var jsonFormatter = new MyJsonFormatter();
// Always return JSON, and accept JSON only
GlobalConfiguration.Configuration.Formatters.Clear();
GlobalConfiguration.Configuration.Formatters.Add(jsonFormatter);
This assumes that your API accepts and returns JSON only, as most modern APIs do. Offering XML or other formats as an alternative if you are not going to test or advertise it is unnecessary at best and a potential security risk at worst.
Be careful when introducing this to an existing API, as it can introduce some unexpected breaking changes, so good testing is advised. Consider it as a cleanup of the error handling in your API.
Just one more thing to look at... my model was marked as [Serializable] and that was causing the failure.

Categories