I want to optimize response returning from ASP.NET Core application. In order to do that I'm serializing the object to JSON UTF-8 bytes and inserting it to SQL as varbinary(max). I initially though that I'll get a stream from the column value using ADO.NET and just return it from Controller. This worked perfectly fine at the beginning as I've simply returned the Stream from the controller and it worked:
public async Task<Stream> GetSingleObject()
{
Response.ContentType = "application/json; charset=UTF-8";
var streamWithUtf8JsonBytesFromDb = await _repository.GetSingleObject();
return streamWithUtf8JsonBytesFromDb;
}
Unfortunately I have to attach additional properties to the response object:
{
"metadata": null,
"data": (here I would like to copy bytes from SQL stream directly to the response without any transformation)
"links": [
"self": "some url"
]
}
How can I use System.Text.Json to achieve this result in performant and usable fashion?
I've came up with two approaches so far, but I think there has to be a better way:
First solution:
//using Microsoft.AspNetCore.Http.Extensions;
//using Microsoft.AspNetCore.Http;
public async Task GetSingleObject()
{
Response.ContentType = "application/json; charset=UTF-8";
await Response.WriteAsync("{\"meta\":null,\"data\":", System.Text.Encoding.UTF8)
var streamWithUtf8JsonBytesFromDb = await _repository.GetSingleObject();
await streamWithUtf8JsonBytesFromDb.CopyToAsync(Response.Body)
await Response.WriteAsync($",\"links\":[{{\"self\":{Request.GetEncodedUrl()}}}]}}", System.Text.Encoding.UTF8);
}
Constructing JSON parts from string and concatenating it in the stream is obviously painful
Second solution:
public class Meta
{ /* Not relevant */ }
public class ObjectResponse
{
public Meta? Meta { get; set; }
public JsonDocument Data { get; set; }
public Dictionary<string, string> Links { get; set; }
}
//using Microsoft.AspNetCore.Http.Extensions;
//using Microsoft.AspNetCore.Http;
public async Task<ObjectResponse> GetSingleObject()
{
var streamWithUtf8JsonBytesFromDb = await _repository.GetSingleObject();
return new ObjectResponse {
Meta = null,
Data = await JsonDocument.ParseAsync(streamWithUtf8JsonBytesFromDb),
Links = new Dictionary<string, string> {
{ "self", Request.GetEncodedUrl()}
};
}
}
This is better than the first approach, but it adds unnecessary parsing of the stream which I'd like to omit.
What I'd think is the best idea is to use the following:
public class Meta
{ /* Not relevant */ }
public class ObjectResponse
{
public Meta? Meta { get; set; }
public Stream Data { get; set; }
public Dictionary<string, string> Links { get; set; }
}
//using Microsoft.AspNetCore.Http.Extensions;
//using Microsoft.AspNetCore.Http;
public async Task<ObjectResponse> GetSingleObject()
{
return new ObjectResponse {
Meta = null,
Data = await _repository.GetSingleObject(),
Links = new Dictionary<string, string> {
{ "self", Request.GetEncodedUrl()}
};
}
}
But it doesn't work, because System.Text.Json tries to serialize this stream instead of copying it directly to the response body (which makes sense of course). Am I missing something? Is there a custom converter I could use to achieve the result I'd like?
Related
This question already has answers here:
System.Text.Json.JsonException: The JSON value could not be converted
(4 answers)
Closed 12 days ago.
I am working on a WPF MVVM application to retrieve some cryptocurrency information from this API. I am able to call the API and get an HTTP response, however, I am having trouble deserializing this response to an object. I understand that the symbol variable is passed but not used, however, I want the deserialization process to work and then I will format the URI accordingly to include the symbol and the API Key. Here is the code:
Crypto Object
public class Crypto
{
public string? Symbol { get; set; }
public string? Name { get; set; }
public double? Price { get; set; }
public double? ChangesPercentage { get; set; }
}
API Call Service Interface
public interface ICryptoService
{
Task<Crypto> GetCrypto(string symbol);
}
API Call Service
public async Task<Crypto> GetCrypto(string symbol)
{
using (HttpClient client = new HttpClient())
{
using var response = await client.GetAsync("https://financialmodelingprep.com/api/v3/quote/BTCUSD?apikey=KEY", HttpCompletionOption.ResponseHeadersRead);
response.EnsureSuccessStatusCode();
if (response.Content is object && response.Content.Headers.ContentType.MediaType == "application/json")
{
var responseStream = await response.Content.ReadAsStreamAsync();
try
{
return await System.Text.Json.JsonSerializer.DeserializeAsync<Crypto>(responseStream, new System.Text.Json.JsonSerializerOptions { IgnoreNullValues = true, PropertyNameCaseInsensitive = true });
}
catch (JsonException)
{
Console.WriteLine("Invalid JSON!");
}
}
else
{
Console.WriteLine("HTTP Response cannot be deserialised");
}
return null;
}
}
}
Main Method
CryptoService cryptoService = new CryptoService();
cryptoService.GetCrypto("BTCUSD").ContinueWith((task) =>
{
var crypto = task.Result;
});
I am attaching the JSON response that the link will provide below:
[
{
"symbol": "BTCUSD",
"name": "Bitcoin USD",
"price": 22887.08,
"changesPercentage": -0.1263,
"change": -28.9473,
"dayLow": 22887.08,
"dayHigh": 23351.51,
"yearHigh": 48086.836,
"yearLow": 15599.047,
"marketCap": 441375461059,
"priceAvg50": 19835.04,
"priceAvg200": 19730.518,
"volume": 27292504064,
"avgVolume": 23965132574,
"exchange": "CRYPTO",
"open": 23267.4,
"previousClose": 23267.4,
"eps": null,
"pe": null,
"earningsAnnouncement": null,
"sharesOutstanding": 19284918,
"timestamp": 1675872360
}
]
This is the exception that I get whenever I run the code:
Exception thrown: 'System.Text.Json.JsonException' in System.Private.CoreLib.dll
Your json is a json array, not a single json object, so you need to deserialize to a collection:
var result = await JsonSerializer.DeserializeAsync<List<Crypto>>(...);
P.S.
In general case it is recommended to just await async functions instead of using ContinueWith.
I am pretty new to C# and want to create a REST Get request with Basic auth.
I'm using RestSharp but i can't find a proper working full example.
This is my code:
using RestSharp;
using RestSharp.Authenticators;
namespace HttpClientAPP
{
class Program
{
static void Main(string[] args)
{
var client = new RestClient("https://data.myhost.de");
client.Authenticator = new HttpBasicAuthenticator("user", "xxxx");
var request = new RestRequest("resource",Method.Get);
client.ExecuteGetAsync(request).Wait();
Console.WriteLine("as");
}
}
}
This is thejson answer I expect and i want to parse it to use the data to store it in a MYSQL DB:
{"info": [
{
"method": "GET",
"url": "https://data.myhost.de/zfa-values/{zfaId}",
"description": "Get a ZFA value by id"
},
{
"method": "GET",
"url": "https://data.myhost.de/zfa-values",
"description": "Get a list of available ZFA values"
},
{
"method": "GET",
"url": "https://data.myhost.de/new-zfa-values",
"description": "Get a list of available ZFA values which have not been viewed yet"
},
{
"method": "GET",
"url": "https://data.myhost.de/",
"description": "Get a list of api endpoints"
}
]}
How do I parse the json response?
At https://restsharp.dev/v107/#recommended-usage there is some guidance (but at time of writing this answer I think it might have a typo)
Get docs says:
public class GitHubClient {
readonly RestClient _client;
public GitHubClient() {
_client = new RestClient("https://api.github.com/")
.AddDefaultHeader(KnownHeaders.Accept, "application/vnd.github.v3+json");
}
public Task<GitHubRepo[]> GetRepos()
=> _client.GetAsync<GitHubRepo[]>("users/aspnet/repos");
^^^^^^^^
//note: I think this should be GetJsonAsync
}
Post docs says:
var request = new MyRequest { Data = "foo" };
var response = await _client.PostAsync<MyRequest, MyResponse>(request, cancellationToken);
^^^^^^^^^ ^^^^^^^
//note: i think this should be PostJsonAsync and the first arg should be a string url path
You have classes that represent the json you send/get back, and RestSharp will de/serialize them for you. If you want some help making classes from JSON, take a look at services like http://app.QuickType.io - you paste your json and you get representative classes. The online generators are usually a bit more sophisticated in terms of what they interpret/the attributes they decorate with. Pasting your json into QT gives (you can choose a better name than SomeRoot):
namespace SomeNamespace
{
using System;
using System.Collections.Generic;
using System.Globalization;
using Newtonsoft.Json;
using Newtonsoft.Json.Converters;
public partial class SomeRoot
{
[JsonProperty("info")]
public Info[] Info { get; set; }
}
public partial class Info
{
[JsonProperty("method")]
public string Method { get; set; }
[JsonProperty("url")]
public string Url { get; set; }
[JsonProperty("description")]
public string Description { get; set; }
}
public partial class SomeRoot
{
public static SomeRoot FromJson(string json) => JsonConvert.DeserializeObject<SomeRoot>(json, SomeNamespace.Converter.Settings);
}
public static class Serialize
{
public static string ToJson(this SomeRoot self) => JsonConvert.SerializeObject(self, SomeNamespace.Converter.Settings);
}
internal static class Converter
{
public static readonly JsonSerializerSettings Settings = new JsonSerializerSettings
{
MetadataPropertyHandling = MetadataPropertyHandling.Ignore,
DateParseHandling = DateParseHandling.None,
Converters =
{
new IsoDateTimeConverter { DateTimeStyles = DateTimeStyles.AssumeUniversal }
},
};
}
}
And use like:
using RestSharp;
using RestSharp.Authenticators;
namespace HttpClientAPP
{
class Program
{
static async Task Main(string[] args)
{
var client = new RestClient("https://data.myhost.de");
client.Authenticator = new HttpBasicAuthenticator("user", "xxxx");
var res = await client.GetJsonAsync<SomeRoot>("");
Console.ReadLine(); //prevent exit
}
}
}
First, you can use newtosoft json nuget package:
Docs: https://www.newtonsoft.com/json
Example:
class ApiEndpoint{
public string Method {get;set;} // or you can use enum
public string Url {get;set;}
public string Description {get;set;}
}
// some logic
public void Fetch()
{
var client = new RestClient("https://data.myhost.de");
client.Authenticator = new HttpBasicAuthenticator("user", "xxxx");
var request = new RestRequest("resource",Method.Get);
var response = client.Get(request);
// read response as json
var json = JsonConvert.DeserializeObject<ICollection<ApiEndpoint>>(response);
// now json will have structure in your example
}
Other way is to add swagger gen for you server, get openapi.json and then generate sharp client using nswag
https://blog.logrocket.com/generate-typescript-csharp-clients-nswag-api/
my deserialize Dictionary's key results in "brand[0]" when I send in "brand" to the api.
I have a class like this:
public class SearchRequest
{
public bool Html { get; set; } = false;
public Dictionary<string, HashSet<string>> Tags { get; set; }
}
// MVC Controller
[HttpPost]
public ActionResult Index(SearchRequest searchRequest)
{
...
}
And a json request like this that I post to the controller:
{
"html": true,
"tags": {
"brand": [
"bareminerals"
]
}
}
The binding seams to work and the searchRequest object is created but the resulting dictionary dose not have the key "brand" in it but insted the key "brand[0]" how can I preserve the real values I send in?
Edit: I need tags to be able to contain multiple tags, with multiple options, this was a simpel example.
One soulution to my problem is to create a custom model bind, so this is what am using now, but I dont understand why I need to, and I feel like there should be a easyer way? But am gonna leve It here anyhow.
public class FromJsonBodyAttribute : CustomModelBinderAttribute
{
public override IModelBinder GetBinder()
{
return new JsonModelBinder();
}
private class JsonModelBinder : IModelBinder
{
public object BindModel(ControllerContext controllerContext, ModelBindingContext bindingContext)
{
var stream = controllerContext.HttpContext.Request.InputStream;
stream.Position = 0;
using (var reader = new StreamReader(stream))
{
var checkoutOrderDataStr = reader.ReadToEnd();
return JsonConvert.DeserializeObject(checkoutOrderDataStr, bindingContext.ModelType);
}
}
}
}
I'm not sure what is going on with your setup. You should not need a custom binder. I still think the problem is most likely with your calling code - whatever you're using as a client.
I'm using Asp.net Core 3.1. Here's what I threw together as a quick test.
Created Asp.net Core web application template with MVC. I declared two classes - a request POCO and a result POCO. The request was your class:
public class SearchRequest
{
public bool Html { get; set; } = false;
public Dictionary<string, HashSet<string>> Tags { get; set; }
}
The result was the same thing with a datetime field added just for the heck of it:
public class SearchResult : SearchRequest
{
public SearchResult(SearchRequest r)
{
this.Html = r.Html;
this.Tags = r.Tags;
}
public DateTime RequestedAt { get; set; } = DateTime.Now;
}
I Added a simple post method on the default HomeController.
[HttpPost]
public IActionResult Index([FromBody] SearchRequest searchRequest)
{
return new ObjectResult(new SearchResult(searchRequest));
}
I added a console Application to the solution to act as a client. I copied the two class definitions into that project.
I added this as the main method. Note you can either have the camel casing options on the request or not - asp.net accepted either.
static async Task Main(string[] _)
{
var tags = new[] { new { k = "brand", tags = new string[] { "bareminerals" } } }
.ToDictionary(x => x.k, v => new HashSet<string>(v.tags));
var request = new SearchRequest() { Html = true, Tags = tags };
var options = new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase };
var json = JsonSerializer.Serialize(request, options);
Console.WriteLine(json);
using (var client = new HttpClient())
{
var content = new StringContent(json, Encoding.UTF8, "application/json");
var response = await client.PostAsync("http://localhost:59276", content);
response.EnsureSuccessStatusCode();
var data = await response.Content.ReadAsStringAsync();
var result = JsonSerializer.Deserialize<SearchResult>(data, options);
Console.WriteLine(data);
var keysSame = Enumerable.SequenceEqual(request.Tags.Keys, result.Tags.Keys);
var valuesSame = Enumerable.SequenceEqual(request.Tags.Values.SelectMany(x => x),
result.Tags.Values.SelectMany(x=>x));
Console.WriteLine($"Keys: {keysSame} Values: {valuesSame}");
}
}
This outputs:
{"html":true,"tags":{"brand":["bareminerals"]}}
{"requestedAt":"2020-10-30T19:22:17.8525982-04:00","html":true,"tags":{"brand":["bareminerals"]}}
Keys: True Values: True
This is my first time calling an API and parsing through the JSON output. I know I called the API and retrieved the data since I printed it onto the console but I now want to print out a certain part of the JSON for example: all the titles of the list (if it had titles and some other information).
public async Task MainAsync()
{
using (var client = new HttpClient()) //creates a new client that will make the call to the api.
{
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new System.Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json")); // Accepts header for JSON format
try
{
HttpResponseMessage response = await client.GetAsync("http://quickstats.nass.usda.gov/api/api_GET/?key=MYKEYGOESHERE&commodity_desc=CORN&year__GE=2012&state_alpha=VA&format=JSON");
if (response.IsSuccessStatusCode)
{
string contents = await response.Content.ReadAsStringAsync();
Console.WriteLine(contents);
//List<quickStats> quickStat = JsonConvert.DeserializeObject<List<quickStats>>(contents);
quickStats quickStat = JsonConvert.DeserializeObject<quickStats>(contents);
Console.WriteLine("Done");
foreach(var item in quickStat)
{
Console.WriteLine(item.source_desc);
}
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
//Console.WriteLine("test");
}
}
}
}
class quickStats
{
public String source_desc { get; set; }
}
The JSON isn't in an array so I can't create a list like in the commented line then do a foreach statement for that list, following this vid: https://www.youtube.com/watch?v=xkB-cIMjXGQ
I am wondering how else I can print the parsed list of "source_desc".
Any help is appreciated!
It's structure like this
{"data":[
{"source_desc": "XXXX"},
{"source_desc": "OOOO"},
{"source_desc": "TTTT"}
]}
so your serialization class should be defined as:
class quickStats
{
public List<quickStatsDetail> data { get; set; }
}
class quickStatsDetail
{
public String source_desc { get; set; }
}
I'm just trying to create a simple POST function that let's me POST a JSON. I've tried to copy examples but I'm not sure what I'm doing differently. Any help would be appreciated, I feel like it's something simple that I'm missing.
What I'm trying to post:
POST Address: http://localhost:49653/save/file
Headers:
Content-Type: application/json
Raw Body:
{
uuid: "someUuid",
fileName: "test",
dateTime: "dateee",
json: "some json"
}
namespace SomeNamespace.Model
{
[Route("/save/file", "POST")]
public class SaveFileRequest
{
public Stream RequestStream { get; set; }
}
public class SaveFileResponse
{
public bool Success { get; set; }
}
}
namespace SomeNamespace.ServiceInterface
{
[EnableCors(allowedMethods:"POST")]
public class SaveFileService : Service
{
public object Any(SaveFileRequest request)
{
var response = new SaveFileResponse { Success = false };
string savedataJson;
using (var reader = new StreamReader(Request.InputStream))
{
savedataJson = reader.ReadToEnd();
}
try
{
Console.WriteLine(savedataJson); // When I debug, the contents are ""
}
catch(Exception ex) {...}
}
}
}
}
Your SaveFileRequest Request DTO needs to implement IRequiresRequestStream.
Here are the docs for reading directly from the request stream:
Reading directly from the Request Stream
Instead of registering a custom binder you can skip the serialization of the request DTO, you can add the IRequiresRequestStream interface to directly retrieve the stream without populating the request DTO.
//Request DTO
public class RawBytes : IRequiresRequestStream
{
/// <summary>
/// The raw Http Request Input Stream
/// </summary>
Stream RequestStream { get; set; }
}
Which tells ServiceStack to skip trying to deserialize the request so you can read in the raw HTTP Request body yourself, e.g:
public object Post(RawBytes request)
{
byte[] bytes = request.RequestStream.ReadFully();
string text = bytes.FromUtf8Bytes(); //if text was sent
}