I am calling an API like so:
using (HttpClient client = new HttpClient())
{
client.DefaultRequestHeaders.UserAgent.Add(new ProductInfoHeaderValue(".NET", "5.0"));
var result = client.GetAsync("https://the.api.com").Result;
var data = result.Content.ReadAsStringAsync().Result;
}
This returns a bunch of results like so:
[
[
1647993600,
2963,
2972.18,
2970.51,
2967.14,
831.59875656
],
[
1647993300,
2967.91,
2972.14,
2970.95,
2970.51,
588.06561288
],
[
1647993000,
2969.91,
2977.03,
2976.83,
2970.84,
444.49625293
]
]
I was wondering if HttpClient in .NET5 offers simple way to convert these results into a List<MyResponseObject>?
Suppose you have an API exposing a GET endpoint at /api/students.
This endpoint returns the following JSON response:
[
{
"name": "Bob",
"age": 20
},
{
"name": "Alice",
"age": 34
}
]
In order to call this endpoint you can define the following POCO class, to deserialize every single array item:
public sealed class Student
{
public string Name { get; set; }
public int Age { get; set; }
}
The simplest approach to call the endpoint and deserialize the HTTP response to a .NET object is the following:
issue the GET request and wait for the request to complete
read the response content as a string, containing the JSON returned by the endpoint
deserialize the JSON into a .NET object by using a library of your choice
In the following example I use the built-in JSON support of .NET core in order to deserialize the JSON string into a .NET object (another common choice is using the Newtonsft JSON library). The .NET core built-in support for JSON serialization and deserialization is inside the System.Text.Json namespace. The documentation is here.
var uri = new Uri("https://my-service.example.com/api/students");
var response = await httpClient.GetAsync(uri).ConfigureAwait(false);
response.EnsureSuccessStatusCode();
var json = response.Content.ReadAsStringAsync().ConfigureAwait(false);
var serializerOptions = new JsonSerializerOptions
{
PropertyNameCaseInsensitive = true
};
var students = JsonSerializer.Deserialize<List<Student>>(json, serializerOptions);
Some important notes on the previous code:
do not manually create and dispose instances of HttpClient. Use, insted, the IHttpClientFactory to do that. You can read more about this topic here and here. In the second article you'll also find an explanation on why you should never manually create and dispose instances of HttpClient.
when you do I/O operations (such as issuing HTTP requests) and the class you are using have native support for async code, never ever block on asynchronous method calls. Instead of using blocking method such as Task.Result or Task.Wait(), make your own code asynchronous (by using the async keyword) and perform asynchronous wait on tasks (by using the await keyword). You can learn more about asynchronous programming in .NET with async and await here. This article explains why you should never block when calling asynchronous methods.
The approach showed above to deserialize the JSON returned by a web API is not very efficient. That's the simplest possible code and it's ok if the JSON response content is not too big and you are not working in a performance critical application.
The main issue with this code is that you are buffering in memory the entire response content as a string before deserializing it to a .NET object.
By doign so you are wasting memory, allocating unnecessary objects and delaying the moment when you actually start working on the response content (you can start working on the response content only when the entire response has been received from the network and its entire content has been buffered in memory as a string).
This article explains this problem in depth.
A better approach is the following:
var uri = new Uri("https://my-service.example.com/api/students");
/*
* The task returned by GetAsync completes as soon as the HTTP response header have been read.
* Remember to Dispose the HttpResponseMessage object.
*/
using var response = await httpClient.GetAsync(uri, HttpCompletionOption.ResponseHeadersRead).ConfigureAwait(false);
response.EnsureSuccessStatusCode();
/*
* Read the response content directly from the network socket.
* By doing so data is processed as it is being received from the network: this is more efficient because we avoid buffering and we start processing
* data as soon as they arrive.
*/
var stream = await response.Content.ReadAsStreamAsync();
var serializerOptions = new JsonSerializerOptions
{
PropertyNameCaseInsensitive = true
};
var students = await JsonSerializer.DeserializeAsync<List<Student>>(stream, serializerOptions);
I'm currently writing a very simple library which helps working with JSON apis from .NET core applications. It already encapsulates these concepts and add some more exception handling and corner case handling. You can find the source code here.
As a sidenote, you mentioned that you are expecting to receive and array of array inside the response content. My suggestion in this case is using the code showed above to deserialize the response inside a List<List<double>>. After that, you can decide to flatten the sequence by using the SelectMany extension method:
var rawResponseContent = await JsonSerializer.DeserializeAsync<List<List<double>>>(stream, serializerOptions);
List<double> numbers = rawResponseContent.SelectMany(x => x).ToList();
In an application I am developing RESTful API and we want the client to send data as JSON. Part of this application requires the client to upload a file (usually an image) as well as information about the image.
I'm having a hard time tracking down how this happens in a single request. Is it possible to Base64 the file data into a JSON string? Am I going to need to perform 2 posts to the server? Should I not be using JSON for this?
As a side note, we're using Grails on the backend and these services are accessed by native mobile clients (iPhone, Android, etc), if any of that makes a difference.
I asked a similar question here:
How do I upload a file with metadata using a REST web service?
You basically have three choices:
Base64 encode the file, at the expense of increasing the data size by around 33%, and add processing overhead in both the server and the client for encoding/decoding.
Send the file first in a multipart/form-data POST, and return an ID to the client. The client then sends the metadata with the ID, and the server re-associates the file and the metadata.
Send the metadata first, and return an ID to the client. The client then sends the file with the ID, and the server re-associates the file and the metadata.
You can send the file and data over in one request using the multipart/form-data content type:
In many applications, it is possible for a user to be presented with
a form. The user will fill out the form, including information that
is typed, generated by user input, or included from files that the
user has selected. When the form is filled out, the data from the
form is sent from the user to the receiving application.
The definition of MultiPart/Form-Data is derived from one of those
applications...
From http://www.faqs.org/rfcs/rfc2388.html:
"multipart/form-data" contains a series of parts. Each part is
expected to contain a content-disposition header [RFC 2183] where the
disposition type is "form-data", and where the disposition contains
an (additional) parameter of "name", where the value of that
parameter is the original field name in the form. For example, a part
might contain a header:
Content-Disposition: form-data; name="user"
with the value corresponding to the entry of the "user" field.
You can include file information or field information within each section between boundaries. I've successfully implemented a RESTful service that required the user to submit both data and a form, and multipart/form-data worked perfectly. The service was built using Java/Spring, and the client was using C#, so unfortunately I don't have any Grails examples to give you concerning how to set up the service. You don't need to use JSON in this case since each "form-data" section provides you a place to specify the name of the parameter and its value.
The good thing about using multipart/form-data is that you're using HTTP-defined headers, so you're sticking with the REST philosophy of using existing HTTP tools to create your service.
I know that this thread is quite old, however, I am missing here one option. If you have metadata (in any format) that you want to send along with the data to upload, you can make a single multipart/related request.
The Multipart/Related media type is intended for compound objects consisting of several inter-related body parts.
You can check RFC 2387 specification for more in-depth details.
Basically each part of such a request can have content with different type and all parts are somehow related (e.g. an image and it metadata). The parts are identified by a boundary string, and the final boundary string is followed by two hyphens.
Example:
POST /upload HTTP/1.1
Host: www.hostname.com
Content-Type: multipart/related; boundary=xyz
Content-Length: [actual-content-length]
--xyz
Content-Type: application/json; charset=UTF-8
{
"name": "Sample image",
"desc": "...",
...
}
--xyz
Content-Type: image/jpeg
[image data]
[image data]
[image data]
...
--foo_bar_baz--
Here is my approach API (i use example) - as you can see, you I don't use any file_id (uploaded file identifier to the server) in API:
Create photo object on server:
POST: /projects/{project_id}/photos
body: { name: "some_schema.jpg", comment: "blah"}
response: photo_id
Upload file (note that file is in singular form because it is only one per photo):
POST: /projects/{project_id}/photos/{photo_id}/file
body: file to upload
response: -
And then for instance:
Read photos list
GET: /projects/{project_id}/photos
response: [ photo, photo, photo, ... ] (array of objects)
Read some photo details
GET: /projects/{project_id}/photos/{photo_id}
response: { id: 666, name: 'some_schema.jpg', comment:'blah'} (photo object)
Read photo file
GET: /projects/{project_id}/photos/{photo_id}/file
response: file content
So the conclusion is that, first you create an object (photo) by POST, and then you send second request with the file (again POST). To not have problems with CACHE in this approach we assume that we can only delete old photos and add new - no update binary photo files (because new binary file is in fact... NEW photo). However if you need to be able to update binary files and cache them, then in point 4 return also fileId and change 5 to GET: /projects/{project_id}/photos/{photo_id}/files/{fileId}.
I know this question is old, but in the last days I had searched whole web to solution this same question. I have grails REST webservices and iPhone Client that send pictures, title and description.
I don't know if my approach is the best, but is so easy and simple.
I take a picture using the UIImagePickerController and send to server the NSData using the header tags of request to send the picture's data.
NSMutableURLRequest *request = [[NSMutableURLRequest alloc] initWithURL:[NSURL URLWithString:#"myServerAddress"]];
[request setHTTPMethod:#"POST"];
[request setHTTPBody:UIImageJPEGRepresentation(picture, 0.5)];
[request setValue:#"image/jpeg" forHTTPHeaderField:#"Content-Type"];
[request setValue:#"myPhotoTitle" forHTTPHeaderField:#"Photo-Title"];
[request setValue:#"myPhotoDescription" forHTTPHeaderField:#"Photo-Description"];
NSURLResponse *response;
NSError *error;
[NSURLConnection sendSynchronousRequest:request returningResponse:&response error:&error];
At the server side, I receive the photo using the code:
InputStream is = request.inputStream
def receivedPhotoFile = (IOUtils.toByteArray(is))
def photo = new Photo()
photo.photoFile = receivedPhotoFile //photoFile is a transient attribute
photo.title = request.getHeader("Photo-Title")
photo.description = request.getHeader("Photo-Description")
photo.imageURL = "temp"
if (photo.save()) {
File saveLocation = grailsAttributes.getApplicationContext().getResource(File.separator + "images").getFile()
saveLocation.mkdirs()
File tempFile = File.createTempFile("photo", ".jpg", saveLocation)
photo.imageURL = saveLocation.getName() + "/" + tempFile.getName()
tempFile.append(photo.photoFile);
} else {
println("Error")
}
I don't know if I have problems in future, but now is working fine in production environment.
FormData Objects: Upload Files Using Ajax
XMLHttpRequest Level 2 adds support for the new FormData interface.
FormData objects provide a way to easily construct a set of key/value pairs representing form fields and their values, which can then be easily sent using the XMLHttpRequest send() method.
function AjaxFileUpload() {
var file = document.getElementById("files");
//var file = fileInput;
var fd = new FormData();
fd.append("imageFileData", file);
var xhr = new XMLHttpRequest();
xhr.open("POST", '/ws/fileUpload.do');
xhr.onreadystatechange = function () {
if (xhr.readyState == 4) {
alert('success');
}
else if (uploadResult == 'success')
alert('error');
};
xhr.send(fd);
}
https://developer.mozilla.org/en-US/docs/Web/API/FormData
Since the only missing example is the ANDROID example, I'll add it.
This technique uses a custom AsyncTask that should be declared inside your Activity class.
private class UploadFile extends AsyncTask<Void, Integer, String> {
#Override
protected void onPreExecute() {
// set a status bar or show a dialog to the user here
super.onPreExecute();
}
#Override
protected void onProgressUpdate(Integer... progress) {
// progress[0] is the current status (e.g. 10%)
// here you can update the user interface with the current status
}
#Override
protected String doInBackground(Void... params) {
return uploadFile();
}
private String uploadFile() {
String responseString = null;
HttpClient httpClient = new DefaultHttpClient();
HttpPost httpPost = new HttpPost("http://example.com/upload-file");
try {
AndroidMultiPartEntity ampEntity = new AndroidMultiPartEntity(
new ProgressListener() {
#Override
public void transferred(long num) {
// this trigger the progressUpdate event
publishProgress((int) ((num / (float) totalSize) * 100));
}
});
File myFile = new File("/my/image/path/example.jpg");
ampEntity.addPart("fileFieldName", new FileBody(myFile));
totalSize = ampEntity.getContentLength();
httpPost.setEntity(ampEntity);
// Making server call
HttpResponse httpResponse = httpClient.execute(httpPost);
HttpEntity httpEntity = httpResponse.getEntity();
int statusCode = httpResponse.getStatusLine().getStatusCode();
if (statusCode == 200) {
responseString = EntityUtils.toString(httpEntity);
} else {
responseString = "Error, http status: "
+ statusCode;
}
} catch (Exception e) {
responseString = e.getMessage();
}
return responseString;
}
#Override
protected void onPostExecute(String result) {
// if you want update the user interface with upload result
super.onPostExecute(result);
}
}
So, when you want to upload your file just call:
new UploadFile().execute();
I wanted send some strings to backend server. I didnt use json with multipart, I have used request params.
#RequestMapping(value = "/upload", method = RequestMethod.POST)
public void uploadFile(HttpServletRequest request,
HttpServletResponse response, #RequestParam("uuid") String uuid,
#RequestParam("type") DocType type,
#RequestParam("file") MultipartFile uploadfile)
Url would look like
http://localhost:8080/file/upload?uuid=46f073d0&type=PASSPORT
I am passing two params (uuid and type) along with file upload.
Hope this will help who don't have the complex json data to send.
You could try using https://square.github.io/okhttp/ library.
You can set the request body to multipart and then add the file and json objects separately like so:
MultipartBody requestBody = new MultipartBody.Builder()
.setType(MultipartBody.FORM)
.addFormDataPart("uploadFile", uploadFile.getName(), okhttp3.RequestBody.create(uploadFile, MediaType.parse("image/png")))
.addFormDataPart("file metadata", json)
.build();
Request request = new Request.Builder()
.url("https://uploadurl.com/uploadFile")
.post(requestBody)
.build();
try (Response response = client.newCall(request).execute()) {
if (!response.isSuccessful()) throw new IOException("Unexpected code " + response);
logger.info(response.body().string());
#RequestMapping(value = "/uploadImageJson", method = RequestMethod.POST)
public #ResponseBody Object jsongStrImage(#RequestParam(value="image") MultipartFile image, #RequestParam String jsonStr) {
-- use com.fasterxml.jackson.databind.ObjectMapper convert Json String to Object
}
Please ensure that you have following import. Ofcourse other standard imports
import org.springframework.core.io.FileSystemResource
void uploadzipFiles(String token) {
RestBuilder rest = new RestBuilder(connectTimeout:10000, readTimeout:20000)
def zipFile = new File("testdata.zip")
def Id = "001G00000"
MultiValueMap<String, String> form = new LinkedMultiValueMap<String, String>()
form.add("id", id)
form.add('file',new FileSystemResource(zipFile))
def urld ='''http://URL''';
def resp = rest.post(urld) {
header('X-Auth-Token', clientSecret)
contentType "multipart/form-data"
body(form)
}
println "resp::"+resp
println "resp::"+resp.text
println "resp::"+resp.headers
println "resp::"+resp.body
println "resp::"+resp.status
}
I am struggling with Rest call. Here is my code and it is working for basic authentication.
public async Task RunAsync(string name, string value)
{
using (var handler = new HttpClientHandler { UseDefaultCredentials = true })
using (var client = new HttpClient(handler))
{
var byteArray = Encoding.ASCII.GetBytes("username:password");
client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Basic", Convert.ToBase64String(byteArray));
var urlRestGet = HomeController.url;
client.BaseAddress = new Uri(urlRestGet + "?name=" + name + "&value=" + value + "");
client.DefaultRequestHeaders.Accept.Clear();
**1. if(HomeController.contentType.ToLower()=="xml"){
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/xml"));
}**
else if (HomeController.contentType.ToLower() == "json")
{
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
}
HttpResponseMessage response = await client.GetAsync(urlRestGet + "?name=" + name + "&value=" + value + "");
if (response.IsSuccessStatusCode)
{
//Get the response
loginJsonString = await response.Content.ReadAsStringAsync();
//Converting to xml
using (var stream = new MemoryStream(Encoding.ASCII.GetBytes(loginJsonString)))
{
var output = new XmlDictionaryReaderQuotas();
xmlResult = XDocument.Load(JsonReaderWriterFactory.CreateJsonReader(stream, output)).ToString();
}
}
}
}
1) If the content type is application/xml am I correct to use line 1 part in the code.
2) How can I make this code more generic. (when the authentication type is different eg: tokenized or cookiebased how can I change this.)
There are a couple of things about your code I do not understand.
What is HomeController.contentType all about? The name "HomeController" suggests you're writing an ASP.NET MVC Controller (serverside). Though you seem to be writing something intended to be used as a HTTP client. But I could be mistaken or mislead here.
You are reading a Json response, then loading it as a Xml document?
I'll try to answer anyway.
1) If the content type is application/xml am I correct to use line 1 part in the code.
The Accept header sent by the client (you) tells the server that you accept the given content type. If you send Accept application/xml you tell the server you prefer if the response is Xml.
Your code seem to assume the response's content type is always Json.
You could include both application/xml and application/json as Accept headers in your request. The server should honor that request and pick the first supported content type for it's response.
When processing the response you should check the actual content type and handle the response content appropriately.
Remember that Accept only tells the server that you prefer those content types. The server may decide not to honor your whishes and can return any content type it desires.
2) How can I make this code more generic. (when the authentication type is different eg: tokenized or cookiebased how can I change this.)
If you mean tokenized as in a query parameter you should probably handle your query parameters as a collection rather than a hardcoded formatted string.
Check out the NameValueCollection class and this SO question on NameValueCollection to query string.
To handle cookies, you basically need to copy/re-use the cookie collection returned in a response in the next request.
See this SO question on how to inject cookies when you create a new HttpClient.
... but it's much easier to use a library
As you already discovered, making a robust REST/HTTP client is not a easy task.
And as #Mafii and #Fildor already pointed out in comments there are already numerous libraries available. RestSharp (https://restsharp.org) being one very popular.
In the latest Web API 2, how do I configure it so that it will only return a reply if the Accept header is application/json? This API will only support json, if any other accept header is sent then an error must be thrown. There will be no xml and even no html interface.
If the client asks for xml or html or anything, we need to throw an error to let them know they used the wrong accept type. We must not mask this problem by replying with the correct json when they have requested a type that is not actually supported.
var request = (HttpWebRequest)WebRequest.Create(url);
request.Accept = "application/json";
var response = request.GetResponse();
And the json result is returned successfully. But if there is any other Accept then an error is returned
var request = (HttpWebRequest)WebRequest.Create(url);
request.Accept = "application/xml"; // or text/html or text/plain or anything
var response = request.GetResponse();
Returns HTTP 501 Not Implemented or similar http error code.
This question is not a duplicate of How do I get ASP.NET Web API to return JSON instead of XML using Chrome? - that question asks how to also return json. My question is how to only return json, and only if the client asks for json. If the client asks for any other type like xml or html, then an error is returned.
This page shows how to access content negotiation directly. You could conceivably instead pass some filtered subset of this.Configuration.Formatters containing only the desired formatters to IContentNegotiator.negotiate, like so:
ContentNegotiationResult result = negotiator.Negotiate(
typeof(Product), this.Request, this.Configuration.Formatters.Where(/* some expression that excludes all but the json formatter */);
This looks quite clumsy and would be a lot of dull boilerplate, so Javad_Amiry's answer is probably better, but this is another option that might be useful in specific cases.
You can clear all formatters except JSON:
configuration.Formatters.Clear();
configuration.Formatters.Add(new JsonMediaTypeFormatter());
Or you can change the default Web API’s content negotiation mechanism:
public class JsonContentNegotiator : IContentNegotiator
{
private readonly JsonMediaTypeFormatter _jsonFormatter;
public JsonContentNegotiator(JsonMediaTypeFormatter formatter)
{
_jsonFormatter = formatter;
}
public ContentNegotiationResult Negotiate(Type type, HttpRequestMessage request, IEnumerable<MediaTypeFormatter> formatters)
{
var result = new ContentNegotiationResult(_jsonFormatter, new MediaTypeHeaderValue("application/json"));
return result;
}
}
// in app_start:
var jsonFormatter = new JsonMediaTypeFormatter();
config.Services.Replace(typeof(IContentNegotiator), new JsonContentNegotiator(jsonFormatter));
See the article.
UPDATE:
Well, if you want to return a HTTP error on non-json request, you can do it by implementing a custom IHttpModule for checking header. But, for self-host apps it won't work. So, it's better to use extend a custom DelegatingHandler. For example, you can use this one:
public class FilterJsonHeaderHandler : DelegatingHandler {
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request, CancellationToken cancellationToken){
if (request.Headers.Accept.All(a => a.MediaType == "application/json")){
// if we have only application/json, so the pipeline continues
return base.SendAsync(request, cancellationToken);
}
// otherwise, do whatever you want:
var response = new HttpResponseMessage(HttpStatusCode.NotImplemented);
var completionSource = new TaskCompletionSource<HttpResponseMessage>();
completionSource.SetResult(response);
return completionSource.Task;
}
}
and register it in app_start:
public static class WebApiConfig {
public static void Register(HttpConfiguration config) {
config.MessageHandlers.Add(new FilterJsonHeaderHandler());
// your other settings...
}
}
NOTE: the code is not tested. Please let me know if there is any error.
I need to get the url of the final destination of a shortened url. At the moment I am doing the following which seems to work:
var request = WebRequest.Create(shortenedUri);
var response = request.GetResponse();
return response.ResponseUri;
But can anyone suggest a better way?
If this shortened url is generated by some online service provider it is only this service provider that is storing the mapping between the short and the actual url. So you need to query this provider by sending an HTTP request to it, exactly as you did. Also don't forget to properly dispose IDisposable resources by wrapping them in using statements:
var request = WebRequest.Create(shortenedUri);
using (var response = request.GetResponse())
{
return response.ResponseUri;
}
If the service provider supports the HEAD verb you could also use this verb and read the Location response HTTP header which must be pointing to the actual url. As an alternative you could set the AllowAutoRedirect property to false on the request object and then read the Location response HTTP header. This way the client won't be redirecting to the actual resource and getting the entire response body when you are not interested in it.
Of course the best way to do this would be if your online service provider offers an API that would allow you to directly give you the actual url from a short url.
You do need to make an HTTP request - but you don't need to follow the redirect, which WebRequest will do by default. Here's a short example of making just one request:
using System;
using System.Net;
class Test
{
static void Main()
{
string url = "http://tinyurl.com/so-hints";
Console.WriteLine(LengthenUrl(url));
}
static string LengthenUrl(string url)
{
var request = WebRequest.CreateHttp(url);
request.AllowAutoRedirect = false;
using (var response = request.GetResponse())
{
var status = ((HttpWebResponse) response).StatusCode;
if (status == HttpStatusCode.Moved ||
status == HttpStatusCode.MovedPermanently)
{
return response.Headers["Location"];
}
// TODO: Work out a better exception
throw new Exception("No redirect required.");
}
}
}
Note that this means if the "lengthened" URL is itself a redirect, you won't get the "final" URI as you would in your original code. Likewise if the lengthened URL is invalid, you won't spot that - you'll just get the URL that you would have redirected to. Whether that's a good thing or not depends on your use case...