How to make HttpClient ignore Content-Length header - c#

I am using HttpClient to communicate with a server which I don't have access to. Sometimes the JSON response from the server is truncated.
The problem occurs when the Content-Length header is smaller than what it should be (8192 vs. 8329). It seems like a bug on the server which gives a smaller Content-Length header than the actual size of the response body. If I use Google Chrome instead of HttpClient, the response is always complete.
Therefore, I want to make HttpClient to ignore the wrong Content-Length header and read to the end of the response. Is it possible to do that? Any other solution is well appreciated. Thank you!
This is the code of my HttpClient:
var client = new HttpClient();
client.BaseAddress = new Uri(c_serverBaseAddress);
HttpResponseMessage response = null;
try
{
response = await client.GetAsync(c_serverEventApiAddress + "?location=" + locationName);
}
catch (Exception e)
{
// Do something
}
var json = response.Content.ReadAsStringAsync().Result;
var obj = JsonConvert.DeserializeObject<JObject>(json); // The EXCEPTION occurs HERE!!! Because the json is truncated!
EDIT 1:
If I use HttpWebRequest, it can read to the end of the JSON response completely without any truncation. However, I would like to use HttpClient since it has better async/await.
This is the code using HttpWebRequest:
var url = c_serverBaseAddress + c_serverEventApiAddress + "?location=" + "Saskatchewan";
var request = (HttpWebRequest)WebRequest.Create(url);
request.ProtocolVersion = HttpVersion.Version10;
request.Method = "GET";
request.ContentType = "application/x-www-form-urlencoded";
var response = (HttpWebResponse)request.GetResponse();
StringBuilder stringBuilder = new StringBuilder();
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
string line;
while ((line = reader.ReadLine()) != null)
{
stringBuilder.Append(line);
}
}
var json = stringBuilder.ToString(); // COMPLETE json response everytime!!!

You could try specifying the buffer size of the response content
var client = new HttpClient();
client.MaxResponseContentBufferSize = <your_buffer_size>;
Where the property MaxResponseContentBufferSize means:
Gets or sets the maximum number of bytes to buffer when reading the response content.

Related

xml data POST from file

i am trying to send xml data from file to sensor what should get XML and make a values from that. I have this code, but i don't know if it is actually working.
I do have response from server but does it actually get those data from xml file? I am still newbie so i still don't know.
Thank you for any help..
class Program
{
static void Main(string[] args)
{
WebRequest request = WebRequest.Create("http://192.168.254.20:5050/token");
request.Method = "POST";
string postData = #"C:\Users\lvrabel\Desktop\Crajsons\finals\Output.xml";
byte[] byteArray = Encoding.UTF8.GetBytes(postData);
request.ContentType = "application/xml";
request.ContentLength = byteArray.Length;
Stream dataStream = request.GetRequestStream();
dataStream.Write(byteArray, 0, byteArray.Length);
dataStream.Close();
WebResponse response = request.GetResponse();
Console.WriteLine(((HttpWebResponse)response).StatusDescription);
dataStream = response.GetResponseStream();
StreamReader reader = new StreamReader(dataStream);
string responseFromServer = reader.ReadToEnd();
Console.WriteLine(responseFromServer);
Console.WriteLine(postData);
Console.ReadKey();
reader.Close();
dataStream.Close();
response.Close();
}
}
You need to read your xml file then you can convert it to bytes and then you can post it to server.
byte[] xmlData= File.ReadAllBytes(#"C:\Users\lvrabel\Desktop\Crajsons\finals\Output.xml");
This is an other approach:
var xmlFile = #"C:\Users\lvrabel\Desktop\Crajsons\finals\Output.xml";
var client = new HttpClient { BaseAddress = new Uri("http://192.168.254.20:5050/")};
using(var content = new StringContent(File.ReadAllText(xmlFile), Encoding.UTF8, "text/xml"))
{
var result = await client.PostAsync("token", content);
var respnseText = await result.Content.ReadAsStringAsync();
}
(Note: if you are accessing the same service endpoint with the same settings multiple times you can reuse the HttpClient instance to call the different service methods even across threads. This is why I have not wrapped it into a using clause)
If you have difficulties communicating with the service you might have other problems. If you don't get an exception then you can inspect the properties of the result to see if the service endpoint returned success (OK=200) or some other result code. You might find details about it. You could also simply reproduce this code in html+javascript, there you can more easily inspect the traffic in the browser. Of course, you could use some network sniffer, but that can be far more complicated.
[Update]
If you really only need to pass the file name as GET parameter, then use something like this:
var xmlFile = #"C:\Users\lvrabel\Desktop\Crajsons\finals\Output.xml";
var uri = new Uri($"http://192.168.254.20:5050/token?content={xmlFile}", UriKind.Absolute);
using (var client = new HttpClient())
{
var result = await client.GetAsync(uri);
var respnseText = await result.Content.ReadAsStringAsync();
}

C# how to consume rest web service

I am having a problem when I try using a rest web service in C#.
When I try via Fiddler it works Ok.
When I try via HTML/Ajax, it works Ok, as well.
When I try via C# (Console Application) I get an error.
This image is captured in fiddler. It is what I get when I try via ajax
this image is also captured in fiddler. It is what I get when I try via C#
As you can see, the JSON field is empty.
This is my C# code
string json = JsonConvert.SerializeObject(abc);
HttpClient client = new HttpClient();
client.BaseAddress = new Uri("MyURL"); //==> I am filling it correctly
client.DefaultRequestHeaders.Accept.Add(
new MediaTypeWithQualityHeaderValue("application/json"));
var response = client.PostAsJsonAsync("MyMethod", json).Result; //==> I am filling my method correctly
But I have tried several others and always getting the same problem. (the code bellow is another one I tried)
var requisicaoWeb = WebRequest.CreateHttp("MyURL");
requisicaoWeb.Method = "POST";
requisicaoWeb.ContentType = "application/json";
requisicaoWeb.ContentLength = dados.Length;
requisicaoWeb.UserAgent = "Console app";
requisicaoWeb.Accept = "Accept:application/json,text/javascript,*/*;q=0.01";
//precisamos escrever os dados post para o stream
using (var stream = requisicaoWeb.GetRequestStream())
{
stream.Write(MyJson, 0, dados.Length);
stream.Close();
}
//ler e exibir a resposta
using (var resposta = requisicaoWeb.GetResponse())
{
var streamDados = resposta.GetResponseStream();
StreamReader reader = new StreamReader(streamDados);
object objResponse = reader.ReadToEnd();
var post = objResponse.ToString();//JsonConvert.DeserializeObject<Post>(objResponse.ToString());
streamDados.Close();
resposta.Close();
}
Everything I try in C#, the JSON field on Fiddler is always empty and the "syntax View" description is always "Request Invalid".
Try it's;
public static string HttpPost(string URI, string Parameters)
{
System.Net.WebRequest req = System.Net.WebRequest.Create(URI);
req.ContentType = "application/json; charset=utf-8";
req.Method = "POST";
req.Timeout = 600000;
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(Parameters);
req.ContentLength = bytes.Length;
System.IO.Stream os = req.GetRequestStream();
os.Write(bytes, 0, bytes.Length);
os.Close();
System.Net.WebResponse resp = req.GetResponse();
if (resp == null)
return null;
System.IO.StreamReader sr = new System.IO.StreamReader(resp.GetResponseStream());
return sr.ReadToEnd().Trim();
}
I have just figure it out.
If anybody else has the same problem, here is the answer
string json = JsonConvert.SerializeObject(abc);
HttpClient client = new HttpClient();
client.BaseAddress = new Uri("MyURL"); //==> I am filling it correctly
client.DefaultRequestHeaders.Accept.Add(
new MediaTypeWithQualityHeaderValue("application/json"));
var stringContent = new StringContent(JsonConvert.SerializeObject(abc), Encoding.UTF8, "application/json");
var response = client.PostAsync("MyURL", stringContent).Result; //==> I am filling my method correctly

JSON format is being returned with missing data using C# (HTTPWEBREQUEST)

I'm working with JSON and C# ( HttpWebRequest ). Basically I have application to download a JSON from and API REST, but the problem is when I download it, the JSON comes missing some data, it seems that is cutting some data, with wrong structure. If I use a software which does the same thing that I'm developing, this problem doesn't happen. I'm sure that is something with my code, if I'm missing something. Here is my code:
var httpWebRequest = (HttpWebRequest)WebRequest.Create("MyURL");
httpWebRequest.ContentType = "application/json";
httpWebRequest.Method = "GET";
string authInfo = "user" + ":" + "pass";
authInfo = Convert.ToBase64String(Encoding.Default.GetBytes(authInfo));
httpWebRequest.Headers["Authorization"] = "Basic " + authInfo;
// Create the HttpContent for the form to be posted.
var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
using (var sr = new StreamReader(httpResponse.GetResponseStream(), Encoding.UTF8))
{
StreamWriter sw = new StreamWriter(#"C:\test\Stores.txt");
sw.Write(sr.ReadToEnd());
}
You can try this.its works in my code.
public static async Task MethodName()
{
using (HttpClientHandler handler = new HttpClientHandler() { UseCookies = false })
{
using (HttpClient httpClient = new HttpClient(handler))
{
httpClient.DefaultRequestHeaders.Authorization = Program.getAuthenticationHeader();
string filterQuery = Program.getURI().ToString();
using (HttpResponseMessage httpResponse = await httpClient.GetAsync(filterQuery).ConfigureAwait(false))
{
var streamContent = await httpResponse.Content.ReadAsStreamAsync();
FileStream fs = new FileStream("C:\test\Stores.Json", FileMode.Create);
streamContent.CopyTo(fs);
streamContent.Close();
fs.Close();
}
}
}
}
This can be an issue with your Http request (GET).
Step 1 - If you have a working software with the API, use Fiddler to analyse what is the http GET request it sends. You need to check the header info as well.
Step 2 - Compare the Http request with the HttpRequest you have created. There can be missing parameters etc.

Getting content from httpwebresponse exception

My remote server is throwing a web exception as bad request. But I know there is more information included in the error than what I get. If I look at the details from the exception it does not list the actual content of the response. I can see the content-type, content-length and content-encoding only. If I run this same message through another library (such as restsharp) I will see detailed exception information from the remote server. How can I get more details from the response since I know the remote server is sending them?
static string getXMLString(string xmlContent, string url)
{
//string Url;
string sResult;
//Url = ConfigurationManager.AppSettings["UserURl"] + url;
var httpWebRequest = (HttpWebRequest)WebRequest.Create(url);
httpWebRequest.ContentType = "application/xml";
httpWebRequest.Method = "POST";
using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
{
streamWriter.Write(xmlContent);
streamWriter.Flush();
streamWriter.Close();
var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
var result = streamReader.ReadToEnd();
sResult = result;
}
}
return sResult;
}
EDIT : Have you tried with a simple try-catch to see if you can get more details ?
try
{
var response = (HttpWebResponse)(request.GetResponse());
}
catch(Exception ex)
{
var response = (HttpWebResponse)ex.Response;
}
In my recherches in an answer for you, I noticed that in code there was something about encoding, that you didn't specified. Look here for exemple with such code.
var encoding = ASCIIEncoding.ASCII;
using (var reader = new System.IO.StreamReader(response.GetResponseStream(), encoding))
{
string responseText = reader.ReadToEnd();
}
Or here, in the doc, also.
// Creates an HttpWebRequest with the specified URL.
HttpWebRequest myHttpWebRequest = (HttpWebRequest)WebRequest.Create(url);
// Sends the HttpWebRequest and waits for the response.
HttpWebResponse myHttpWebResponse = (HttpWebResponse)myHttpWebRequest.GetResponse();
// Gets the stream associated with the response.
Stream receiveStream = myHttpWebResponse.GetResponseStream();
Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
// Pipes the stream to a higher level stream reader with the required encoding format.
StreamReader readStream = new StreamReader( receiveStream, encode );
Console.WriteLine("\r\nResponse stream received.");
Have you tried with such ?

Get webpage page content and HTTP status code in C#

In a C# Windows Forms application I can get the contents of a webpage using:
string content = webClient.DownloadString(url);
And I can get the HTTP header using:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
string response = ((HttpWebResponse)request.GetResponse()).StatusCode.ToString();
Is there a way to get both the contents and the HTTP status code (if it fails) in one trip to the server instead of twice?
Thanks.
You can read the data from the Stream inside the HttpWebResponse object:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
using (var response = request.GetResponse())
using (var stream = response.GetResponseStream())
using (var reader = new StreamReader(stream))
{
HttpStatusCode statusCode = ((HttpWebResponse)response).StatusCode;
string contents = reader.ReadToEnd();
}
In this way you will have to detect the encoding by hand, or using a library to detect encoding. You can read the encoding as a string from the HttpWebResponse object as well, when one exists, it is inside the ContentType property. If the page is Html, then you will have to parse it for a possible encoding change in the top of the document or inside the head.
Read handling the encoding from ContentType header
var request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
string content;
HttpStatusCode statusCode;
using (var response = request.GetResponse())
using (var stream = response.GetResponseStream())
{
var contentType = response.ContentType;
Encoding encoding = null;
if (contentType != null)
{
var match = Regex.Match(contentType, #"(?<=charset\=).*");
if (match.Success)
encoding = Encoding.GetEncoding(match.ToString());
}
encoding = encoding ?? Encoding.UTF8;
statusCode = ((HttpWebResponse)response).StatusCode;
using (var reader = new StreamReader(stream, encoding))
content = reader.ReadToEnd();
}
WebClient
I assume you use WebClient because its easy webrequest-to-string handling. Unfortunately, WebClient does not expose the HTTP response code. You can either assume the response was positive (2xx) unless you get an exception and read it:
try
{
string content = webClient.DownloadString(url);
}
catch (WebException e)
{
HttpWebResponse response = (System.Net.HttpWebResponse)we.Response;
var statusCode = response.StatusCode;
}
Or if you're really interested in the success code you can use reflection as explained here.
HttpClient
You can also use HttpClient if you're on .NET 4.5, which does expose the response code, as explained here:
using (HttpClient client = new HttpClient())
{
HttpResponseMessage response = await client.GetAsync(url);
string content = await response.Content.ReadAsStringAsync();
var statusCode = response.StatusCode;
}
HttpWebRequest
Alternatively, you can just use HttpWebRequest to get the status and response as explained here:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
var response = (HttpWebResponse)request.GetResponse();
using (Stream stream = response.GetResponseStream())
{
StreamReader reader = new StreamReader(stream);
string content = reader.ReadToEnd();
var statusCode = response.StatusCode;
}
I think, you have not realised, that in the second case you have access to the content as well (although it takes a little more effort to get as a string).
Look at the Microsoft documentation: http://msdn.microsoft.com/en-us/library/system.net.httpwebresponse.getresponsestream(v=vs.110).aspx which shows you how to ge a response stream from the web response, and then how to get the string data from that stream.
And I can get the HTTP header using:
request.Method = "GET";
Method GET returns HEAD and BODY sections in response.
HTTP also support a method HEAD - which returns HEAD section only.
You can get BODY from HttpWebResponse using GetResponseStream method.

Categories