We Keep Coding

Read HTTPWebReponse using GetResponseStream readToEnd return strange characters

we are reading response from HttpWebResponse and the response itself is quite normal, we think.
But when we try to read the string representation of that, we only receive strange characters instead of json formatted normal chars.
Can anyone please tell us what is wrong here? Any question or help is welcomed, if more informations are needed, please comment. Thanks

The response content encoding header is gzip indicating the content has been compressed by the server.
Set request.AutomaticDecompression to tell the HttpWebResponse you want it to decompress the data automatically.

HTTP request whose headers can be controlled and is automatically decompressed

I'm trying to send HTTP requests in C# that look like HTTP requests from a certain software. I wanted to use System.Net.HttpWebRequest but it doesn't give me the control I need over its headers: their letter-casing can't be changed (e.g. I want the Connection header to be keep-alive and not Keep-Alive), I don't have full control over the headers ordering, etc.
I tried using HttpClient from CodeScales library. Unfortunately, it doesn't decompress responses automatically (see HttpWebRequest.AutomaticDecompression). I decompressed it myself with System.IO.Compression.GZipStream and DeflateStream, but it didn't work when the response had the header Transfer-Encoding: chunked.
System.Net.Http.HttpRequestHeaders seems to give more control over headers than HttpWebRequest, but still not enough.
How can it be done?
Edit: I know that HTTP accepts those headers as valid anyway, but I'm working with a server that validates the headers and refuses to respond if they're not exactly what it expects.

To set some headers in the HTTPWebRequest class, you have to either use an attribute from the class (for example HttpWebRequest.KeepAlive = true), or you have to add the custom header to the request by calling the add method to the request headers.
Something important is that is you try to add the header (in a custom way) while it's already an attribute of the request, it'll send you an error.
objRequest.Headers.Add("Accept", "some data");
is incorrect. You'd rather say.
objRequest.Accept = "some data";
In your case you can :
objRequest.KeepAlive = true;
Don't worry to much for the letter-casing, it doesn't matter as far as you're sending the appropriate headers to the server.

Get XML body of a 400 or 500 HTTP XML response

Long story short, I am sending an XML HTTP post request to an application server, and I am getting back a response, also in the form of XML HTTP.
I have a test site available to me which allows me to see what the server's actual response is, visually, in the form of XML, but I cannot access this XML from my C# code the way it is.
The XML coming back from the application server in my test case looks like this:
<Error><Message>StringErrorMessage</Message></Error>
However, I have had no luck accessing this basic XML to retrieve the value of "StringErrorMessage" for the creation of a detailed error report.
... More code above, all wrapped in a try{}...
_response = Serializer.DeserializeObject<T>(ObjectRequest.GetResponse().GetResponseStream());
}
catch (System.Net.WebException exceptionParameter)
{
var response = (HttpWebResponse)exceptionParameter.Response;
string webExceptionStatus = exceptionParameter.Message;
_exception = exceptionParameter;
return false;
}
I have consulted
C# - Getting the response body from a 403 error
and
Get response body on 400 HTTP response in Android?
The first link's solution doesn't seem to give me access to the basic XML as part of any response object's properties. I am almost positive that there must be a byte[] in there somewhere (in the response, or in the exception object) that can be converted into a char[], which can be converted to a string, which can be converted to my XML body, but I have not been able to find it. The second link's solution is not exactly viable for me because I have to get the response body back in the form of XML, as it might not be an error, but an object that must be deserialized. This particular side of things, I cannot change.
Any advice would be very much appreciated.
- Eli
EDIT: Just wanted to clarify that my basic code is working okay for non-error situations, and is deserializing the XML just fine. It's when my code encounters a HTTP 400 or an HTTP 500 error, where accessing the XML from the catch statement becomes a problem, because my code immediately throws an exception.

The body of a HTTP message (the XML in your case) can be retrieved with the GetResponseStream method of the HttpWebResponse object you have. And, since it's a stream, you can for instance read it with a StreamReader, like so:
HttpWebResponse myWebResponse; // Get this from whereever you want
Stream responseStream = myWebResponse.GetResponseStream();
StreamReader reader = new StreamReader(responseStream);
string niceStringForYou = reader.ReadToEnd();
...and from that point on, you can do whatever to it.
If you're absolutely sure it's always gonna be XML you get back from the service, you can probably even use an XmlReader to get XML directly from the stream:
XmlReader foo = XmlReader.Create(responseStream);
Comment to edit: As long as you have the HttpWebResponse object, reading it's response stream (GetResponseStream()) should work. And as you point out in your own code, you can get the HttpWebResponse by looking at (HttpWebResponse)exceptionParameter.Response.

Is there a way to know if a HttpResponse is in Https?

In a method, I need to know if the response is in Http or Https in order to redirect the response or to make a BinaryWrite but I only have a HttpResponse in parameter.
Anyone know if it's possible to use the HttpResponse to know that?
Thanks.
EDIT:
I want to know if it's possible to know if a HttpResponse is HTTP or HTTPS, because I prefer not to use HttpContext.Current.Request if possible.

I don't believe so. You can find out from the HttpRequest, but a response is just a stream of data returned in relation to the Request.

This should give you what you need;
System.Uri currentUrl = System.Web.HttpContext.Current.Request.Url;
if (!currentUrl.Scheme.Equals(Uri.UriSchemeHttps, stringComparison.CurrentCultureIgnoreCase))
{
//Do something here
}
It is not possible to extract this information from the HttpResponse object as this is what will be sent back to the client, over the same communication protocol that the request was probably requested from. You will need to use the Request object to check for an SSL connection.

HttpRequest.IsSecureConnection property will do the job look at the example on the MS link http://msdn.microsoft.com/en-us/library/system.web.httprequest.issecureconnection(v=vs.110).aspx

Add request headers with WebClient C#

I have the following code with which I download a web-page into a byte array and then print it with Response.Write:
WebClient client = new WebClient();
byte[] data = client.DownloadData(requestUri);
/*********** Init response headers ********/
WebHeaderCollection responseHeaders = client.ResponseHeaders;
for (int i = 0; i < responseHeaders.Count; i++)
{
Response.Headers.Add(responseHeaders.GetKey(i), responseHeaders[i]);
}
/***************************************************/
Besides of the response headers, I need to add request headers as well. I try to do it with the following code:
/*********** Init request headers ********/
NameValueCollection requestHeaders = Request.Headers;
foreach (string key in requestHeaders)
{
client.Headers.Add(key, requestHeaders[key]);
}
/***************************************************/
However it does not work and I get the following exception:
This header must be modified using the appropriate property.Parameter name: name
Could anybody help me with this? What's the correct way of adding request headers with WebClient?
Thank you.

The headers collection "protects" some of the possible headers as described on the msdn page here: http://msdn.microsoft.com/en-us/library/system.net.webclient.headers.aspx
That page seems to give all the answer you need but to quote the important part:
Some common headers are considered restricted and are protected by the
system and cannot be set or changed in a WebHeaderCollection object.
Any attempt to set one of these restricted headers in the
WebHeaderCollection object associated with a WebClient object will
throw an exception later when attempting to send the WebClient
request.
Restricted headers protected by the system include, but are not
limited to the following:
Date
Host
In addition, some other headers are also restricted when using a
WebClient object. These restricted headers include, but are not
limited to the following:
Accept
Connection
Content-Length
Expect (when the value is set to "100-continue"
If-Modified-Since
Range
Transfer-Encoding
The HttpWebRequest class has properties for setting some of the above
headers. If it is important for an application to set these headers,
then the HttpWebRequest class should be used instead of the WebRequest
class.
I suspect the reason for this is that many of the headers such as Date and host must be set differently on a different request. You should not be copying them. Indeed I would personally probably suggest that you should not be copying any of them. Put in your own user agent - If the page you are getting relies on a certain value then I'd think you want to make sure you always send a valid value rather than relying on the original user to give you that information.
Essentially work out what you need to do rather than finding something that works and doing that without fully understanding what you are doing.

Looks like you're trying to set some header which is must be set using one of the WebClient properties (CachePolicy, ContentLength or ContentType)
Moreover, it's not very good to blindly copy all the headers, you need to get just those you really need.