I need to read a body of a response where a Content-Length is explicitly specified as zero. But the stream.Read does not read anything and returns 0. When the Content-Length header is not present, it reads all the bytes successfully.
When I open the web page in the browser it is empty, but I can see the contents in Fiddler.
Is there any way to get the Stream returned from GetResponseStream to read the bytes when the Content-Length is 0?
Sample response:
HTTP/1.1 200 OK
Cache-Control: public, must-revalidate
ETag: "c7f40cb26b1e95c2245f1584371465582f996a8a88b34a2cc99bbe922b1a2857"
Vary: Accept-Encoding
X-Request-Id: 4b799b6d-cd68-47f7-a392-4fd0a327f5de
X-Runtime: 0.001646
Content-Length: 0
- some content -
Simplified version of the code:
using(var stream = httpRes.GetResponseStream())
{
while (true)
{
var bytesRead = stream.Read(
responseBuffer,
0,
responseBuffer.Length);
// other stuff
}
}
When an HTTP/1.1 has Content-Length: 0, then, by definition, there is no payload to read. If bytes follow, they by definition are the response to the subsequent request.
Now, obviously, this is not the case here. So, just to be absolutely clear: you're asking for a workaround to deal with a broken server. It would be much cleaner to fix the server instead.
Related
I'm trying to "repurpose" a third-party API used by a desktop application. I've found that the below code gets me very close to matching the packets sent by the app:
var formData = new FormUrlEncodedContent(new[]
{
new KeyValuePair<string, string>(JsonConvert.SerializeObject(myPayload), "")
});
var response = Client.PostAsync(myURL, formData).Result;
var json = response.Content.ReadAsStringAsync().Result;
This gets me almost exactly the same payload sent by the application, except it encodes the data (I know, "encoded" is right there in the name). I need to get the exact same request but without the data being encoded, but I can't quite find the right object(s) to pull it off. How do I keep this payload from being URL encoded?
Edit:
This is a login request I pulled from Wireshark emanating from the application:
POST /Login HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Host: 1.1.1.1
Content-Length: 161
Expect: 100-continue
Connection: Close
{"username":"myuser","auth-id":"0a0a140f81a2ce0c303386e93cec41bf04660c22a881be9a"}
This is what the above will generate:
POST /Login HTTP/1.1
Expect: 100-continue
Connection: Close
Content-Type: application/x-www-form-urlencoded
Content-Length: 221
Host: 1.1.1.1
%7B%22user-name%22%3A%22myuser%22%2C%22auth-id%22%3A%220a0a140f81a2ce0c303386e93cec41bf04660c22a881be9a%22%7D=
I've edited them for brevity so the Content-Length is wrong. I realize it might not be the best way to send this data, but I have no control over how it's consumed.
Since you're actually trying to send JSON, I think you need to wrap the JSON in a StringContent object rather than a FormUrlEncoded object. Form-encoded data and JSON data are two different ways of formatting a payload (another commonly used format would be XML, for example). Using them both together doesn't make any sense.
I think something like this should work:
var content = new StringContent(JsonConvert.SerializeObject(myPayload), Encoding.UTF8, "application/json");
var response = Client.PostAsync(myURL, content).Result;
var json = response.Content.ReadAsStringAsync().Result;
(P.S. the Content-Type: application/x-www-form-urlencoded header sent by the application appears to be misleading, since the request body clearly contains JSON. Presumably the receiving server is tolerant of this nonsense, or just ignores it because it's always expecting JSON.)
I am creating a very simple web server where its only purpose is to return an mp3 file.
// create server
TcpListener server = new TcpListener(8585);
server.Start();
while (true)
{
// accept a new client
using (var client = server.AcceptTcpClient())
{
byte[] buffer = new byte[1024 * 2];
var i = client.Client.Receive(buffer);
// get request client sent us
var request = System.Text.Encoding.UTF8.GetString(buffer, 0, i).ToLower();
// if the browser is requesting an icon return
if (request.Contains("/favicon.ico"))
{
continue;
}
// with fiddler when I downloaded great.mp3 I saved the response as a binary file to make sure I am returning the right output
client.Client.Send(System.IO.File.ReadAllBytes("cheat.bin"));
}
}
cheat.bin can be downloaded from here
it basically consists of
HTTP/1.1 200 OK
Server: Apache/2.2.3 (CentOS)
X-HOST: sipl-web-233.oddcast.com
X-TTSCache: HIT
X-TTSCacheInsert: NO
Content-Type: audio/mpeg
Access-Control-Allow-Origin: *
Vary: Accept-Encoding
Date: Thu, 01 May 2014 00:02:15 GMT
Content-Length: 5471
Connection: keep-alive
ID3���� .... rest of mp3 file!
So my question is why is it that when I go to http://localhost:8585/getSong on my web browser the song is downloaded twice? In other words if I place a breakpoint in my code I hit it twice. Also I will not be able to play the audio in the browser until I return the song a second time.
I am asking this question primarily to learn. I dont understand what is wrong.
Browsers often make a HTTP head call before a HTTP get don't they? Could that be why?
https://ochronus.com/http-head-request-good-uses/
Anyway I would use something like Fiddler to see exactly what HTTP requests the browser is making.
I have tried unseccessfully to send the next header to the Twitter API:
POST /oauth2/token HTTP/1.1
Host: api.twitter.com
User-Agent: My Twitter App v1.0.23
Authorization: Basic eHZ6MWV2R ... o4OERSZHlPZw==
Content-Type: application/x-www-form-urlencoded;charset=UTF-8
Content-Length: 29
Accept-Encoding: gzip
grant_type=client_credentials
I searched and found that there's a method called WebClient.UploadData(this method, implicitly sets HTTP POST as the request method) but I dont really
know how to work with it.
I know how to change the current headers using Set method.
But what about the HTTP body message? how can I add some body to the header?(grant_type)
PS: I read the documention.
Not much to it unless you are dealing with multi-part data. Just build a string with the post data (URL encode if the data requires it), get the bytes, set the content-length, and write your data to the request stream.
string postData = String.Format("field1=value1&field2=value2");
byte[] postBytes = System.Text.ASCIIEncoding.UTF8.GetBytes(postData);
HttpWebRequest req = HttpWebRequest.Create("http://myurl.com");
// ... other request setup stuff
req.ContentLength = postBytes.Length;
using (var stream = req.GetRequestStream())
{
stream.Write(postBytes, 0, postBytes.Length);
}
PLEASE HELP!! Can't figure out why this simple code given by MSDN doesn't work....
I am using the following code in GetAccessToken() as given in the this MSDN article to get the access token to be used in windows notifications, but it returns "Bad Request 400"
PACKAGE_SECURITY_IDENTIFIER, CLIENT_SECRET are the values obtained when the app was registered with the Windows Store Dashboard
string urlEncodedSid = HttpUtility.UrlEncode(PACKAGE_SECURITY_IDENTIFIER);
string urlEncodedSecret = HttpUtility.UrlEncode(CLIENT_SECRET);
string body = String.Format("grant_type=client_credentials&client_id={0}&client_secret={1}&scope=notify.windows.com", urlEncodedSid, urlEncodedSecret);
string response;
using (WebClient client = new WebClient())
{
client.Headers.Add("Content-Type", "application/x-www-form-urlencoded");
response = client.UploadString("https://login.live.com/accesstoken.srf", body);
}
Any help would be highly appreciated.......
I suspect the problem has to do with either an incorrect package identifier, and / or incorrect client secret.
From the MSDN page Push notification service request and response headers:
RESPONSE DESCRIPTION
--------------- --------------------------
200 OK The request was successful.
400 Bad Request The authentication failed.
Update - I ran the code from the question, using FAKE credentials.
Here is the RAW HTTP request:
POST https://login.live.com/accesstoken.srf HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Host: login.live.com
Content-Length: 88
Expect: 100-continue
Connection: Keep-Alive
grant_type=client_credentials&client_id=test&client_secret=test&scope=notify.windows.com
Here is the server's RAW response:
HTTP/1.1 400 Bad Request
Cache-Control: no-store
Content-Length: 66
Content-Type: application/json
Server: Microsoft-IIS/7.5
X-WLID-Error: 0x80045A78
PPServer: PPV: 30 H: BAYIDSLGN2A055 V: 0
Date: Thu, 21 Mar 2013 12:34:19 GMT
Connection: close
{"error":"invalid_client","error_description":"Invalid client id"}
You will note that the response is a 400. There is also some json that indicates the type of error. In my case, the error is Invalid client id. You probably want to take a look at your response - it will give you an indication of what happened.
I used Fiddler to debug the request/ response.
I found the reason for the error response. In fact it is the wrong PACKAGE_SECURITY_IDENTIFIER and CLIENT_SECRET.
DO NOT type the values. Because associated ASCII values differ. Therefore it is always better to copy and paste directly.
You will probably will get the access token with the simple code snippet.
Cheers
If you're using the new HttpClient API and you're sure you've copied and pasted the SID/secret values correct, you might be experiencing this issue because of encoding, provided you're using the FormUrlEncodedContent class as the content of your POST operation.
Contrary to the examples in the MSDN documentation, you don't want to URL encode the SID and secret values before adding them to the KeyValuePair collection. This is because encoding is implied by the FormUrlEncodedContent class, though I'm not seeing any documentation for this behavior. Hopefully this saves someone some time because I've been wrestling with this all night...
If I have a raw HTTP response as a string:
HTTP/1.1 200 OK
Date: Tue, 11 May 2010 07:28:30 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=UTF-8
Server: gws
X-XSS-Protection: 1; mode=block
Connection: close
<!doctype html><html>...</html>
Is there an easy way I can parse it into an HttpListenerResponse object? Or at least some kind .NET object so I don't have to work with raw responses.
What I'm doing currently is extracting the header key/value pairs and setting them on the HttpListenerResponse. But some headers can't be set, and then I have to cut out the body of the response and write it to the OutputStream. But the body could be gzipped, or it could be an image, which I can't get to work yet. And some responses contain random characters everywhere, which looks like an encoding problem. It's a lot of trouble.
I'm getting a raw response because I'm using SOCKS to send an HTTP request. The program I'm working on is basically an HTTP proxy that can route requests through a SOCKS proxy, like Privoxy does.
Looks like there's no easy way to convert them. These articles helped:
How to implement custom proxy server?
http://www.jeffcrossman.com/2009/08/27/using-httplistener-to-build-a-proxy
I ended up doing something very similar.
Maybe you want to take a look at the ResponseHeaders property of the Webclient:
WebClient wc = new WebClient();
wc.DownloadStringAsync(new Uri("http://www.foo.com"));
WebHeaderCollection myWebHeaderCollection = myWebClient.ResponseHeaders;
for (int i=0; i < myWebHeaderCollection.Count; i++)
Console.WriteLine ("\t" + myWebHeaderCollection.GetKey(i) +
" = " + myWebHeaderCollection.Get(i));
Please tell me if that isnt what you were searching for.