I'm trying to receive data from compressor, but it's answer is always empty. Here is my code:
WebClient client = new WebClient();
client.Headers[HttpRequestHeader.ContentType]= "application/x-www-form-urlencoded";
string question = "online_pressure";
string URL = "http://10.0.163.51/getVar.cgi";
string answer = client.UploadString(URL, "POST", question);
Console.WriteLine(answer);
When I use this code for another compressor, which different only 2 strings, it works great and I can see answer in console:
string question = "QUESTION=300201";
string URL = "http://10.0.163.50/cgi-bin/mkv.cgi";
Code in VBS works great for both compressors. I can see answer in MsgBox from first and second compressors:
Dim objHTTP
strToSend = "online_pressure"
Set objHTTP = CreateObject("Microsoft.XMLHTTP")
Call objHTTP.Open("POST", "http://10.0.163.51/getVar.cgi", false)
objHTTP.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
objHTTP.Send strToSend
MsgBox(objHTTP.ResponseText)
HttpRequest code works just for second compressor too:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(URL);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = data.Length;
StreamWriter requestWriter = new StreamWriter(request.GetRequestStream(), System.Text.Encoding.ASCII);
requestWriter.Write(data);
requestWriter.Close();
try
{
// get the response
WebResponse webResponse = request.GetResponse();
Stream webStream = webResponse.GetResponseStream();
StreamReader responseReader = new StreamReader(webStream);
string response = responseReader.ReadToEnd();
responseReader.Close();
Console.WriteLine(response);
}
catch (WebException we)
{
string webExceptionMessage = we.Message;
}
What I can try else to get data from first compressor in C#?
I compared the three requests in Fiddler 4 and realized that the only difference (apart from some other headers which won't affect behavior) between vbs script and both WebClient
and HttpWebRequest is that, managed API's send the Expect: 100-continue header and vbs script does not.
This can be the issue if the software running on the compressor device does not support this.
Please try the following, which tells the HttpWebRequest to not send this header:
For the HttpWebRequest, you can simply prevent sending this by:
request.ServicePoint.Expect100Continue = false;
Note: For the WebClient, the assignment requires accessing to the HttpWebRequest object used internally, but this has "protected" access modifier and can be worked around.
Before setting this value:
POST http://oguzozgul.com.tr/getVar.cgi HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Host: oguzozgul.com.tr
Content-Length: 15
Expect: 100-continue
Connection: Keep-Alive
online_pressure
After setting this value:
POST http://oguzozgul.com.tr/getVar.cgi HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Host: oguzozgul.com.tr
Content-Length: 15
Connection: Keep-Alive
online_pressure
I also want to put the vb script request here so you can see other differences as well. The Accept-Encoding and some other headers are also not sent by default:
POST http://oguzozgul.com.tr/getVar.cgi HTTP/1.1
Accept: */*
Content-Type: application/x-www-form-urlencoded
Accept-Language: tr,en-US;q=0.8,en-GB;q=0.7,en;q=0.5,zh-Hans-CN;q=0.3,zh-Hans;q=0.2
UA-CPU: AMD64
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.3; Win64; x64; Trident/7.0; .NET4.0E; .NET4.0C; Tablet PC 2.0; .NET CLR 3.5.30729; .NET CLR 2.0.50727; .NET CLR 3.0.30729; wbx 1.0.0; Zoom 3.6.0; wbxapp 1.0.0)
Host: oguzozgul.com.tr
Content-Length: 15
Connection: Keep-Alive
Pragma: no-cache
online_pressure
WebClient and HttpWebRequest still do not works with this comressor. I tried to add any headers, but no result.
I tried PowerShell and it returns empty response too:
Invoke-WebRequest -Uri http://10.0.163.51/getVar.cgi -Method POST -Body "package_discharge_pressure" -ContentType "application/x-www-form-urlencoded"
Curl not working too, it hangs without errors
curl -i -X POST -H "Content-Type:application/x-www-form-urlencoded" -d "sump_pressure" http://10.0.163.51/getVar.cgi
Javascript and VBS works good. And today I found c# code wich works too
WinHttpRequest req = new WinHttpRequest();
try {
req.Open("POST", "http://10.0.163.51/getVar.cgi", true);
req.SetRequestHeader("Content-Type", "application/x-www-form-urlencoded");
req.Send("sump_pressure");
req.WaitForResponse();
}
catch(Exception ex) {
Console.WriteLine("Error : " + ex.Message.Trim());
}
Console.WriteLine(req.ResponseText);
In visual studio need to make reference to WinHttp
Related
i want send this data using post method in c#
POST https://lyncweb.contoso.com/ucwa/oauth/v1/applications/103...740/onlineMeetings/ myOnlineMeetings HTTP/1.1
Accept: application/json
Content-Type: application/json
Authorization: Bearer cwt=AAEB...buHc
X-Ms-Origin: http://localhost
X-Requested-With: XMLHttpRequest
Referer: https://lyncweb.contoso.com/Autodiscover/XFrame/XFrame.html
Accept-Language: en-US
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.2; Trident/6.0;.NET4.0E; .NET4.0C; .NET CLR 3.5.30729; .NET CLR 2.0.50727; .NET CLR 3.0.30729; InfoPath.3)
Host: lyncweb.contoso.com
Content-Length: 185
DNT: 1
Connection: Keep-Alive
Cache-Control: no-cache
{
"attendanceAnnouncementsStatus":"Disabled",
"description":"hey guys let's do a musical!",
"subject":"holiday party",
"attendees": ["sip:Chris#contoso.com","sip:Alex#contoso.com"],
"leaders": []
}
please help me to write code in c# desktop application.
You can use Restsharp it is an lib that you can get from Nuget. very easy to use:
here is an example:
var client = new RestClient("http://example.com");
// client.Authenticator = new HttpBasicAuthenticator(username, password);
var request = new RestRequest("resource/{id}", Method.POST);
request.AddParameter("name", "value"); // adds to POST or URL querystring based on Method
request.AddUrlSegment("id", "123"); // replaces matching token in request.Resource
// easily add HTTP Headers
request.AddHeader("header", "value");
// add files to upload (works with compatible verbs)
request.AddFile(path);
// execute the request
IRestResponse response = client.Execute(request);
var content = response.Content; // raw content as string
// or automatically deserialize result
// return content type is sniffed but can be explicitly set via RestClient.AddHandler();
RestResponse<Person> response2 = client.Execute<Person>(request);
var name = response2.Data.Name;
// easy async support
client.ExecuteAsync(request, response => {
Console.WriteLine(response.Content);
});
// async with deserialization
var asyncHandle = client.ExecuteAsync<Person>(request, response => {
Console.WriteLine(response.Data.Name);
});
// abort the request on demand
asyncHandle.Abort();
I have a problem with HttpWebRequest class.
I am trying to get source code of website:
http://www.filmweb.pl/film/Igrzyska+%C5%9Bmierci%3A+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983
but I am always getting an error:
System.Net.WebException occurred
HResult=-2146233079
Message=Too many automatic redirections were attempted.
Source=System
StackTrace:
at System.Net.HttpWebRequest.GetResponse()
at ProjectName.ClassName.MethodName(String urlAddress)
InnerException:
That is my code:
Uri uri = new Uri(#"http://www.filmweb.pl/film/Igrzyska+%C5%9Bmierci%3A+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983");
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
I have used a Fiddler Web Debugger tool to compare Firefox request with my C# .NET request, but still have no answer.
Firefox:
GET http://www.filmweb.pl/film/Igrzyska+%C5%9Bmierci%3A+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983 HTTP/1.1
Host: www.filmweb.pl
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:43.0) Gecko/20100101 Firefox/43.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
HTTP/1.1 200 OK
Cache-Control: private, no-cache, no-store, max-age=0, must-revalidate, proxy-revalidate
Content-Type: text/html;charset=UTF-8
Content-Language: pl-PL
Transfer-Encoding: chunked
Date: Wed, 07 Oct 2015 13:36:31 GMT
X-Cache: HIT from blade110.non.3dart.com
X-Cache-Hits: 116
Server: Apache
C# .NET:
GET http://www.filmweb.pl/film/Igrzyska+%C5%9Bmierci:+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983 HTTP/1.1
Host: www.filmweb.pl
Connection: Keep-Alive
HTTP/1.1 301 Moved Permanently
Cache-Control: private, no-cache, no-store, max-age=0, must-revalidate, proxy-revalidate
Content-Type: text/html;charset=UTF-8
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Content-Language: pl-PL
Location: /film/Igrzyska+%C5%9Bmierci%3A+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983
Content-Length: 0
Accept-Ranges: bytes
Date: Wed, 07 Oct 2015 13:34:51 GMT
X-Cache: MISS from blade712.non.3dart.com
Server: Apache
I have read other posts and update my code by different things, eg.
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
request.TransferEncoding = "gzip, deflate";
request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:43.0) Gecko/20100101 Firefox/43.0";
request.Referer = "http://www.filmweb.pl/film/Igrzyska+%C5%9Bmierci%3A+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983";
request.KeepAlive = true;
request.AllowAutoRedirect = true;
request.MaximumAutomaticRedirections = 250;
request.Proxy = null;
request.UseDefaultCredentials = true;
CookieContainer cookieContainer = new CookieContainer();
request.CookieContainer = cookieContainer;
but nothing works :-/
Can anybody help me with this problem?
You need to have the initial cookies when the website load before you fetch a deep-link.
The following code works for me:
// cookies
CookieContainer cookieContainer = new CookieContainer();
// make one call to the root of the website
// to get the cookies set
Uri uri = new Uri(#"http://www.filmweb.pl");
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.CookieContainer = cookieContainer;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
using(var s = response.GetResponseStream())
{
using(var sr = new StreamReader(s))
{
// linqpad
sr.ReadToEnd().Dump(); // to check for errors
}
}
// we have cookies now
// do the deep link fetch
uri = new Uri(#"http://www.filmweb.pl/film/Igrzyska+%C5%9Bmierci%3A+Kosog%C5%82os.+Cz%C4%99%C5%9B%C4%87+1-2014-626983");
request = (HttpWebRequest)WebRequest.Create(uri);
request.CookieContainer = cookieContainer;
response = (HttpWebResponse)request.GetResponse();
//store the result
using(var f = File.Create("C:\\temp\\pl.txt"))
{
response.GetResponseStream().CopyTo(f);
}
Make sure that if you scrape a website that you adhere to their license and usage policies. Don't do anything that goes beyond fair use or against any copy-righted materials.
I am trying to upload a file via a POST to a REST API in a c# winform.
If I run the following command with curl the file is uploaded successfully:
curl.exe -H "Content-type: application/octet-stream" -X POST http://myapiurl --data-binary #C:\test.docx
I have tried using WebClient in my WinForm:
using (var client = new WebClient())
{
client.Headers.Add("Content-Type", "application/octet-stream");
byte[] result = client.UploadFile(url, file);
string responseAsString = Encoding.Default.GetString(result);
tb_result.Text += responseAsString;
}
But I just get a (500) Internal Server.
Checking this with fiddler the following headers are added with CURL:
POST http://myapiurl HTTP/1.1
User-Agent: curl/7.33.0
Host: 10.52.130.121:90
Accept: */*
Connection: Keep-Alive
Content-type: application/octet-stream
Content-Length: 13343
Expect: 100-continue
But checking my WebClient method shows the following:
POST http://myapiurl HTTP/1.1
Accept: */*
Content-Type: multipart/form-data; boundary=---------------------8d220bbd95f8b18
Host: 10.52.130.121:90
Content-Length: 13536
Expect: 100-continue
Connection: Keep-Alive
How can I simulate the CURL command above from my app?
How can I simulate the CURL command above from my app?
Use HttpWebRequest. It offers you more flexibility. As follows:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://myapiurl");
request.Method = "POST";
request.UserAgent = "curl/7.33.0";
request.Host = "10.52.130.121:90";
request.Accept = "Accept=*/*";
request.Connection = "Keep-Alive";
request.ContentType = "application/octet-stream";
request.ContentLength = 13343;
request.Expect = "100-continue";
I'm trying to automate configuring a wireless router's SSID and Password via c# webclient. The router has no API that I know of. It's an unbranded chinese router. The web config seems to be the only option for configuration. It uses http-basic-authentication (you browse to the IP address of the router and get a generic dialog asking for username and password).
I've used Wireshark to get the headers and form fields that the http-post requests use when I manually update the SSID and Password (two separate forms). I then attempted to use webclient to emulate those post requests.
Here is a snippet of code that I am using to attempt to save a new SSID (NameValueCollection is defined elsewhere):
private const string FORM_SSID = "http://192.168.1.2/formWlanSetup.htm";
private const string REF_SSID = "http://192.168.1.2/formRedirect.htm?redirect-url=wlbasic.htm&wlan_id=0";
private NameValueCollection mFields = HttpUtility.ParseQueryString(string.Empty, Encoding.ASCII);
public string SaveConfigResponse()
{
try
{
using (WebClient wc = new WebClient())
{
wc.Headers[HttpRequestHeader.Accept] = "text/html, application/xhtml+xml, */*";
wc.Headers[HttpRequestHeader.Referer] = REF_SSID;
wc.Headers[HttpRequestHeader.AcceptLanguage] = "en-US";
wc.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko";
wc.Headers[HttpRequestHeader.ContentType] = "application/x-www-form-urlencoded";
wc.Headers[HttpRequestHeader.AcceptEncoding] = "gzip, deflate";
wc.Headers[HttpRequestHeader.Host] = "192.168.1.2";
wc.Headers[HttpRequestHeader.Connection] = "Keep-Alive";
wc.Headers[HttpRequestHeader.ContentLength] = Encoding.ASCII.GetBytes(mFields.ToString()).Length.ToString();
wc.Headers[HttpRequestHeader.CacheControl] = "no-cache";
string credentials = Convert.ToBase64String(Encoding.ASCII.GetBytes(config_user + ":" + config_pass));
wc.Headers[HttpRequestHeader.Authorization] = string.Format("Basic {0}", credentials);
//wc.Credentials = new NetworkCredential("admin", "admin");
return Encoding.ASCII.GetString(wc.UploadValues(FORM_SSID, "POST", mFields));
}
}
catch (Exception ex)
{
return ex.Message;
}
}
This results in an http-status-code-401 not authorized response. Is what I'm trying to do just impossible?
UPDATE
Here are the HTTP headers of both the browser post/response and the WebClient post/response. Again, I tried to match what I saw the browser posting as well as I could with my WebClient post.
Browser:
POST /formWlanSetup.htm HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
Referer: http://192.168.1.2/formRedirect.htm?redirect-url=wlbasic.htm&wlan_id=0
Accept-Language: en-US
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip, deflate
Host: 192.168.1.2
Content-Length: 524
Connection: Keep-Alive
Cache-Control: no-cache
Authorization: Basic YWRtaW46YWRtaW4=
HTTP/1.1 302 Found
Location: wlbasic.htm
Content-Length: 183
Date: Thu, 23 Oct 2014 18:18:27 GMT
Server: eCos Embedded Web Server
Connection: close
Content-Type: text/html
Transfer-Encoding: chunked
Cache-Control: no-cache
WebClient:
POST /formWlanSetup.htm HTTP/1.1
Accept-Language: en-US
Accept-Encoding: gzip, deflate
Cache-Control: no-cache
Authorization: Basic YWRtaW46YWRtaW4=
Accept: text/html, application/xhtml+xml, */*
Content-Type: application/x-www-form-urlencoded
Referer: http://192.168.1.2/formRedirect.htm?redirect-url=wlbasic.htm&wlan_id=0
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Host: 192.168.1.2
Content-Length: 524
Connection: Keep-Alive
HTTP/1.1 401 Not Authorized
WWW-Authenticate: Basic realm="AP"
Date: Thu, 23 Oct 2014 18:18:41 GMT
Server: eCos Embedded Web Server
Connection: close
Content-Type: text/html
Transfer-Encoding: chunked
Cache-Control: no-cache
Again, that was all gleaned from Wireshark. I'm not very familiar with Wireshark, but I was able to get this far. If I knew how to properly extract the raw packet data and pastebin it, I would.
Important New Observations
The Wireshark captures of the post packets from both Browser and WebClient obviously differ in the order of the headers. I don't know how significant that might or might not be, though, as the data for each header is clearly the same.
One stark difference between the packets that I noticed is that Wireshark reports the Browser packet to be significantly larger than the WebClient packet. Looking at the itemized view, I couldn't find any obvious differences. I assume posting raw data for comparison would reveal a lot, but again, I don't really know how to do that.
I had a bewildering revelation. Despite the response clearly stating '(401) Unauthorized', the post is in fact being accepted by the router! Driving in to the router's web config after my WebClient post shows that the settings were accepted and saved.
That last one is a biggie. I find myself in a situation where I can get my config to save with a WebClient post, but I have to ignore a 401 response in order to do so. Obviously, this is far from ideal. So close, yet so far!
FINAL UPDATE (RESOLUTION)
I've solved the issue of failing basic authentication, though not with WebClient. I used the suggestion from #caesay and went with HttpWebRequest (together with WebResponse). My form posts result in redirects, so I had to allow for that.
This is essentially what I went with:
private bool ConfigureRouter()
{
bool passed = false;
string response = "";
HttpWebRequest WEBREQ = null;
WebResponse WEBRESP = null;
// Attempt to POST form to router that saves a new SSID.
try
{
var uri = new Uri(FORM_SSID); // Create URI from URL string.
WEBREQ = HttpWebRequest.Create(uri) as HttpWebRequest;
// If POST will result in redirects, you won't see an "OK"
// response if you don't allow those redirects
WEBREQ.AllowAutoRedirect = true;
// Basic authentication will first send the request without
// creds. This is protocol standard.
// When the server replies with 401, the HttpWebRequest will
// automatically send the request again with the creds when
// when PreAuthenticate is set.
WEBREQ.PreAuthenticate = true;
WEBREQ.AuthenticationLevel = System.Net.Security.AuthenticationLevel.MutualAuthRequested;
// Mimic all headers known to satisfy the request
// as discovered with a tool like Wireshark or Fiddler
// when the form was submitted from a browser.
WEBREQ.Method = "POST";
WEBREQ.Accept = "text/html, application/xhtml+xml, */*";
WEBREQ.Headers.Add("Accept-Language", "en-US"); // No AcceptLanguage property built-in to HttpWebRequest
WEBREQ.UserAgent = USER_AGENT;
WEBREQ.Referer = REF_SSID;
WEBREQ.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
WEBREQ.KeepAlive = true;
WEBREQ.Headers.Add("Pragma", "no-cache"); // No Pragma property built-in to HttpWebRequest
// Use a cached credential so that the creds are properly
// submitted with subsequent redirect requests.
CredentialCache creds = new CredentialCache();
creds.Add(uri, "Basic", new NetworkCredential(config_user, config_pass));
WEBREQ.Credentials = creds;
// Submit the form.
using (Stream stream = WEBREQ.GetRequestStream())
{
SSID ssid = new SSID(ssid_scanned); // Gets predefined form fields with new SSID inserted (NameValueCollection PostData)
stream.Write(ssid.PostData, 0, ssid.PostData.Length);
}
// Get the response from the final redirect.
WEBRESP = WEBREQ.GetResponse();
response = ((HttpWebResponse)WEBRESP).StatusCode.ToString();
if (response == "OK")
{
StatusUpdate("STATUS: SSID save was successful.");
passed = true;
}
else
{
StatusUpdate("FAILED: SSID save was unsuccessful.");
passed = false;
}
WEBRESP.Close();
}
catch (Exception ex)
{
StatusUpdate("ERROR: " + ex.Message);
return false;
}
return passed;
}
Is what I'm trying to do just impossible?
No, its not impossible. I have had many headaches with web scraping like this over the years because some web servers are picky, and your router interface is likely a custom web server implementation that isnt as forgiving as apache or iis.
I would do a wireshark capture and get the raw packet data that chrome sends (w/ payload etc), and then do the same capture for your application. Make sure the packets are as similar as you can get them. If you still have issues, post the packet captures to pastebin or something so we can have a look.
EDIT::
Instead of using the limited WebClient API, try using some lower level items, I wonder if the following code will work for you:
var uri = new Uri("http://192.168.1.2/formWlanSetup.htm");
var cookies = new CookieContainer();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.CookieContainer = cookies;
request.ServicePoint.Expect100Continue = false;
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko";
request.Referer = "http://192.168.1.2/formRedirect.htm?redirect-url=wlbasic.htm&wlan_id=0";
request.Credentials = new NetworkCredential(config_user, config_pass);
request.PreAuthenticate = true;
var response = request.GetResponse();
var reader = new StreamReader(response.GetResponseStream());
string htmlResponse = reader.ReadToEnd();
I'm currently writing an API that gets data from a Point of Sale System's web interface. So far, I haven't had any problems logging in and generating reports to get data from until this situation.
In general, I can use the follow method to return an HttpWebRequest object that does the trick for most request to the web server.
private HttpWebRequest DefaultRequestObject(string path)
{
var request = (HttpWebRequest)WebRequest.Create(_baseUrl + path);
request.Method = "GET";
request.Host = _baseUrl.Substring(8); // Cut off the https://
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:30.0) Gecko/20100101 Firefox/30.0";
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
request.Headers.Add("Accept-Language", "en-US,en;q=0.5");
request.Headers.Add("Accept-Encoding", "gzip, deflate");
request.SendChunked = false;
request.AllowAutoRedirect = true;
request.ProtocolVersion = HttpVersion.Version11;
var sp = request.ServicePoint;
sp.Expect100Continue = false;
var prop = sp.GetType().GetProperty("HttpBehaviour", BindingFlags.Instance | BindingFlags.NonPublic);
prop.SetValue(sp, (byte)0, null);
request.CookieContainer = _cookieJar;
if (!String.IsNullOrEmpty(_cookieString))
request.Headers.Add(HttpRequestHeader.Cookie, _cookieString);
return request;
}
When I send a GET request, I use the following method:
public MgrngResponse GetContent(string serverPath, Dictionary<string, string> args)
{
if (!serverPath.StartsWith("/"))
serverPath = "/~pos/mgrng/" + serverPath;
var requestString = serverPath + "?" + HttpDataFormat(args);
var request = DefaultRequestObject(requestString);
try
{
var response = (HttpWebResponse)request.GetResponse();
var mgrngResponse = new MgrngResponse(response);
if (!String.IsNullOrEmpty(mgrngResponse.HttpResponse.GetResponseHeader("Set-Cookie")))
SaveMgrngResponseCookies(mgrngResponse);
_sessionExpiration = DateTime.Now.AddMinutes(15);
UpdateStatus(mgrngResponse.Content);
return mgrngResponse;
}
catch (WebException webException)
{
using (WebResponse response = webException.Response)
{
var httpResponse = (HttpWebResponse)response;
Console.WriteLine("Error code: {0}", httpResponse.StatusCode);
using (Stream data = response.GetResponseStream())
using (var reader = new StreamReader(data))
{
string text = reader.ReadToEnd();
Console.WriteLine(text);
}
}
var eventArgs = new SessionUpdateEventArgs(SessionStatus.ConnectionError, "Unable to GET data");
RaiseStatusChangeEvent(eventArgs);
return null;
}
}
This works well for all of the pages I've attempted so far, but now I'm running into a new problem where when I try to get the response for a particular page, the method throw a WebException with a 500 Internal Server Error.
I used Fiddler to match a browser's request exactly with mine (with the exception of the order of the headers and the cookie values obviously) but I'm still getting the Internal Server Error.
Below are the Raw Requests from Fiddler. The first one is from Firefox and the second one is from my program.
GET https://location.posprovider.com/~pos/mgrng/Report.php?boc_brand=7&csv_delimeter=csv_delimeter_comma&format=text&format1=csv&format1=txt&format1=pdf&format1=html HTTP/1.1
Host: location.posprovider.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:30.0) Gecko/20100101 Firefox/30.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: USER=af182fda473354eb3199522726ca61c9d5516c95f165cffd63b9522726ca61c9dc714cb52a46278e4399720706ea41e9dc714cb52a46278e4399720706ea41e9dc714cb52a46278e4399720706ea41e9dc714cb52a46278e4399720706ea41e9fc516c950a6607ae63b9522726ca61c9; PHPSESSID=9d7f54f9a1769a3e0572745fe0db3d97
Connection: keep-alive
GET https://location.posprovider.com/~pos/mgrng/Report.php?boc_brand=7&csv_delimeter=csv_delimeter_comma&format=text&format1=csv&format1=txt&format1=pdf&format1=html HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:30.0) Gecko/20100101 Firefox/30.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Host: location.posprovider.com
Cookie: PHPSESSID=16ad21c9d69fe85b3d185ab284f8620b; USER=30dca66f355e0ba89b6eda3c3e822ea24a95e5209e0f90bec94eda3c3e822ea243b5c500582b78cde96efa1c1ea20e8243b5c500582b78cde96efa1c1ea20e8243b5c500582b78cde96efa1c1ea20e8243b5c500582b78cde96efa1c1ea20e826395e520780b58edc94eda3c3e822ea2
Connection: Keep-Alive
I even tried logging into the web interface and then copy and pasting my generated request string into the browser and I got the desired data.
Any ideas?
Pull your programs request into Fiddlers composer and gradually eliminate the remaining differences. What do you find doing that?
Great advice! Apparently there's something wrong with my cookie line.
I copy and pasted the cookie line from the browser's version into mine
and it worked. Now I just need to resolve why my cookies aren't
correct...
I agree with that diagnosis. Who knows what's going on inside of the server. Probably some fragile code that expect a very exact cookie string format.