Dotnet webclient timesout but browser works file for json webservice - c#

I am trying to get the result of the following json webservice https://mtgox.com/code/data/getDepth.php into a string using the following code.
using (WebClient client = new WebClient())
{
string data = client.DownloadString("https://mtgox.com/code/data/getDepth.php");
}
but it always returns a timeout exception and no data. I plan to use fastjson to turn the response into objects and expected that to be that hard part not the returning of the content of the page.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://mtgox.com/code/data/getDepth.php");
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (StreamReader sr = new StreamReader(response.GetResponseStream()))
{
string data = sr.ReadToEnd();
}
}
Also resulted in the same error. Can anyone point out what i am doing wrong?

Hmm, strage, this works great for me:
class Program
{
static void Main()
{
using (var client = new WebClient())
{
client.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0) Gecko/20100101 Firefox/4.0";
var result = client.DownloadString("https://mtgox.com/code/data/getDepth.php");
Console.WriteLine(result);
}
}
}
Notice that I am specifying a User Agent HTTP header as it seems that the site is expecting it.

I had similar issue before. request.KeepAlive = false solved my problem. Try this:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://mtgox.com/code/data/getDepth.php");
request.KeepAlive = false;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (StreamReader sr = new StreamReader(response.GetResponseStream()))
{
string data = sr.ReadToEnd();
}
}

Related

Unable to call onesignal api within office but working from external wifi

I am using C# to call OneSignal Create Notifications API. Its working from my home but when i try to send it from my office, it gives a 400 BAD REQUEST.
The error that i get is
The error that comes is Please include a case-sensitive header of
Authorization: Basic <YOUR-REST-API-KEY-HERE> with a valid REST API key."],
"reference":["https://documentation.onesignal.com/docs/accounts-and-keys#section-keys-ids"]
Also, in office network, when i send it from OneSignal Dashboard it works.
Is it a firewall issue?
Tested with POSTMAN and got the same results. POSTMAN working outside office network and giving 400 Bad Request inside office network.
using System.IO;
using System.Net;
using System.Text;
var request = WebRequest.Create("https://onesignal.com/api/v1/notifications") as HttpWebRequest;
request.KeepAlive = true;
request.Method = "POST";
request.ContentType = "application/json; charset=utf-8";
request.Headers.Add("authorization", "Basic NGEwMGZmMjItY2NkNy0xMWUzLTk5ZDUtMDAwYzI5NDBlNjxx");
var serializer = new JavaScriptSerializer();
var obj = new { app_id = "5eb5a37e-b458-11e3-ac11-000c2940e6xx",
contents = new { en = "English Message" },
include_player_ids = new string[] {"all"} };
var param = serializer.Serialize(obj);
byte[] byteArray = Encoding.UTF8.GetBytes(param);
string responseContent = null;
try {
using (var writer = request.GetRequestStream()) {
writer.Write(byteArray, 0, byteArray.Length);
}
using (var response = request.GetResponse() as HttpWebResponse) {
using (var reader = new StreamReader(response.GetResponseStream())) {
responseContent = reader.ReadToEnd();
}
}
}
catch (WebException ex) {
System.Diagnostics.Debug.WriteLine(ex.Message);
System.Diagnostics.Debug.WriteLine(new StreamReader(ex.Response.GetResponseStream()).ReadToEnd());
}
System.Diagnostics.Debug.WriteLine(responseContent);
try following
System.Net.CredentialCache credentialCache = new System.Net.CredentialCache();
credentialCache.Add(
new System.Uri("http://www.yoururl.com/"),
"Basic",
new System.Net.NetworkCredential("username", "password")
);
...
...
httpWebRequest.Credentials = credentialCache;
but i think you should use test RESTFUL tools to check what parameters can return correct value at all first.
The issue is solved by adding the onesignal host to our firewall as a trusted site.

Integration of Rightmove Real Time Data Feed (RTDF) asp.net

i am integrating rightmove real time data feed (rtdf) in my property site for listing my properties on rightmove website. i am using asp.net web api to post data on rightmove listing.
they have provide me with these SSL Files [.p12,.pem,.jks]. i have imported .p12 certificate in my local machine personal store and sending it in my http request
to rightmove test api link provide by rightmove.
i am getting the following error from server.
The remote server returned an error: 403 forbidden.
i checked my certificate loaded successfully in the request, below is my code
public static string PostData(string data, string url)
{
String result = "";
try
{
byte[] bytebuffer = Encoding.UTF8.GetBytes(data);
HttpWebRequest objRequest = (HttpWebRequest)WebRequest.Create(url);
objRequest.Method = "POST";
objRequest.ContentLength = bytebuffer.Length;
objRequest.ContentType = "application/json";
objRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:44.0) Gecko/20100101 Firefox/44.0";
objRequest.PreAuthenticate = true;
objRequest.Accept = "application/json";
objRequest.ClientCertificates.Add(CertificateHelper.GetRightmoveApiX509Certificate());
using (Stream stream = objRequest.GetRequestStream())
{
stream.Write(bytebuffer, 0, bytebuffer.Length);
stream.Close();
}
HttpWebResponse objResponse = (HttpWebResponse)objRequest.GetResponse();
using (StreamReader streamReader = new StreamReader(objResponse.GetResponseStream()))
{
result = streamReader.ReadToEnd();
// Close and clean up the StreamReader
streamReader.Close();
}
}
catch (Exception e)
{
result = "Exception: " + e.Message;
}
return result;
}
help me to get rid from 403 forbidden error.
Use the following.
I have tested it and it's working fine in my case.
// Grab Certificate
X509Certificate2 cert2 = new X509Certificate2(
AppDomain.CurrentDomain.BaseDirectory + "CertificateName.p12",
CertificatePasswordHere,
X509KeyStorageFlags.MachineKeySet);
var httpWebRequest = (HttpWebRequest)WebRequest.Create("https://adfapi.adftest.rightmove.com/v1/property/sendpropertydetails");
httpWebRequest.ContentType = "application/json";
httpWebRequest.Method = "POST";
httpWebRequest.ClientCertificates.Clear();
httpWebRequest.ClientCertificates.Add(cert2);
using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
{
streamWriter.Write(data);
streamWriter.Flush();
streamWriter.Close();
}
var httpResponse = (HttpWebResponse)httpWebRequest.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
var result = streamReader.ReadToEnd();
}

C# Httpwebrequest Log In System

The following code should log user in, but it does not work. The code uses 9gag as an example, but the general idea should work elsewhere too. The user does not get access his/hers profile.
What is wrong with the code?
using System;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
namespace ttp_B
{
internal class Program
{
private static void Main(string[] args)
{
bool flag = false;
//string word_to_stop = "profile";
string urltosite = "https://9gag.com/login"; // site that i'm trying to log in
string emailaddress = "";
string password = "";
var coo = new System.Net.CookieContainer();
try
{
var request = System.Net.WebRequest.Create(urltosite) as System.Net.HttpWebRequest;
request.CookieContainer = coo;
request.Method = "POST";
request.Proxy = new WebProxy("127.0.0.1", 8888); // fiddler
//some extra headers
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:43.0) Gecko/20100101 Firefox/43.0";
using (var stream = request.GetRequestStream())
{
byte[] buffer =
Encoding.UTF8.GetBytes(
string.Format(
"csrftoken=&next=http%3A%2F%2F9gag.com%2F&location=1&username={0}&password={1}", emailaddress, password));
//this text of request is correct I guess. Got it from fiddler btw.
stream.Write(buffer, 0, buffer.Length);
stream.Close();
}
using (var response = request.GetResponse() as HttpWebResponse)
{
coo.Add(response.Cookies); // adding cookies, just to make this working properly
using (var sr = new System.IO.StreamReader(response.GetResponseStream()))
{
string http_code = sr.ReadToEnd(); // gettin' that new html document
//flag = (http_code.Contains(word_to_stop)); // looking for word that I'm sure exist after succesfull loggin' in
//if(flag == true)
//{
// console.writeline("Works");
//}
}
}
}
catch (WebException e)
{
Console.Write(e.ToString());
}
}
}
}
As far as I can see, the request stream isn't closed before you go for the response.
simplified it should look like this:
var stream = request.GetRequestStream();
tStream(buffer, 0, buffer.Length);
//close the stream
tStream.Close();
//go for the response
request.GetResponse();

HttpWebRequest authentication issue(windows phone 8)

I'm stuck on authentication to web site with webrequest on windows phone 8.
I have the following code:
PostMessage = string.Concat("job=LOGIN&password=&giftCode=&language=ua&login=%D0%9D%D0%BE%D0%BC%D0%B5%D1%80+%D0%BA%D0%B0%D1%80%D1%82%D0%BA%D0%B8&actn=&custId=&dateFrom=&dateTo=&showGift=&type=&card=&trnId=&catId=&subCatId=&awrId=&ordId=&qstId=&acqId=&", "crdNo=", cardNumber, "&PIN=", pin);
this.postDataBytes = Encoding.UTF8.GetBytes(PostMessage);
public void Load()
{
HttpWebRequest loginRequest = HttpWebRequest.CreateHttp(loginUrl);
loginRequest.ContentLength = this.postDataBytes.Length;
loginRequest.Method = "POST";
loginRequest.Accept = #"text/html, application/xhtml+xml, */*";
loginRequest.ContentType = "application/x-www-form-urlencoded";
loginRequest.Headers["Host"] = "*host*";
loginRequest.Headers["Referer"] = "*refer*";
loginRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko";
loginRequest.CookieContainer = new CookieContainer();
loginRequest.BeginGetRequestStream(OnBeginGetRequestStream, loginRequest);
}
private void OnBeginGetRequestStream(IAsyncResult result)
{
HttpWebRequest loginRequest = result.AsyncState as HttpWebRequest;
using (Stream stream = loginRequest.EndGetRequestStream(result))
{
stream.Write(this.postDataBytes, 0, this.postDataBytes.Length);
}
loginRequest.BeginGetResponse(OnAuthenticated, loginRequest);
}
private void OnAuthenticated(IAsyncResult result)
{
HttpWebRequest request = result.AsyncState as HttpWebRequest;
using (WebResponse response = request.EndGetResponse(result))
{
Stream responseStream = response.GetResponseStream();
using (StreamReader reader = new StreamReader(responseStream))
{
string page = reader.ReadToEnd();
}
}
}
It works sometimes... I mean this code do successfully authentication and return correct response. But sometimes it doesn't. Just ran this code few times and at some point I got correct response.
I tried to use fiddler but he doesn't want track requests from emulator.
Maybe someone know the reason of this strange behavior?
Did you look at this answer? I had the same problem, and the reason was cached response data.This answer is about win phone 7 but works for me on win phone 8.
Httpwebrequest caching in windows phone 7

C# Downloading website into string using C# WebClient or HttpWebRequest

I am trying to download the contents of a website. However for a certain webpage the string returned contains jumbled data, containing many � characters.
Here is the code I was originally using.
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create(url);
req.Method = "GET";
req.UserAgent = "Mozilla/5.0 (Windows; U; MSIE 9.0; WIndows NT 9.0; en-US))";
string source;
using (StreamReader reader = new StreamReader(req.GetResponse().GetResponseStream()))
{
source = reader.ReadToEnd();
}
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(source);
I also tried alternate implementations with WebClient, but still the same result:
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
using (WebClient client = new WebClient())
using (var read = client.OpenRead(url))
{
doc.Load(read, true);
}
From searching I guess this might be an issue with Encoding, so I tried both the solutions posted below but still cannot get this to work.
http://blogs.msdn.com/b/feroze_daud/archive/2004/03/30/104440.aspx
http://bytes.com/topic/c-sharp/answers/653250-webclient-encoding
The offending site that I cannot seem to download is the United_States article on the english version of WikiPedia (en . wikipedia . org / wiki / United_States).
Although I have tried a number of other wikipedia articles and have not seen this issue.
Using the built-in loader in HtmlAgilityPack worked for me:
HtmlWeb web = new HtmlWeb();
HtmlDocument doc = web.Load("http://en.wikipedia.org/wiki/United_States");
string html = doc.DocumentNode.OuterHtml; // I don't see no jumbled data here
Edit:
Using a standard WebClient with your user-agent will result in a HTTP 403 - forbidden - using this instead worked for me:
using (WebClient wc = new WebClient())
{
wc.Headers.Add("user-agent", "Mozilla/5.0 (Windows; Windows NT 5.1; rv:1.9.2.4) Gecko/20100611 Firefox/3.6.4");
string html = wc.DownloadString("http://en.wikipedia.org/wiki/United_States");
HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(html);
}
Also see this SO thread: WebClient forbids opening wikipedia page?
The response is gzip encoded.
Try the following to decode the stream:
UPDATE
Based on the comment by BrokenGlass setting the following properties should solve your problem (worked for me):
req.Headers[HttpRequestHeader.AcceptEncoding] = "gzip, deflate";
req.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
Old/Manual solution:
string source;
var response = req.GetResponse();
var stream = response.GetResponseStream();
try
{
if (response.Headers.AllKeys.Contains("Content-Encoding")
&& response.Headers["Content-Encoding"].Contains("gzip"))
{
stream = new System.IO.Compression.GZipStream(stream, System.IO.Compression.CompressionMode.Decompress);
}
using (StreamReader reader = new StreamReader(stream))
{
source = reader.ReadToEnd();
}
}
finally
{
if (stream != null)
stream.Dispose();
}
This is how I usually grab a page into a string (its VB, but should translate easily):
req = Net.WebRequest.Create("http://www.cnn.com")
Dim resp As Net.HttpWebResponse = req.GetResponse()
sr = New IO.StreamReader(resp.GetResponseStream())
lcResults = sr.ReadToEnd.ToString
and haven't had the problems you are.

Categories