We Keep Coding

c# Reading source code with HttpWebRequest

My code is as below. But I am getting error. How can I read the url source code in the example.
Error Message 'The remote server returned an error: (403) Forbidden'
CookieContainer cookierJar = new CookieContainer();
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create("https://www.hepsiburada.com/dyson-v11-absolute-extra-kablosuz-supurge-dyson-turkiye-garantili-p-HBV00000W4W6U");
webRequest.AllowAutoRedirect = true;
webRequest.CachePolicy = new System.Net.Cache.RequestCachePolicy(System.Net.Cache.RequestCacheLevel.NoCacheNoStore);
webRequest.CookieContainer = cookierJar;
webRequest.UserAgent = "My Thirsty Browser";
int statusCode = 0;
string sourceCode = string.Empty;
try
{
HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse();
statusCode = (int)webResponse.StatusCode;
StreamReader readContent = new StreamReader(webResponse.GetResponseStream());
sourceCode = readContent.ReadToEnd();
webResponse.Close();
webResponse = null;
}
catch (WebException xc)
{
if (xc.Response is HttpWebResponse)
{
HttpWebResponse rs = xc.Response as HttpWebResponse;
StreamReader readContent = new StreamReader(rs.GetResponseStream());
if (readContent != null)
{
sourceCode = readContent.ReadToEnd();
}
statusCode = (int)rs.StatusCode;
}
else
{
statusCode = (int)xc.Status;
sourceCode = xc.Message;
}
}
catch (Exception xc)
{
sourceCode = xc.Message;
}

I get a 403 in IE, I guess you need to be logged in to retrieve the resource. Your browser may have the credentials cached but your app isn't designed to log you in. Or are you logged in to Google in your browser - try logging out and see if you still have access....

C# - WebRequest.GetResponce(): The request was aborted: The connection was closed unexpectedly

i'm having a problem with this simple code which sends a request using provided url and reads html from responce. Looks like it's something with encoding of cyrillic symbols after ?q=, but i can't see why (url was actually obtained from browser address bar, not generated or anything else).
url =
"http://www.avito.ru/nizhniy_novgorod/kvartiry/sdam/na_dlitelnyy_srok?q=%D0%93%D0%B5%D0%BD%D0%B5%D1%80%D0%B0%D0%BB%D0%B0+%D0%98%D0%B2%D0%BB%D0%B8%D0%B5%D0%B2%D0%B0+10%D0%BA1";
string html = "";
try
{
Uri uri = new Uri(url);
WebRequest request = WebRequest.Create(uri);
request.Timeout = 100000;
using (WebResponse responce = request.GetResponse())
{
Stream stream = responce.GetResponseStream();
StreamReader reader = new StreamReader(stream);
html = reader.ReadToEnd();
}
}
catch (Exception e)
{
System.Console.WriteLine(e.Message);
}
Error occures in GetResponce() method. The message is:
The request was aborted: The connection was closed unexpectedly.

You should cast your request and response to HttpWebRequest and HttpWebResponse, respectively.
var url = "http://www.avito.ru/nizhniy_novgorod/kvartiry/sdam/na_dlitelnyy_srok?q=%D0%93%D0%B5%D0%BD%D0%B5%D1%80%D0%B0%D0%BB%D0%B0+%D0%98%D0%B2%D0%BB%D0%B8%D0%B5%D0%B2%D0%B0+10%D0%BA1";
string html = "";
try
{
Uri uri = new Uri(uri);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Timeout = 100000;
using (HttpWebResponse responce = (HttpWebResponse)request.GetResponse())
{
Stream stream = responce.GetResponseStream();
StreamReader reader = new StreamReader(stream);
html = reader.ReadToEnd();
Console.WriteLine(html);
}
}
catch (Exception e)
{
System.Console.WriteLine(e.Message);
}
Also, It seems that the url http://www.avito.ru/nizhniy_novgorod/kvartiry/sdam/na_dlitelnyy_srok?q=%D0%93%D0%B5%D0%BD%D0%B5%D1%80%D0%B0%D0%BB%D0%B0+%D0%98%D0%B2%D0%BB%D0%B8%D0%B5%D0%B2%D0%B0+10%D0%BA1 is invalid.
Using Fiddler, the url returns a 404 error.

"An existing connection was forcibly closed by the remote host c#"

I am getting the exception error for this code, I have created this form in windows application, the code is to login to the site automatically and downloading the file, and store that in a folder, while debugging I am getting the exception error as “The underlying connection was closed: An unexpected error occurred on a send." and the inner exception is “an existing connection was forcibly closed by the remote host c#”
private void Form1_Load(object sender, EventArgs e)
{
try
{
string formUrl = "https://secure.gmail.com/account/doLogin.html?icmp=HO_LOGIN_BANNER"; // NOTE: This is the URL the form POSTs to, not the URL of the form (you can find this in the "action" attribute of the HTML's form tag
string formParams = string.Format("username={0}&password={1}", "username", "password");
string cookieHeader;
WebRequest req = WebRequest.Create(formUrl);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
string pageSource;
string getUrl = "https://secure.gmail.com/account/summary.html";
WebRequest getRequest = WebRequest.Create(getUrl);
getRequest.Headers.Add("Cookie", cookieHeader);
WebResponse getResponse = getRequest.GetResponse();
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
WebClient Client = new WebClient();
Client.DownloadFile("https://secure.gmail.com.com/account/callActivity/print.csv?lineNumber=16145838755&startDate=MNTH2014-04-24&fragment=true", #"C:\Downloads");
}
catch(Exception ex)
{
throw ex;
}
}
}
}

Text API intergration issue - Error 405 page not found

I am using Innovative Text SMS API gateway which is through me error.
Documentation is available at link http://www.innovativetxt.com/services/sms_api_gateway.htm, the error is happening on my local machine, but works fine at the production server.
The remote server returned an error: (405) Method Not Allowed.
at this line of code:
HttpWebResponse objResponse = (HttpWebResponse)objRequest.GetResponse();
try
{
// InnovativeTXT POST URL
string url = "http://innovativetxt.com/cp/api";
// XML-formatted data
string toSender = "4477878585244";
string fromSender = "Test";
string textMessage = "Thanks for Choosing Innovative Text Messaging Solution.";
string fields = "?to=" + toSender + "&from=" + fromSender + "&text=" + textMessage + "&api_key=xxx&api_secret=xxxxx";
url = url + fields;
// web request start
Uri uri = new Uri(url);
string data = "field-keywords=ASP.NET 2.0";
if (uri.Scheme == Uri.UriSchemeHttp)
{
// create a request on behalf of uri
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(uri);
// setting parameter for the request
request.Method = WebRequestMethods.Http.Post;
request.ContentLength = data.Length;
request.ContentType = "application/x-www-form-urlencoded";
// a stream writer for the request
StreamWriter writer = new StreamWriter(request.GetRequestStream());
// write down the date
writer.Write(data);
//close the stream writer
writer.Close();
// getting response from the request
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
// Get the stream associated with the response.
Stream receiveStream = response.GetResponseStream();
// Pipes the stream to a higher level stream reader with the required encoding format.
StreamReader readStream = new StreamReader(receiveStream, System.Text.Encoding.UTF8);
// to write a http response from the characters
Response.Write(readStream.ReadToEnd());
// close the response
response.Close();
// close the reader
readStream.Close();
}
}
catch (Exception exp)
{
// catch for unhelded exception
Response.Write(exp.Message);
}

This issue relates to IIS Handler mappings.
You should specify the file name at the end of string URL:
string url = "http://innovativetxt.com/cp/api/somepage.aspx";
You can use default pages like, default.aspx or index.php etc...

Twitter Streaming filter API: getting IOE exception on a regular basis

I am using twitter streaming API (C#) and getting IOE exception on a regular basis (every 12 hrs or something like that) while Stream reader is trying to read JSON object(readLine). Please look into the exception & the code below. Any help would be highly appreciated. Thank you in advance.
Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection ?failed because connected host has failed to respond.
string userName = "XXXX";
string password = "XXXX";
string url = #"https://stream.twitter.com/1.1/statuses/filter.json";
string parameters = "follow=" + userIds +"&track=" + keywords;
// Request and Response
HttpWebRequest webRequest = null;
HttpWebResponse webResponse = null;
StreamReader streamReader = null;
byte[] byteArray;
// Establishing connection with twitter filter.json
byteArray = Encoding.UTF8.GetBytes(parameters);
webRequest = (HttpWebRequest) WebRequest.Create(url);
webRequest.Credentials = new NetworkCredential(userName, password);
webRequest.Timeout = 100000;
Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
webRequest.Method = "POST";
webRequest.ContentType = "application/x-www-form-urlencoded";
byte[] postData = encode.GetBytes(parameters);
webRequest.ContentLength = postData.Length;
Stream twitterPost = webRequest.GetRequestStream();
twitterPost.Write(postData, 0, postData.Length);
twitterPost.Close();
webResponse = (HttpWebResponse)webRequest.GetResponse();
streamReader = new StreamReader(webResponse.GetResponseStream(), encode);
Stopwatch sw = new Stopwatch();
sw.Start();
while (sw.Elapsed < TimeSpan.FromDays(365))
{
String json = streamReader.ReadLine(); // [The program stop here. throws exception]
if (json == "")
{
continue;
}
Packet dataPacket = new Packet() { json = json, id = id++ };
ThreadPool.QueueUserWorkItem(new WaitCallback(Execute), dataPacket);
}
// Aborting the request and closing the stream
webRequest.Abort();
sw.Stop();
streamReader.Close();
streamReader = null;
webResponse.Close();
webResponse = null;

Sounds like your internet connection dropped or your connection to your router.
In my country most ISP's refresh your ip address daily at midnight, that creates a connection drop for a few seconds, perhaps your ISP is doing something similar?