I've googled and searched here. Some suggest that streams were not being close, others suggested that it's a connection limit with ServicePointManager.DefaultConnectionLimit being set to 1. However, none of these seem to work.
My problem is, when i use this for the first time, it works:
using (var stream = request.GetRequestStream())
{
var data = Encoding.UTF8.GetBytes(post.ToString());
stream.Write(data, 0, data.Length);
}
When I use it a second time, it freezes. Yes, I'm disposing my stream, yes im Aborting and closing my response and requests.
Here is my entire code segment:
public string get_hash(string strUsername, string strPassword, string strUniverse)
{
// Request VAR
var request = (HttpWebRequest)WebRequest.Create("http://website.com/");
// Response VAR
var response = (HttpWebResponse)request.GetResponse();
// Cookie Var
var cookie = new CookieContainer();
ServicePointManager.DefaultConnectionLimit = 100;
request.Timeout = 10;
request = (HttpWebRequest)WebRequest.Create("http://website.com/main/login");
request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8";
request.Headers.Add("Accept-Encoding", "gzip,deflate,sdch");
request.Headers.Add("Accept-Language", "en-US,en,q=0.8");
request.Headers.Add("Cache-Control", "max-age=0");
request.ContentType = "application/x-www-form-urlencoded";
request.Host = "website.com";
request.Headers.Add("Origin", "http://website.com");
request.Referer = "http://website.com/";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.117 Safari/537.36";
request.Method = "POST";
request.CookieContainer = new CookieContainer();
request.CookieContainer.Add(cookie.GetCookies(request.RequestUri));
// SET POST DATA HERE
var post = HttpUtility.ParseQueryString(string.Empty);
post.Add("uni", strUni);
post.Add("login", strUsername);
post.Add("pass", strPassword);
using (var stream = request.GetRequestStream())
{
var data = Encoding.UTF8.GetBytes(post.ToString());
stream.Write(data, 0, data.Length);
stream.Close();
stream.Dispose();
}
response = (HttpWebResponse)request.GetResponse();
string strSSID = "Failed";
if (response.StatusCode == HttpStatusCode.OK)
{
var data = string.Empty;
using (var sReader = new StreamReader(response.GetResponseStream()))
{
data = sReader.ReadToEnd();
sReader.Close();
sReader.Dispose();
}
string strSSIDurl = response.ResponseUri.ToString();
int intSSIDurlStart = strSSIDurl.IndexOf("PHPSESSID=") + 10;
strSSID = strSSIDurl.Substring(intSSIDurlStart);
}
request.Abort();
response.Close();
response.Dispose();
return strSSID;
}
you are not disposing the response from the first request.
var request = (HttpWebRequest)WebRequest.Create("http://website.com/");
var response = (HttpWebResponse)request.GetResponse(); // Not disposed
We had similar problem and disposing the response properly helped us.
Related
Trying to POST a webrequest but not getting the actual results.. webrequest not getting submit. Previously this code was working fine in 2018. I'm not sure if they changed something at their end. I copied all the latest parameters, and other stuff from fiddler. Is there anyone please look into it as my deadline for the actual project is very tight and this code is just an integral part of the main project and was assuming it is functional already.. already spent few hours on this..
CookieContainer _cookies = new CookieContainer();
string urls = "https://ww3.freddiemac.com/loanlookup?Phone=&Email=&Contact=&lang=en&fname=TEST&lname=PERSON&strt_nbr=123&unit_num=&strt_name=FAKE STREET&strt_suffix=&addr_city=CHICAGO&addr_state=IL&addr_zip=60628&ssn=1234&Verify=Yes";
var request = (HttpWebRequest)WebRequest.Create(urls);
request.Host = "ww3.freddiemac.com";
request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36 Edg/83.0.478.61";
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9";
WebHeaderCollection myWebHeaderCollection = request.Headers;
myWebHeaderCollection.Add("Accept-Language", "en-US,en;q=0.9");
request.Headers["Accept-Encoding"] = "gzip, deflate, br";
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
request.Referer = "https://ww3.freddiemac.com/loanlookup";
var encoding = new ASCIIEncoding();
var postBytes = encoding.GetBytes(urls);
request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip | DecompressionMethods.None;
request.ContentLength = postBytes.Length;
request.CookieContainer = _cookies;
request.KeepAlive = true;
var requestStream = request.GetRequestStream();
requestStream.Write(postBytes, 0, postBytes.Length);
requestStream.Close();
var response = (HttpWebResponse)request.GetResponse();
string html;
using (var reader = new StreamReader(response.GetResponseStream()))
{
html = reader.ReadToEnd();
}
if (html.Contains("No. Our records show"))
{
_qualified = false;
}
I'm currently trying to login to a website and submit a form once logged into the website.
I've tried looking through all the input fields and adding it to the code but still no luck.
The website I'm trying to login to is http://sythe.org/login/
Here's what I currently have:
string formUrl = "http://www.sythe.org/login/";
string formParams = string.Format("login={0}&password={1}&_xfToken={2}®ister={3}&remember={4}&cookie_check={5}&redirect={6}", "", "", "", "0", "1", "1", "https://sythe.org");
string cookieHeader;
WebRequest req = WebRequest.Create(formUrl);
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
string pageSource;
using (StreamReader sr = new StreamReader(resp.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
Console.WriteLine(pageSource);
Thanks guys.
You can try this code which is generated using the C# plugin for Fiddler:
private void MakeRequests()
{
HttpWebResponse response;
if (Request_www_sythe_org(out response))
{
response.Close();
}
}
private bool Request_www_sythe_org(out HttpWebResponse response)
{
response = null;
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://www.sythe.org/login/login");
request.KeepAlive = true;
request.Headers.Set(HttpRequestHeader.CacheControl, "max-age=0");
request.Headers.Add("Origin", #"https://www.sythe.org");
request.Headers.Add("Upgrade-Insecure-Requests", #"1");
request.ContentType = "application/x-www-form-urlencoded";
request.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36";
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3";
request.Referer = "https://www.sythe.org/";
request.Headers.Set(HttpRequestHeader.AcceptEncoding, "gzip, deflate, br");
request.Headers.Set(HttpRequestHeader.AcceptLanguage, "en-GB,en-US;q=0.9,en;q=0.8");
request.Headers.Set(HttpRequestHeader.Cookie, #"__cfduid=de8826b5a5df064d6f2ccf2d0952ce52d1554285418; _ga=GA1.2.1744286790.1554285420; _gid=GA1.2.1835584013.1554285420; G_ENABLED_IDPS=google; sythe_bug_token=c0FFSCNAJio4MzJnZWEjWXlaV1NCempSdWFrUzRnUHVZUXJKOENicnkwSmZFTXROR1M3MDhUR1ovWTFIVmlvRFZaaVN2MjFHRWtaM2JhT0N1bTJHbEdGSG5VY1dUNjdmVGNZd1RoekNmTmgzaHB4T29OTTZkNytJQWh3cDdjeWlndWpidkJFY0NiRXF4b1ljSVJhN2VGVVd2eEsvK2hrak1pclhvYjRFeWRhT09UUU5seXZDUVlETDlYWUUyazZGajRub3VWV0k5aVhkVVJiWUtxcFRuaTljRElnWEVKd2o4T3ZSZlRheFEzRWRzekgxSXN4d0doMS9YRFR6L1d0OGRONUVLcDZHUEN3eVFRWEQ%3D; _gat=1; xf_session=63943f38ebba0827a4208d363cae5dcb");
request.Headers.Add("AlexaToolbar-ALX_NS_PH", #"AlexaToolbar/alx-4.0.3");
request.Method = "POST";
request.ServicePoint.Expect100Continue = false;
string body = #"login=testtest%40testtest.testtest®ister=0&password=testtesttesttest&remember=1&cookie_check=1&redirect=%2F&_xfToken=";
byte[] postBytes = System.Text.Encoding.UTF8.GetBytes(body);
request.ContentLength = postBytes.Length;
Stream stream = request.GetRequestStream();
stream.Write(postBytes, 0, postBytes.Length);
stream.Close();
response = (HttpWebResponse)request.GetResponse();
}
catch (WebException e)
{
if (e.Status == WebExceptionStatus.ProtocolError) response = (HttpWebResponse)e.Response;
else return false;
}
catch (Exception)
{
if(response != null) response.Close();
return false;
}
return true;
}
try to replace string formUrl = "http://www.sythe.org/login/"; to string formUrl = "http://www.sythe.org/login/login";
I keep getting garbage text like this
�\b\0\0\0\0\0\0�\a`I�%&/m�{J�J��t�\b�`$ؐ#������iG#
in my html variable at the end of this function. As you can see, by the commented out code, I have tried both methods of inserting Cookies.
I am very new to HTTPwebRequest / Response methods like this. But from everything I can find on the web I am setting up my method correctly. I would love some help if possible.
Also, when using Fiddler to decode my requests, not all of my cookies are getting sent. I am only sending 1 utma, 1 utmb, 1 utmc, and 1 utmz when my code runs. However, when I log into the site normally, i receive 2 utma, 1 utmb, 2 utmc, and 2 utmz.
I feel that this is the source of my connecting issues but I am not sure.
static void FillCookieJar()
{
Console.WriteLine("Filling cookie jar...\r\n");
try
{
string parameters = "SUPER LONG POST DATA found from TEXTVIEW in Fidler";
Uri target = new Uri("https://foo.bar.com/UserSignIn");
byte[] buffer = Encoding.ASCII.GetBytes(parameters);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(target);
//Cookie chipOne = new Cookie("__utma", "XXXXXXXXX.XXXXXXXXX.XXXXXXXXX.XXXXXXXXX.XXXXXXXXX.XX") { Domain = target.Host };
//Cookie chipTwo = new Cookie("__utma", "XXXXXXXXX.XXXXXXXXX.XXXXXXXXX.XXXXXXXXX.XXXXXXXXX.X") { Domain = target.Host };
//Cookie chipThree = new Cookie("__utmb", "XXXXXXXXX.X.XX.XXXXXXXXX") { Domain = target.Host };
//Cookie chipFour = new Cookie("__utmc", "XXXXXXXXX") { Domain = target.Host };
//Cookie chipFive = new Cookie("__utmc", "XXXXXXXXX") { Domain = target.Host };
//Cookie chipSix = new Cookie("__utmz", "XXXXXXXXX.XXXXXXXXX.X.X.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)") { Domain = target.Host };
//Cookie chipSeven = new Cookie("__utmz", "XXXXXXXXX.XXXXXXXXX.XX.X.utmcsr=titlesource.com|utmccn=(referral)|utmcmd=referral|utmcct=/") { Domain = target.Host };
//cookieJar.Add(chipOne);
//cookieJar.Add(chipTwo);
//cookieJar.Add(chipThree);
//cookieJar.Add(chipFour);
//cookieJar.Add(chipFive);
//cookieJar.Add(chipSix);
//cookieJar.Add(chipSeven);
request.Headers.Add("__utma", "XXXXXXXXX.XXXXXXXXX.XXXXXXXXX.XXXXXXXXX.XXXXXXXXX.XX");
request.Headers.Add("__utma", "XXXXXXXXX.XXXXXXXXX.XXXXXXXXX.XXXXXXXXX.XXXXXXXXX.X");
request.Headers.Add("__utmb", "XXXXXXXXX.X.XX.XXXXXXXXX");
request.Headers.Add("__utmc", "XXXXXXXXX");
request.Headers.Add("__utmc", "XXXXXXXXX");
request.Headers.Add("__utmz", "XXXXXXXXX.XXXXXXXXX.X.X.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)");
request.Headers.Add("__utmz", "XXXXXXXXX.XXXXXXXXX.XX.X.utmcsr=titlesource.com|utmccn=(referral)|utmcmd=referral|utmcct=/");
request.CookieContainer = cookieJar;
request.Method = WebRequestMethods.Http.Post;
request.KeepAlive = true;
request.Accept = "*/*";
request.Headers.Add("Cache-Control", "no-cache");
request.Headers.Add("Accept-Encoding: gzip,deflate,sdch");
request.Headers.Add("Accept-Language: en-US,en;q=0.8");
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36";
request.Headers.Add("X-Requested-With: XMLHttpRequest");
request.ContentType = "application/x-www-form-urlencoded; charset=UTF-8";
request.Referer = "https://foo.bar.com/UserSignIn";
request.Headers.Add("Origin", "https://foo.bar.com");
request.Headers.Add("X-MicrosoftAjax", "Delta=true");
request.ContentLength = buffer.Length;
Stream PostData = request.GetRequestStream();
PostData.Write(buffer, 0, buffer.Length);
PostData.Close();
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream Answer = response.GetResponseStream();
StreamReader _Answer = new StreamReader(Answer);
String html = _Answer.ReadToEnd();
response.Close();
}
catch (Exception ex)
{
Console.WriteLine("Error FillingCookieJar");
}
}
You need to decompress the gzipped stream to get back the plain text, which is likely in UTF-8 rather than ASCII (look for a charset attribute on the response's Content-Type header).
You can use the AutomaticDecompression property to automatically decompress the content.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(target);
request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
The url you see is UrlEncoded. You can use HttpUtility.UrlDecode to get the URL to look like https://foo.bar.com/Vendor.
string decoded = HttpUtility.UrlDecode(html);
Hi I am using a facebook posts feeding application which will read all the new posts from my facebook page and will save to my sharepoint list. Earlier, i.e, before June it has worked properly it feeds all the posts from fabebook to my Share point list. But nowadays its throws an error while getting response from the Facebook authorize url. I don't know what went wrong. Please help me if you guys have any suggestion to resolve this issue. I am appending my code part below.
private void btnSendToList_Click(object sender, EventArgs e)
{
try
{
if (ConfigurationSettings.AppSettings["fbClientID"] != null)
strClientID = ConfigurationSettings.AppSettings["fbClientID"];
if (ConfigurationSettings.AppSettings["fbRedirectURL"] != null)
strURL = ConfigurationSettings.AppSettings["fbRedirectURL"];
if (ConfigurationSettings.AppSettings["fbCltSecret"] != null)
strCltSecret = ConfigurationSettings.AppSettings["fbCltSecret"];
if (ConfigurationSettings.AppSettings["fbUserId"] != null)
strUserId = ConfigurationSettings.AppSettings["fbUserId"];
if (ConfigurationSettings.AppSettings["SPSiteURL"] != null)
strSiteURL = ConfigurationSettings.AppSettings["SPSiteURL"];
CookieCollection cookies = new CookieCollection();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://www.facebook.com");
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
request.CookieContainer = new CookieContainer();
request.CookieContainer.Add(cookies);
//Get the response from the server and save the cookies from the first request..
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
cookies = response.Cookies;
string getUrl = "https://www.facebook.com/login.php?login_attempt=1";
string postData = String.Format("email={0}&pass={1}", "testuser#gmail.com", "test123$"); // Masking credentials.
getRequest = (HttpWebRequest)WebRequest.Create(getUrl);
getRequest.CookieContainer = request.CookieContainer;
getRequest.CookieContainer.Add(cookies);
getRequest.Method = WebRequestMethods.Http.Post;
getRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
getRequest.AllowWriteStreamBuffering = true;
getRequest.ProtocolVersion = HttpVersion.Version11;
getRequest.AllowAutoRedirect = true;
getRequest.ContentType = "application/x-www-form-urlencoded";
byteArray = Encoding.ASCII.GetBytes(postData);
getRequest.ContentLength = byteArray.Length;
newStream = getRequest.GetRequestStream(); //open connection
newStream.Write(byteArray, 0, byteArray.Length); // Send the data.
getResponse = (HttpWebResponse)getRequest.GetResponse();
getRequest = (HttpWebRequest)WebRequest.Create(string.Format("https://graph.facebook.com/oauth/authorize?client_id={0}&redirect_uri={1}", strClientID, "http://www.facebook.com/connect/login_success.html"));
getRequest.Method = WebRequestMethods.Http.Get;
getRequest.CookieContainer = request.CookieContainer;
getRequest.CookieContainer.Add(cookies);
getRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
getRequest.AllowWriteStreamBuffering = true;
//getRequest.ProtocolVersion = HttpVersion.Version11;
getRequest.AllowAutoRedirect = true;
getRequest.ContentType = "application/x-www-form-urlencoded";
getResponse = (HttpWebResponse)getRequest.GetResponse();
The above line throws WebExceptopn which is teeling like Process has timed out.
strAccessToken = getResponse.ResponseUri.Query;
strAccessToken = strAccessToken.Replace("#_=_", "");
strAccessToken = strAccessToken.Replace("?code=", "");
newStream.Close();
if (!string.IsNullOrEmpty(strAccessToken))
strCode = strAccessToken;
if (!string.IsNullOrEmpty(strCode) && !string.IsNullOrEmpty(strClientID) &&
!string.IsNullOrEmpty(strURL) && !string.IsNullOrEmpty(strCltSecret) &&
!string.IsNullOrEmpty(strUserId))
{
SaveToList();
}
}
catch (Exception ex)
{
LogError(ex);
}
}
Hi Finally i got the solution.
Here i am using getResponse = (HttpWebResponse)getRequest.GetResponse();
The problem is we can use only one HttpWebResponse at a time. In my case i am using the same object in two times without any disposing and closing. That's make me the error.
So I updated my code like this.
byteArray = Encoding.ASCII.GetBytes(postData);
getRequest.ContentLength = byteArray.Length;
newStream = getRequest.GetRequestStream(); //open connection
newStream.Write(byteArray, 0, byteArray.Length); // Send the data.
getResponse = (HttpWebResponse)getRequest.GetResponse();
getresponse.Close();
This solved my error. Thanks
I'm using this code to use proxy with HttpWebRequest
public string GetBoardPageResponse(string url, string proxy = "")
{
ServicePointManager.Expect100Continue = false;
HttpWebRequest request = HttpWebRequest.Create(url) as HttpWebRequest;
HttpWebResponse response = null;
WebProxy myProxy = new WebProxy(proxy);
request.Proxy = myProxy;
request.Timeout = 20000;
request.ReadWriteTimeout = 20000;
request.Accept = "*/*";
request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 (.NET CLR 3.5.30729)";
request.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip,deflate");
request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
// SEND POST
Stream os = null;
StreamReader sr = null;
try
{
//post data
byte[] bytes = Encoding.ASCII.GetBytes(param);
if (param.Length > 0)
{
request.ContentLength = bytes.Length; //Count bytes to send
os = request.GetRequestStream();
os.Write(bytes, 0, bytes.Length); //Send it
}
// Get the response
HttpWebResponse webResponse;
using (webResponse = (HttpWebResponse)request.GetResponse())
if (webResponse == null)
return "";
sr = new StreamReader(webResponse.GetResponseStream(), Encoding.GetEncoding(webResponse.CharacterSet));
string encoding = webResponse.CharacterSet;
string data = sr.ReadToEnd().Trim();
return data;
}
catch (Exception ex)
{
return "";
}
finally
{
if (sr != null)
sr.Close();
if (response != null)
response.Close();
if (os != null)
os.Close();
}
}
now this function works fine if I don't use proxy server. but If I add any proxy it will return null result. if I use same proxy with WebClient it works like charm.. I really have no idea what's really blocking or bugging this..
any ideas or help will be appreciated!
just changed: using (webResponse = (HttpWebResponse)request.GetResponse())
to webResponse = (HttpWebResponse)request.GetResponse();
nooby miskate..