We Keep Coding

Sequence of HttpWebRequest

I am currently in the process of automating a web interface for administrating users of an FTP.
I am trying to do this with HttpWebRequest, i have one call that logs me on the site and the second call is supose to add a new user for the FTP access.
I have tried my two urls in the browser and they work, they end up creating a user.
string login = "https://www.ftpsite.net/./panel/index.php?txvUsername=myaccount&txvPassword=myPassword&submitButton=Login";
this gets me logged in when i enter it in the browser address bar.
the second call to create a user is as follows.
string createUser = "https://www.ftpSite.net/panel/ftpsites/updatelogin?login_id=&login=toto123&realname=realnametoto&homedir=root&passwd=superpassword11&expdate=01-01-2100&neverExpire=on&quota_value=0&quota_unit=GB&group_id=0&status=on&ftp=on&filelist=on&ftp_download=on&http=on&web_filelist=on&web_download=on&email=";
This creates a user when i enter it in the browser's address bar if it follows the one that logs us in.
My problem is that i am trying to do this using HttpWebRequest and without success. I can get myself logged in but when i try to create the user it seems to return a "bad" error code saying i have created too many users already which isnt the case since i can create more after that call. Here is the code i use with HtttpRequest
_datCookie = new CookieContainer();
HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create(login);
httpWebRequest.Method = "POST";
httpWebRequest.CookieContainer = _datCookie;
WebResponse response = httpWebRequest.GetResponse();
referer = response.ResponseUri.AbsoluteUri;
Stream requestStream = response.GetResponseStream();
StreamReader streamReader = new StreamReader(requestStream);
_datCookie = httpWebRequest.CookieContainer;
response.Close();
httpWebRequest = (HttpWebRequest)WebRequest.Create(createUser);
httpWebRequest.CookieContainer = _datCookie;
httpWebRequest.Referer = referer;
httpWebRequest.Method = "POST";
response = httpWebRequest.GetResponse();
requestStream = response.GetResponseStream();
streamReader = new StreamReader(requestStream);
webBrowser.DocumentText = streamReader.ReadToEnd();
response.Close();
What i caught and tried to imitate without success here.

Are you sure they should be POST requests? The URLs seem to have all of the fields in the query-string, which suggests they should be GET requests instead.
Based on the Fiddler screen-shot, you need to make a POST request with the fields in the body, not the query-string:
var cookies = new CookieContainer();
// Request 1 : Login
var request = (HttpWebRequest)WebRequest.Create("https://www.ftpsite.net/./panel/index.php");
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.CookieContainer = cookies;
string postData = "txvUsername=myaccount&txvPassword=myPassword&submitButton=Login";
byte[] postBytes = Encoding.Default.GetBytes(postData);
request.ContentLength = postBytes.Length;
using (Stream bod = request.GetRequestStream())
{
body.Write(postBytes, 0, postBytes.Length);
}
WebResponse response = request.GetResponse();
string referer = response.ResponseUri.AbsoluteUri;
// Request 2 : Create user
request = (HttpWebRequest)WebRequest.Create("https://www.ftpSite.net/panel/ftpsites/updatelogin");
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.CookieContainer = cookies;
postData = "login_id=&login=toto123&realname=realnametoto&homedir=root&passwd=superpassword11&expdate=01-01-2100&neverExpire=on&quota_value=0&quota_unit=GB&group_id=0&status=on&ftp=on&filelist=on&ftp_download=on&http=on&web_filelist=on&web_download=on&email=";
postBytes = Encoding.Default.GetBytes(postData);
request.ContentLength = postBytes.Length;
using (Stream bod = request.GetRequestStream())
{
body.Write(postBytes, 0, postBytes.Length);
}
response = request.GetResponse();
requestStream = response.GetResponseStream();
streamReader = new StreamReader(requestStream);
webBrowser.DocumentText = streamReader.ReadToEnd();
response.Close();

Login to Https Website with WebRequest and WebResponse Not Working C#

I am working on a program that will log into a website and get certian data. However I am having trouble posting the login parameters and dealing with the cookies, as each time I get a page saying "You have logged out or Session has expired." So clearly I'm doing something wrong with posting the parameters or dealing with the cookies, but don't know which. I have been working on this for a while and just can't get my head around why this is not working correctly.
void Login2(string username, string password)
{
string pageSource;
string formUrl = "https://forUrl.com";
string formParams = string.Format("login={0}&sslProt={1}&pwd={2}&gru={3}", username, "", password, "115237091");
string cookieHeader;
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(formUrl);
req.AllowAutoRedirect = false;
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
string getUrl = "https://Urlbehindform.com";
HttpWebRequest getRequest = (HttpWebRequest)WebRequest.Create(getUrl);
getRequest.Method = "GET";
getRequest.AllowAutoRedirect = false;
getRequest.Headers.Add("Cookie", cookieHeader);
HttpWebResponse getResponse = (HttpWebResponse)getRequest.GetResponse();
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
Response.Redirect(getUrl);
}
I am getting the cookie when I do the POST and sending it back when I do the GET, but for some reason this doesn't seem to work. At first I thought it was the parameters, but after looking at the issue further using Tamper Data with Firefox the login parameters seem to be working fine. Any help would be great, as I have been working on this for a while and can't wrap my head around it. Thanks!
UPDATE:
After trying out a few suggestions I still can't get this to work. However Upon looking deeper into Data Tamper, It appears that there is a POST with the login parameters, then a GET to a different page and then finally the GET to the page after the login page (The one I'm trying to get to). After some further debugging I actually discovered that my login POST is not working as I thought, As the response header location is showing "/cv/scripts/A028/eng/logErr.asp". Meaning the rest of my code could have been fine all a long, it was that the POST wasn't giving me a valid login. Any Sugguestions as to why I am always getting the login error page? As always thanks for the help.
UPDATE:
After playing around further with Tamper Data is appears that the reason I am unable to get a successful login is that in order have a successful POST of the parameters there needs to be a cookie already obtained. How do I go about doing this?

Use a single CookieContainer for both requests. Then you don't have to copy cookies manually.
I [BMW1] added in a CookieContainer called cookies, but it still not working, Im not sure if im using the CookieContainer the right way. Here is an updated version of my code.
And edited by me [Hans Kesting], see comments with [HK]
void Login2(string username, string password)
{
string pageSource;
string formUrl = "https://server/cv/scripts/A028/eng/logProc.asp?ntry=0&dbg=";
string formParams = string.Format("login={0}&sslProt={1}&pwd={2}&gru={3}", username, "", password, "115237091");
// [HK] create a container for the cookies, where they are added automatically
CookieContainer cookies = new CookieContainer();
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(formUrl);
req.CookieContainer = cookies;
req.AllowAutoRedirect = false;
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
// [HK] no need to add cookies "by hand", that will happen automatically
//cookies.Add(resp.Cookies);
string getUrl = "https://server/cv/scripts/A028/eng/home.asp";
HttpWebRequest getRequest = (HttpWebRequest)WebRequest.Create(getUrl);
// [HK] use the same cookiecontainer as on the first request - correct
getRequest.CookieContainer = cookies;
getRequest.Method = "GET";
getRequest.AllowAutoRedirect = false;
HttpWebResponse getResponse = (HttpWebResponse)getRequest.GetResponse();
// [HK] no need to add cookies, they should be there already
//cookies.Add(getResponse.Cookies);
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
// [HK] no need to add cookies, they should be there already
// cookies.Add(getResponse.Cookies);
Response.Redirect(getUrl);
}

You could use a Cookie aware web client,
public class CookieAwareWebClient : WebClient
{
public CookieContainer CookieContainer { get; set; }
public Uri Uri { get; set; }
public CookieAwareWebClient() : this (new CookieContainer())
{
}
public CookieAwareWebClient(CookieContainer cookies)
{
this.CookieContainer = cookies;
}
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest request = base.GetWebRequest(address);
if (request is HttpWebRequest)
{
(request as HttpWebRequest).CookieContainer = this.CookieContainer;
}
HttpWebRequest httpRequest = (HttpWebRequest) request;
httpRequest.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
return httpRequest;
}
protected override WebResponse GetWebResponse(WebRequest request)
{
WebResponse response = base.GetWebResponse(request);
String setCookieHeader = response.Headers[HttpResponseHeader.SetCookie];
if (setCookieHeader != null)
{
//do something if needed to parse out the cookie.
if (setCookieHeader != null)
{
Cookie cookie = new Cookie(); //create cookie
this.CookieContainer.Add(cookie);
}
}
return response;
}
}
Example usage:
var wc = new CookieAwareWebClient ();
wc.Headers["Content-type"] = "application/x-www-form-urlencoded";
string HtmlResult = wc.UploadString(URI, myParameters);

Why is my HttpWebRequest redirecting to Login page?

I have an HttpWebRequest to grab a session ID. I then get the cookie from the response, add it to a second request to get the page I need.
Using IIS/7.5, what are possible scenarios for this failing?
I am using Fiddler, and get a 302 status. I am getting the ASPNET SessionID.
CookieContainer myCookies = new CookieContainer();
HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create("https://www.secure.com/login.aspx");
req.Method = "POST";
req.Credentials = fileretrieve.Credentials;//Network credentials.
req.CookieContainer = myCookies;
req.AllowAutoRedirect = true;
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(req.Credentials.ToString());
req.ContentLength = bytes.Length;
System.IO.Stream os = req.GetRequestStream();
os.Write(bytes, 0, bytes.Length);
os.Close();
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
HttpWebRequest xmlreq = (HttpWebRequest)HttpWebRequest.Create("https://www.secure.com/file");
xmlreq.Method = "GET";
xmlreq.KeepAlive = true;
xmlreq.AllowAutoRedirect = true;
xmlreq.CookieContainer = req.CookieContainer;
HttpWebResponse xmlresp = (HttpWebResponse)xmlreq.GetResponse();
string webpage;
System.IO.StreamReader sr = new System.IO.StreamReader(xmlresp.GetResponseStream());
webpage = sr.ReadToEnd();

I think it is failing because you are trying to use .Credentials property to authenticate in FormsAuthentication-based application. Credentials property would work for Basic or NTLM authentication, but not for Forms authentication, in that case you just need to POST expected fields to Login.aspx page in order to get Auth cookie back.

HttpWebRequest pass credentials to next HttpWebRequest

I am logging into a page using HttpWebRequest and getting some information. I then use that information to create a new HttpWebRequest to get some more information. I do not want to use WebClient.
How can I pass the credentials I obtained from logging in using the first HttpWebRequest to the second one?
EDIT: If I use a CookieCollection then this is coming back as empty. I just tried using WebClient as a last resort and even for that it is not working, the second request takes me back to the login screen. I noticed that in a WebBrowser there is a cookie.

Add a CookieContainer to each request before you send it. Add the cookies you get from the first response to the second request. Assuming they use cookies for authentication, this should authenticate the second request.
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(urlWithParameters);
request.CookieContainer = new CookieContainer();
HttpWebResponse response = (HttpWebResponse) request.GetResponse();
var cookies = new CookieContainer();
cookies.Add( response.Cookies );
request = (HttpWebRequest)WebRequest.Create(secondUrlWithParameters);
request.CookieContainer = cookies;
...

this is just a sample running code based on answer 2. Maybe be redundant maybe illustrate somebody.
string url = "http://servername/place-where-data-is.extension"
string loginUrl = "https://servername/sampleLogin?email=eeeeee&passwd=xxxxxxx";
HttpWebRequest loginRequest = (HttpWebRequest)HttpWebRequest.Create(loginUrl);
loginRequest.CookieContainer = new CookieContainer();
loginRequest.Method = WebRequestMethods.Http.Get;
HttpWebResponse loginResponse = (HttpWebResponse)loginRequest.GetResponse();
var cookies = new CookieContainer();
cookies.Add(loginResponse.Cookies);
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.CookieContainer = cookies;
request.Method = WebRequestMethods.Http.Get;
WebResponse response = (WebResponse)request.GetResponse();
Stream responseStream = response.GetResponseStream();

This is a really old question and I know it states no WebClient but I will post here for everyone who comes across this from Google. The original concept is not my code. I do not know where I originally found it.
using (WebClientEx client = new WebClientEx())
{
client.IntTimeout = intTimeout;
client.DownloadString(strReportUrlPrefix + strReportUrlQuery);
NameValueCollection auth = new NameValueCollection
{
{ "j_username", strReportUsername},
{ "j_password", strReportPassword}
};
byte[] data = client.UploadValues(strReportUrlPrefix + "j_security_check", auth);
// LOGIC HERE WITH DATA
}
WebClientEx Class:
public class WebClientEx : WebClient
{
private CookieContainer _cookieContainer = new CookieContainer();
public int IntTimeout { get; set; }
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest request = base.GetWebRequest(address);
if (request != null)
request.Timeout = IntTimeout;
if (request is HttpWebRequest)
(request as HttpWebRequest).CookieContainer = _cookieContainer;
return request;
}
}

HTTP 401 Error while posting XML to REST

I am trying to post a XML to REST Service. Here's the code I am using:
I am getting following error, while calling the service.
The remote server returned an error: (401) Unauthorized.
I have also tried setting NetworkCredentials directly i.e.
NetworkCredential nc = new NetworkCredential(username, password);
serviceRequest.Credentials = nc;
Thanks for your help.
Uri address = new Uri("https://localhost:30000/restservice/");
// Create the web request
HttpWebRequest request = WebRequest.Create(address) as HttpWebRequest;
// Set type to POST
request.Method = "POST";
request.ContentType = "application/json";
string data = #"<Sample XML Here>";
// Create a byte array of the data we want to send
byte[] byteData = UTF8Encoding.UTF8.GetBytes(data);
// Set the content length in the request headers
request.ContentLength = byteData.Length;
// Write data
using (Stream postStream = request.GetRequestStream())
{
postStream.Write(byteData, 0, byteData.Length);
}
string usernamePassword = username + ":" + password;
CredentialCache mycache = new CredentialCache();
mycache.Add(address, "Basic", new NetworkCredential(username, password));
request.Credentials = mycache;
// Get response
using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
{
// Get the response stream
StreamReader reader = new StreamReader(response.GetResponseStream());
// Console application output
Response.Write(reader.ReadToEnd());
}

Use Fiddler and look in the WWW-Authenticate header that is returned from the server. That will tell you what kind of authentication scheme the server supports.

Couple of things to try:
Change the content type as you're not posting json to it
Not encode the data as its expecting xml, not a binary stream
Hope this helps.

try set Credentials in request like this
request.Credentials = new NetworkCredential(username, password);