I am trying to download daily a content of a webpage using csEXWB for my c# application usage. The site requires authentication for this. Using my browser and fiddler i got the cookie which contains my authentication information.
I have 2 questions:
1.How can i send a download request using this cookie to the webpage?
2.Can I prevent my cookie from expiration?
I am really new to cookie usage. Any pointer would be helpful.
THANKS!
Here is a method that creates an HTTP request with a session cookie:
private HttpWebRequest CreateRequest(string url, string method, string sessionCookie)
{
if (string.IsNullOrEmpty(url))
throw new ArgumentException("Empty value", "url");
HttpWebRequest result = WebRequest.Create(url) as HttpWebRequest;
if (result == null)
throw new ArgumentException("Only the HTTP protocol is supported", "url");
result.CookieContainer = new CookieContainer();
if (sessionCookie != null) {
result.CookieContainer.Add(
new Uri(url),
new Cookie(SessionCookieName, sessionCookie)
);
}
result.Method = method;
return result;
} // CreateRequest
where 'SessionCookieName' is the name of the cookie to send.
The cookie expiration is controlled by the originating Web site, you cannot prevent it from expiring.
I think that what you should really do is to write code that does the log-in to the Web site - how to do it would depend on how the login works on the Web site you want to access.
Related
I'm trying to get the cookies from here: https://www.etoro.com/people/wesl3y/portfolio
If I call the Webpage with my Edge Browser, there are a lot of cookies, which are not flagged as "HTTP". For example the follwing cookie:
TMIS2 9a74f8b353780f2fbe59d8dc1d9cd901437be0b823f8ee60d0ab3637053d1bd9675fd2caa6404a74b4fc2a8779e1e7d486862875b11af976466a696de4a89c87e0e942f11ceed91c952d629646dc9cb71bdc2c2fd95a0a71698729f733717a775675add1a06bd9883e47c30e3bd162fabd836467a1cc16870a752373581adfd2ca .etoro.com / 2021-03-20T17:40:43.000Z 263 Medium
I want to get this cookie in my c# Program via HttpClient also, but until now I don't know how :( As far as I understood this cookies are not accessed via http, else Javascript is used to set this cookies on clientside.
Currently I used the following code for accessing the page and show the found cookies:
public void Test()
{
CookieContainer CookieJar = new CookieContainer();
var handler = new HttpClientHandler
{
CookieContainer = CookieJar,
UseCookies = true,
};
HttpClient client = new HttpClient(handler);
HttpResponseMessage response = client.GetAsync("https://www.etoro.com/people/wesl3y/portfolio").Result;
Uri uri = new Uri("https://www.etoro.com/people/wesl3y/portfolio");
IEnumerable<Cookie> responseCookies = CookieJar.GetCookies(uri).Cast<Cookie>();
foreach (Cookie cookie in responseCookies)
Console.WriteLine(cookie.Name + ": " + cookie.Value);
}
With my code I got all Cookies with Parameter HTTP=true, but miss all the other ones. Is there a way to get the missing cookies from that site with HttpClient? Or is there any other type of c# integrated Browser i have to use? The Main focus of my Tool is getting Json Information from the etoro.com API. But the API Access is very Rate limited when the right cookies are not in place :(
Thanks, Mag.
How to set cookies in the MS-Edge based Microsoft.Toolkit.Forms.UI.Controls.WebView?
I need to send an autentication token cookie to the website I'm navigating to.
What I've tried:
Passing a cookie header to the Navigate method: The header won't be passed to the website (verified by Fiddler). Other headers (like "MyCustomHeader" in the example below) are passed to the site though.
string cookieHeader = cookieContainer.GetCookieHeader(siteUri);
var headers = new Dictionary<string, string>();
headers.Add("Cookie", "MyAuthCookie=MyAuthToken; Domain=.somesite.net; Path=/");
headers.Add("MyCustomHeader", "MyCustomHeader-Value");
_browser.Navigate(siteUri, HttpMethod.Get, headers: headers);
Setting the cookie in CookieManager before calling WebView.Navigate:
var siteUri = new Uri("http://wwww.somesite.net/");
var filter = new Windows.Web.Http.Filters.HttpBaseProtocolFilter();
var cookieManager = filter.CookieManager;
var cookie = new Windows.Web.Http.HttpCookie("MyAuthCookie", siteUri.Host, "/");
cookie.Value = "MyAuthToken";
cookieManager.SetCookie(cookie);
webView.Navigate(siteUri);
This also does not work when calling NavigateWithHttpRequestMessage Microsoft.Toolkit.Win32.UI.Controls.Interop.WinRT.WebViewControlHost (via reflection) instead of WebView.Navigate.
It also does not work when requesting the same URL by HttpClient before calling WebView.Navigate:
using (var client = new Windows.Web.Http.HttpClient(filter) { })
{
var result = client.GetAsync(siteUri).AsTask().Result;
result.EnsureSuccessStatusCode();
}
webView.Navigate(siteUri);
That way, the cookie header is only sent with the HttpClient's request, but not with the subsequent WebView.Navigate's request. I guess that the reason for this could be the fact that WebView runs in it's own process.
Is there any way to pass the cookie to the website? Note that the cookie does not originate from the site. The authentication token is retrieved from some other system, and needs to be passed to the website.
I'm trying to get list of torrents from uTorrent using Web API. Getting required token goes O.K.:
WebClient client = new WebClient() { Credentials = new NetworkCredential(UserName, pass) };
StreamReader Reader = new StreamReader(client.OpenRead("http://localhost:" + port + "/gui/token.html"));
string token = Reader.ReadToEnd();
token = token.Split('>')[2].Split('<')[0];
// token is now something like 3LemfrO_-A-SNBXlnQ2QcQWTYydx7qOqKb1W1S54JJW74Ly3EYGgu0xQSU4AAAAA
But when I try to use it to get list of torrents:
Reader = new StreamReader(client.OpenRead("http://localhost:" + port + "/gui/?list=1&token=" + token));
all I get is "Error 400 Bad request".
I've tried to get token manually. In browser page "http://localhost:30303/gui/?list=1&token=3LemfrO_-A-SNBXlnQ2QcQWTYydx7qOqKb1W1S54JJW74Ly3EYGgu0xQSU4AAAAA" opens as it should, but in C# with the same link without any variables I still get error 400.
The interesting part is that if switch off token authentication WebClient load page perfectly with and without
"&token=3LemfrO_-A-SNBXlnQ2QcQWTYydx7qOqKb1W1S54JJW74Ly3EYGgu0xQSU4AAAAA"
but token auth enabled by default, so my and any app should use it.
And yes, WebRequest/HttpWebRequest didn't help also.
P.S. sorry for my English, I was never able to make it work right
you have to save the cookie from the request
Classes.CookieAwareWebClient client = new Classes.CookieAwareWebClient() { Credentials = new NetworkCredential("shehab", "shehab") };
StreamReader Reader = new StreamReader(client.OpenRead("http://localhost:" + "8080" + "/gui/token.html"));
string token = HtmlRemoval.StripTagsRegexCompiled(Reader.ReadToEnd());
MessageBox.Show(token);
Reader = new StreamReader(client.OpenRead("http://localhost:" + "8080" + "/gui/?list=1&token=" + token));
MessageBox.Show(Reader.ReadToEnd());
and for the cookie aware class go to the following link(Using CookieContainer with WebClient class) as web client doesn't support cookies.
You should save cookies from request
WebRequest request = WebRequest.Create("http://localhost:" + port + "/gui/token.html");
CookieContainer cookies = new CookieContainer();
(request as HttpWebRequest).CookieContainer = cookies;
And then use it in every other request to uTorrent when using the same token:
request = WebRequest.Create("http://localhost:" + port + "/gui/?list=1&token=" + token);
(request as HttpWebRequest).CookieContainer = cookies;
I have a simple 3-step suggestion:
When you use your browser with the token, use Fiddler2 to analyze the HTTP traffic between the server and browser.
Open up your C# app and use Fiddler2 to analyze the HTTP traffic between the server and your app.
Compare the HTTP requests and responses for the browser with the requests and responses for the C# app. If you see a significant difference, there is a good chance that could be the problem.
I created RESTful webservice (WCF) where I check credentials on each request. One of my clients is Android app and everything seems to be great on server side. I get request and if it's got proper header - I process it, etc..
Now I created client app that uses this service. This is how I do GET:
// Create the web request
var request = WebRequest.Create(Context.ServiceURL + uri) as HttpWebRequest;
if (request != null)
{
request.ContentType = "application/json";
// Add authentication to request
request.Credentials = new NetworkCredential(Context.UserName, Context.Password);
// Get response
using (var response = request.GetResponse() as HttpWebResponse)
{
// Get the response stream
if (response != null)
{
var reader = new StreamReader(response.GetResponseStream());
// Console application output
var s = reader.ReadToEnd();
var serializer = new JavaScriptSerializer();
var returnValue = (T)serializer.Deserialize(s, typeof(T));
return returnValue;
}
}
}
So, this code get's my resource and deserializes it. As you see - I'm passing credentials in my call.
Then when debugging on server-side I noticed that I get 2 requests every time - one without authentication header and then server sends back response and second request comes bach with credentials. I think it's bad for my server - I'd rather don't make any roundtrips. How should I change client so it doesn't happen? See screenshot of Fiddler
EDIT:
This is JAVA code I use from Android - it doesn't do double-call:
MyHttpResponse response = new MyHttpResponse();
HttpClient client = mMyApplication.getHttpClient();
try
{
HttpGet request = new HttpGet(serviceURL + url);
request.setHeader(new BasicHeader(HTTP.CONTENT_TYPE, "application/json"));
request.addHeader("Authorization", "Basic " + Preferences.getAuthorizationTicket(mContext));
ResponseHandler<String> handler = new BasicResponseHandler();
response.Body = client.execute(request, handler);
response.Code = HttpURLConnection.HTTP_OK;
response.Message = "OK";
}
catch (HttpResponseException e)
{
response.Code = e.getStatusCode();
response.Message = e.getMessage();
LogData.InsertError(mContext, e);
}
The initial request doesn't ever specify the basic header for authentication. Additionally, since a realm is specified, you have to get that from the server. So you have to ask once: "hey, I need this stuff" and the server goes "who are you? the realm of answering is 'secure area'." (because realm means something here) Just because you added it here:
request.Credentials = new NetworkCredential(Context.UserName, Context.Password);
doesn't mean that it's going to be for sure attached everytime to the request.
Then you respond with the username/password (in this case you're doing BASIC so it's base64 encoded as name:password) and the server decodes it and says "ok, you're all clear, here's your data".
This is going to happen on a regular basis, and there's not a lot you can do about it. I would suggest that you also turn on HTTPS since the authentication is happening in plain text over the internet. (actually what you show seems to be over the intranet, but if you do go over the internet make it https).
Here's a link to Wikipedia that might help you further: http://en.wikipedia.org/wiki/Basic_access_authentication
Ok, I got it. I manually set HttpHeader instead of using request.Credentials
request.Headers.Add(HttpRequestHeader.Authorization, "Basic " + Convert.ToBase64String(Encoding.UTF8.GetBytes(Context.UserName + ":" + Context.Password)));
Now I see only single requests as expected..
As an option you can use PreAuthenticate property of HttpClientHandler. This would require a couple of lines more
var client = new HttpClient(new HttpClientHandler
{
Credentials = yourCredentials,
PreAuthenticate = true
});
With using this approach, only the first request is sent without credentials, but all the rest requests are OK.
I am running in a stupid problem. I have an method which returns initialized request resposible for loggin in on some external web site.
protected HttpWebRequest GetLoginRequest()
{
const string url = "https://someurl.com/login";
var queryParams = new ArrayList
{
String.Format("{0}={1}", "email", Email),
String.Format("{0}={1}", "password", DecryptedPassword)
};
var parameters = String.Join("&", (String[])queryParams.ToArray(typeof(String)));
var request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = parameters.Length;
request.Timeout = 10000;
var streamWriter = new StreamWriter(request.GetRequestStream());
streamWriter.Write(parameters);
streamWriter.Close();
return request;
}
I'm calling this method from two places in my code. First call looks like that:
var request = GetLoginRequest();
var response = (HttpWebResponse)request.GetResponse();
And the second one has CookieContainer assigned to request:
var cookieContainer = new CookieContainer();
var request = GetLoginRequest();
request.CookieContainer = cookieContainer;
var response = (HttpWebResponse)request.GetResponse();
because I need to store CookieContainer.
The thing is that the logon is performed only in second case. In the first case i'm getting response from the login page. I've checked all the cases and both resulting requests seem identic. I would suggest that it is target site secific, but still I don't see any reason for that.
Can you please explain what is the reason, because this behavior seems pretty unobvious to me.
When you set the CookieContainer property on your request, the response is populating that CookieContainer instance with the cookies received from the executed request.
Most login mechanisms use a cookie to store the state related to the established login. I.e. in the case of Forms authentication the cookie is the container for the forms authentication ticket. The ticket is passed as the value of the forms authentication cookie with each request and is used by forms authentication, on the server, to identify an authenticated user.
In short you need a CookieContainer for each request after you login, and it needs to contain the forms authentication cookie that you received when you logged in.
Edit to clarify comment - from MSDN:
CookieContainer is null by default.
You must assign a CookieContainer
object to the property to have cookies
returned in the Cookies property of
the HttpWebResponse returned by the
GetResponse method.
For security reasons, cookies are disabled by default. If you want to
use cookies, use the CookieContainer
property to enable cookies.