C# Get Javascript Cookies with HttpClient - c#

I'm trying to get the cookies from here: https://www.etoro.com/people/wesl3y/portfolio
If I call the Webpage with my Edge Browser, there are a lot of cookies, which are not flagged as "HTTP". For example the follwing cookie:
TMIS2 9a74f8b353780f2fbe59d8dc1d9cd901437be0b823f8ee60d0ab3637053d1bd9675fd2caa6404a74b4fc2a8779e1e7d486862875b11af976466a696de4a89c87e0e942f11ceed91c952d629646dc9cb71bdc2c2fd95a0a71698729f733717a775675add1a06bd9883e47c30e3bd162fabd836467a1cc16870a752373581adfd2ca .etoro.com / 2021-03-20T17:40:43.000Z 263 Medium
I want to get this cookie in my c# Program via HttpClient also, but until now I don't know how :( As far as I understood this cookies are not accessed via http, else Javascript is used to set this cookies on clientside.
Currently I used the following code for accessing the page and show the found cookies:
public void Test()
{
CookieContainer CookieJar = new CookieContainer();
var handler = new HttpClientHandler
{
CookieContainer = CookieJar,
UseCookies = true,
};
HttpClient client = new HttpClient(handler);
HttpResponseMessage response = client.GetAsync("https://www.etoro.com/people/wesl3y/portfolio").Result;
Uri uri = new Uri("https://www.etoro.com/people/wesl3y/portfolio");
IEnumerable<Cookie> responseCookies = CookieJar.GetCookies(uri).Cast<Cookie>();
foreach (Cookie cookie in responseCookies)
Console.WriteLine(cookie.Name + ": " + cookie.Value);
}
With my code I got all Cookies with Parameter HTTP=true, but miss all the other ones. Is there a way to get the missing cookies from that site with HttpClient? Or is there any other type of c# integrated Browser i have to use? The Main focus of my Tool is getting Json Information from the etoro.com API. But the API Access is very Rate limited when the right cookies are not in place :(
Thanks, Mag.

Related

Microsoft.Toolkit.Forms.UI.Controls.WebView: Howto Set Cookies

How to set cookies in the MS-Edge based Microsoft.Toolkit.Forms.UI.Controls.WebView?
I need to send an autentication token cookie to the website I'm navigating to.
What I've tried:
Passing a cookie header to the Navigate method: The header won't be passed to the website (verified by Fiddler). Other headers (like "MyCustomHeader" in the example below) are passed to the site though.
string cookieHeader = cookieContainer.GetCookieHeader(siteUri);
var headers = new Dictionary<string, string>();
headers.Add("Cookie", "MyAuthCookie=MyAuthToken; Domain=.somesite.net; Path=/");
headers.Add("MyCustomHeader", "MyCustomHeader-Value");
_browser.Navigate(siteUri, HttpMethod.Get, headers: headers);
Setting the cookie in CookieManager before calling WebView.Navigate:
var siteUri = new Uri("http://wwww.somesite.net/");
var filter = new Windows.Web.Http.Filters.HttpBaseProtocolFilter();
var cookieManager = filter.CookieManager;
var cookie = new Windows.Web.Http.HttpCookie("MyAuthCookie", siteUri.Host, "/");
cookie.Value = "MyAuthToken";
cookieManager.SetCookie(cookie);
webView.Navigate(siteUri);
This also does not work when calling NavigateWithHttpRequestMessage Microsoft.Toolkit.Win32.UI.Controls.Interop.WinRT.WebViewControlHost (via reflection) instead of WebView.Navigate.
It also does not work when requesting the same URL by HttpClient before calling WebView.Navigate:
using (var client = new Windows.Web.Http.HttpClient(filter) { })
{
var result = client.GetAsync(siteUri).AsTask().Result;
result.EnsureSuccessStatusCode();
}
webView.Navigate(siteUri);
That way, the cookie header is only sent with the HttpClient's request, but not with the subsequent WebView.Navigate's request. I guess that the reason for this could be the fact that WebView runs in it's own process.
Is there any way to pass the cookie to the website? Note that the cookie does not originate from the site. The authentication token is retrieved from some other system, and needs to be passed to the website.

HttpClient calling API doesnt not pass cookie auth

Trying to call an API from a controller using HttpClient and the API does not recognize the user as authenticated and logged in. When calling the API from JS I have no issue. I noticed the HttpClient was only sending via HTTP 1.1 and so I upgraded to 2.0 settings the DOTNET_SYSTEM_NET_HTTP_USESOCKETSHTTPHANDLER flag but this made no difference. I have tried all combinations of the HttpClientHandler properties including UseCookies and the request is never authenticated.
using (var handler = new HttpClientHandler {UseDefaultCredentials = true})
{
using (var httpClient = new HttpClient(handler))
{
var response = httpClient.GetStringAsync(new Uri($"https://localhost:64366/api/")).Result;
}
}
Will move to token based auth in the future but for now would like to understand why there is a difference between calling the API from C# vs JS. This is all HTTPS on localhost using asp net core 2.2.
Difference between JS and C# is that browsers attach cookies automatically to requests and you have to attach cookies manually in C# as juunas mentioned.
To obtain and use authentication cookie you may use the following pattern
CookieContainer cookies = new CookieContainer(); //this container saves cookies from responses and send them in requests
var handler = new HttpClientHandler
{
CookieContainer = cookies
};
var client = new HttpClient(handler);
string authUrl = ""; //your auth url
string anyUrl = ""; //any url that requires you to be authenticated
var authContent = new FormUrlEncodedContent(
new List<KeyValuePair<string, string>> {
new KeyValuePair<string, string>("login", "log_in"),
new KeyValuePair<string, string>("password", "pass_word")
}
);
//cookies will be set on this request
HttpResponseMessage auth = await client.PostAsync(authUrl, authContent);
auth.EnsureSuccessStatusCode(); //retrieving result is not required but you will know if something goes wrong on authentication
//and here retrieved cookies will be used
string result = await client.GetStringAsync(anyUrl);

C# csExWB and cookie

I am trying to download daily a content of a webpage using csEXWB for my c# application usage. The site requires authentication for this. Using my browser and fiddler i got the cookie which contains my authentication information.
I have 2 questions:
1.How can i send a download request using this cookie to the webpage?
2.Can I prevent my cookie from expiration?
I am really new to cookie usage. Any pointer would be helpful.
THANKS!
Here is a method that creates an HTTP request with a session cookie:
private HttpWebRequest CreateRequest(string url, string method, string sessionCookie)
{
if (string.IsNullOrEmpty(url))
throw new ArgumentException("Empty value", "url");
HttpWebRequest result = WebRequest.Create(url) as HttpWebRequest;
if (result == null)
throw new ArgumentException("Only the HTTP protocol is supported", "url");
result.CookieContainer = new CookieContainer();
if (sessionCookie != null) {
result.CookieContainer.Add(
new Uri(url),
new Cookie(SessionCookieName, sessionCookie)
);
}
result.Method = method;
return result;
} // CreateRequest
where 'SessionCookieName' is the name of the cookie to send.
The cookie expiration is controlled by the originating Web site, you cannot prevent it from expiring.
I think that what you should really do is to write code that does the log-in to the Web site - how to do it would depend on how the login works on the Web site you want to access.

Using uTorrent Web API via .NET

I'm trying to get list of torrents from uTorrent using Web API. Getting required token goes O.K.:
WebClient client = new WebClient() { Credentials = new NetworkCredential(UserName, pass) };
StreamReader Reader = new StreamReader(client.OpenRead("http://localhost:" + port + "/gui/token.html"));
string token = Reader.ReadToEnd();
token = token.Split('>')[2].Split('<')[0];
// token is now something like 3LemfrO_-A-SNBXlnQ2QcQWTYydx7qOqKb1W1S54JJW74Ly3EYGgu0xQSU4AAAAA
But when I try to use it to get list of torrents:
Reader = new StreamReader(client.OpenRead("http://localhost:" + port + "/gui/?list=1&token=" + token));
all I get is "Error 400 Bad request".
I've tried to get token manually. In browser page "http://localhost:30303/gui/?list=1&token=3LemfrO_-A-SNBXlnQ2QcQWTYydx7qOqKb1W1S54JJW74Ly3EYGgu0xQSU4AAAAA" opens as it should, but in C# with the same link without any variables I still get error 400.
The interesting part is that if switch off token authentication WebClient load page perfectly with and without
"&token=3LemfrO_-A-SNBXlnQ2QcQWTYydx7qOqKb1W1S54JJW74Ly3EYGgu0xQSU4AAAAA"
but token auth enabled by default, so my and any app should use it.
And yes, WebRequest/HttpWebRequest didn't help also.
P.S. sorry for my English, I was never able to make it work right
you have to save the cookie from the request
Classes.CookieAwareWebClient client = new Classes.CookieAwareWebClient() { Credentials = new NetworkCredential("shehab", "shehab") };
StreamReader Reader = new StreamReader(client.OpenRead("http://localhost:" + "8080" + "/gui/token.html"));
string token = HtmlRemoval.StripTagsRegexCompiled(Reader.ReadToEnd());
MessageBox.Show(token);
Reader = new StreamReader(client.OpenRead("http://localhost:" + "8080" + "/gui/?list=1&token=" + token));
MessageBox.Show(Reader.ReadToEnd());
and for the cookie aware class go to the following link(Using CookieContainer with WebClient class) as web client doesn't support cookies.
You should save cookies from request
WebRequest request = WebRequest.Create("http://localhost:" + port + "/gui/token.html");
CookieContainer cookies = new CookieContainer();
(request as HttpWebRequest).CookieContainer = cookies;
And then use it in every other request to uTorrent when using the same token:
request = WebRequest.Create("http://localhost:" + port + "/gui/?list=1&token=" + token);
(request as HttpWebRequest).CookieContainer = cookies;
I have a simple 3-step suggestion:
When you use your browser with the token, use Fiddler2 to analyze the HTTP traffic between the server and browser.
Open up your C# app and use Fiddler2 to analyze the HTTP traffic between the server and your app.
Compare the HTTP requests and responses for the browser with the requests and responses for the C# app. If you see a significant difference, there is a good chance that could be the problem.

Problem with http 'POST' request

I am running in a stupid problem. I have an method which returns initialized request resposible for loggin in on some external web site.
protected HttpWebRequest GetLoginRequest()
{
const string url = "https://someurl.com/login";
var queryParams = new ArrayList
{
String.Format("{0}={1}", "email", Email),
String.Format("{0}={1}", "password", DecryptedPassword)
};
var parameters = String.Join("&", (String[])queryParams.ToArray(typeof(String)));
var request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = parameters.Length;
request.Timeout = 10000;
var streamWriter = new StreamWriter(request.GetRequestStream());
streamWriter.Write(parameters);
streamWriter.Close();
return request;
}
I'm calling this method from two places in my code. First call looks like that:
var request = GetLoginRequest();
var response = (HttpWebResponse)request.GetResponse();
And the second one has CookieContainer assigned to request:
var cookieContainer = new CookieContainer();
var request = GetLoginRequest();
request.CookieContainer = cookieContainer;
var response = (HttpWebResponse)request.GetResponse();
because I need to store CookieContainer.
The thing is that the logon is performed only in second case. In the first case i'm getting response from the login page. I've checked all the cases and both resulting requests seem identic. I would suggest that it is target site secific, but still I don't see any reason for that.
Can you please explain what is the reason, because this behavior seems pretty unobvious to me.
When you set the CookieContainer property on your request, the response is populating that CookieContainer instance with the cookies received from the executed request.
Most login mechanisms use a cookie to store the state related to the established login. I.e. in the case of Forms authentication the cookie is the container for the forms authentication ticket. The ticket is passed as the value of the forms authentication cookie with each request and is used by forms authentication, on the server, to identify an authenticated user.
In short you need a CookieContainer for each request after you login, and it needs to contain the forms authentication cookie that you received when you logged in.
Edit to clarify comment - from MSDN:
CookieContainer is null by default.
You must assign a CookieContainer
object to the property to have cookies
returned in the Cookies property of
the HttpWebResponse returned by the
GetResponse method.
For security reasons, cookies are disabled by default. If you want to
use cookies, use the CookieContainer
property to enable cookies.

Categories