I want to clear ALL the cookies received in a CookieContainer without the need to initialize a new CookieContainer, HttpClientHandler and HttpClient. Is there a way? I've checked MSDN but it seems I can only use GetCookies(Uri) to get all the cookies associated with a particular Uri.
var cc = new CookieContainer();
var handler = new HttpClientHandler
{
CookieContainer = cc
};
var client = new HttpClient(handler);
The only solution I know is to expire all cookies:
cc.GetCookies(new Uri(...))
.Cast<Cookie>()
.ToList()
.ForEach(c => c.Expired = true);
Related
I'm migrating a WinForms App to a .Net Core (3.1) API and I've run into trouble with a feature which makes a call to a SSO-protected API within our network.
In the WinForms App, I had the following:
public async void Authenticate()
{
var wi = System.Security.Principal.WindowsIdentity.GetCurrent();
var wic = wi.Impersonate();
try
{
// Initial request to grab cookies and redirect endpoint
var apiEndPoint = $"https://{_host}{_apiPath}";
var cookieRedirectRequest = new HttpRequestMessage(HttpMethod.Get, apiEndPoint);
var response = await _httpClient.SendAsync(cookieRedirectRequest);
if (response != null)
{
var cookieContainer = new CookieContainer();
foreach (var cookie in response.Headers.GetValues("Set-Cookie"))
{
var parsedCookie = cookie.ToCookie();
parsedCookie.Domain = TARGET_DOMAIN;
cookieContainer.Add(parsedCookie);
}
var redirectURL = response.Headers.GetValues("Location").FirstOrDefault();
// Add the cookies to the client to continue the authentication
_httpClient = new HttpClient(new HttpClientHandler()
{
UseDefaultCredentials = true,
AllowAutoRedirect = true,
ClientCertificateOptions = ClientCertificateOption.Automatic,
UseCookies = true,
CookieContainer = cookieContainer
});
// Second request to grab code and state values
var codeAndStateRequest = new HttpRequestMessage(HttpMethod.Get, redirectURL);
response = await _httpClient.SendAsync(codeAndStateRequest);
...
The initial call to the API redirects us to the SSO-authentication page. I use the cookies from that initial response to create my own request so that I can pass in the proper credentials and get back the appropriate SSO tokens (the response the SendSync call on the last line returns back the tokens), which I then use to authenticate on future requests to the API.
I'm trying to recreate this functionality in .NET Core using a Service Account, and instead of doing the WindowsIdentity impersonation, I'm trying to explicitly set the NetworkCredentials property of my HttpClientHandler:
handler = new HttpClientHandler()
{
UseDefaultCredentials = false,
Credentials = new NetworkCredential("svc_acct_username", "svc_act_pass", "mydomain"),
AllowAutoRedirect = true,
ClientCertificateOptions = ClientCertificateOption.Automatic,
UseCookies = true,
CookieContainer = cookieContainer,
};
However, the response I'm getting back from my second request is a 401.
This is consistent with what I used to get in my original app -- I would initially get a 401 with a "WWW-Authenticate: Negotiate" header, but then there would be an automatic subsequent request where the network credentials were passed in and I'd get a 200. I'm not getting this second request with the creds in .Net Core.
I tried the Credentials Cache solution suggested here:
https://github.com/dotnet/runtime/issues/24490
But this has not worked.
Thanks in advance for any help you guys can give me. I'm quite stuck!
Quick Edit:
I can recreate the .Net Core behavior in my WinForms app by simply setting the Credentials property of the HttpClientHandler explicitly and setting UseDefaultCredentials = false. So, there's some magic happening by setting UseDefaultCredentials that seemingly can't be recreated when using explicit creds.
I got this working with help from a co-worker! The issue is that when you use explicit Network Credentials, you lose the "Negotiate" authentication type, which is what our SSO-endpoint requires. By using a CredentialCache object and setting the Negotiate auth-type explicitly, the call works!
var credCache = new CredentialCache { { new Uri("https://example.com"), "Negotiate", new NetworkCredential("svc_acct_user", "svc_acct_password", "mydomain") } };
// Add the cookies to the client to continue the authentication
handler = new HttpClientHandler()
{
UseDefaultCredentials = false,
Credentials = credCache,
AllowAutoRedirect = true,
ClientCertificateOptions = ClientCertificateOption.Automatic,
UseCookies = true,
CookieContainer = cookieContainer,
};
Edit: I apologize for the confusion: I'm attempting to scrape a site that I did not write; I'm not writing an ASP app. I'm only attempting to scrape one.
After making a post request to a login page, I attempt to read the cookies from another page. I do not see all the cookies I expect, however. My code is as follows:
// Downloads login cookies for subsequent requests
public async Task<CookieContainer> loginCookies()
{
var cookies = new CookieContainer();
var handler = new HttpClientHandler {
UseCookies = true,
AllowAutoRedirect = true,
CookieContainer = cookies
};
var client = new HttpClient(handler);
var loginUri = new Uri("https://connect.example.edu/login.aspx");
var credentials = new Dictionary<string, string> {
{"example$txtUsername", this.Username},
{"example$txtPassword", this.Password}
};
var formCredentials = new FormUrlEncodedContent(credentials);
await client.PostAsync(loginUri, content: formCredentials);
var pointsUri = new Uri("https://info.example.edu/");
Console.WriteLine("COOKIES:");
foreach (Cookie cookie in cookies.GetCookies(pointsUri))
Console.WriteLine($"{cookie.Name} --> {cookie.Value}");
return cookies;
}
I believe the error is a result of loginUri and pointsUri having different subdomains. The info I need to scrape exists at the pointsUri page, but the login exists at loginUri.
A cookie that I'm missing in particular is ASP.NET_SessionID.
Running a UWP app*
So I have an HttpClient and it's associated handler. I am making a request to a website, passing in specified headers, and using a specified CookieContainer, which is empty at the beginning of the request.
When I send the request, Fiddler shows extra cookies being sent that I have not added. Where are they coming from?
CookieContainer cookieJar = new CookieContainer();
HttpClientHandler handler = new HttpClientHandler( );
handler.UseDefaultCredentials = false;
handler.CookieContainer = cookieJar;
handler.Proxy = null;
using (var client = new HttpClient(handler as HttpClientHandler))
{
HttpResponseMessage response = new HttpResponseMessage();
String loginUrl = Const.BUNGIE_LOGIN_URI + (provider == Const.XBOX_PLATFORM ? Const.XBOX_LOGIN : Const.PS_LOGIN);
client.BaseAddress = new Uri(loginUrl);
//client.Timeout = new TimeSpan(0, 0, 0, 0, 450);
client.DefaultRequestHeaders.Add("Referer", "http://www.bungie.net");
client.DefaultRequestHeaders.Add("User-Agent", Const.USERAGENT);
client.DefaultRequestHeaders.Add("X-API-Key", Const.X_API_KEY);
client.DefaultRequestHeaders.Add("Connection", "Keep-Alive");
client.DefaultRequestHeaders.Add("Accept", "application/json");
client.DefaultRequestHeaders.Add("Accept-Language", "en-GB,en-US;q=0.8,en;q=0.6");
handler.AutomaticDecompression = DecompressionMethods.GZip;
response = client.GetAsync("").Result;
response.ReadCookies(); //adds cookies to cookieJar
}
What fiddler shows
Now, as the associated CookieContainer is empty before the request is made, where are these cookies coming from? Are they accessible? If I wanted the values from them, how would I obtain them?
Edit: Where are they being added to my HttpClient request from? Does HttpClient have a common CookieContainer / cache? I have two separate HttpClient instances, and when Client (A) makes a request, it received a "set-cookie" header back, setting a "bungled" cookie.
Later on, a separate instance of HttpClient, Client (B), makes a request to the same website, sending the cookie set within Client (A).
I did not explicitly append this cookie to Client (B)'s request, so how is it being added?
On Windows 10 build 10240, System.Net.Http.HttpClient is a wrapper on top of Windows.Web.Http.HttpClient (more info here), and so, cookies now work little bit different.
To delete those extra cookies, you will need to delete the cookies using a HttpCookieManager:
HttpBaseProtocolFilter filter = new HttpBaseProtocolFilter();
HttpCookieManager cookieManager = filter.CookieManager;
foreach (HttpCookie cookie in cookieManager.GetCookies(uri))
{
cookieManager.DeleteCookie();
}
I have to call a restful API, but the only details I have are an API ID and an API key.
I am trying to use Restsharp library code like this.
var client = new RestClient("https://xxxxxxx");
client.Authenticator = new HttpBasicAuthenticator("xxxxx", "yyyyyy");
I get a 401 authorization required error.
Could you please point me in the right direction.
Thanks.
Here is the solution that i came up with.
this returns the client object
private RestClient InitializeAndGetClient()
{
var cookieJar = new CookieContainer();
var client = new RestClient("https://xxxxxxx")
{
Authenticator = new HttpBasicAuthenticator("xxIDxx", "xxKeyxx"),
CookieContainer = cookieJar
};
return client;
}
and you can use the method like
var client = InitializeAndGetClient();
var request = new RestRequest("report/transaction", Method.GET);
request.AddParameter("option", "value");
//Run once to get cookie.
var response = client.Execute(request);
//Run second time to get actual data
response = client.Execute(request);
Hope this helps you.
Prakash.
I know this is old but this is partial to what I needed:
var cookieJar = new CookieContainer();
var client = new RestClient("https://xxxxxxx")
{
Authenticator = new HttpBasicAuthenticator("[username]", "[password]"),
CookieContainer = cookieJar
};
Then right after I made the request object I had to add the API_KEY header:
var request = new RestRequest("[api]/[method]"); // this will be unique to what you are connecting to
request.AddHeader("API_KEY", "[THIS IS THE API KEY]");
I'm using VS2010 +.NET 4.0 + System.Net.Http (from Nuget).
For a reason which I don't manage to understand, the session cookie which I receive in my HttpResponseMessage is not automatically saved in the HttpClient CookieContainer.
Here is what my code looks like:
CookieContainer cookies = new CookieContainer();
HttpClientHandler handler = new HttpClientHandler();
handler.CookieContainer = cookies;
HttpClient client = new HttpClient(handler);
Uri site = new Uri("https://www.mywebsite.com");
var response1 = client.SendAsync(new HttpRequestMessage(HttpMethod.Get,site)).Result;
I can see in the response headers that I have the following:
Set-Cookie: JSESSIONID=FC8110E434C2C6DAB78B4E335024A639; Path=/member; Secure
However my cookie container remains empty ...why ?
Use this piece of code to retrieve cookies from response:
/// <summary>
/// Read web cookies
/// </summary>
public static CookieContainer ReadCookies(this HttpResponseMessage response)
{
var pageUri = response.RequestMessage.RequestUri;
var cookieContainer = new CookieContainer();
IEnumerable<string> cookies;
if (response.Headers.TryGetValues("set-cookie", out cookies))
{
foreach (var c in cookies)
{
cookieContainer.SetCookies(pageUri, c);
}
}
return cookieContainer;
}
I guess the problem is that your cookies are secure. The problem is that, CookieContainer won't send secure cookies back to the server in subsequent HTTP requests. It might be a bug, or maybe it has some reasons behind it.
A workaround is to re-add the cookie to CookieContainer manually. This way, cookie would be sent back in HTTP request header, as no secure would be defined when you send cookies back to the server.
See this article for more information.
I had problem with cookies because of case difference in path. I logged in to /mysite/login, and set cookie for mysite, but then redirect to /MySite, and HttpClient suddenly discard all the cookies! When I changed adress to /MySite/login all become ok.
Perhaps the problem is that your request Url path is to the root of the web site ("/"), but your Set-Cookie header indicates a Path of "/member".
Make sure you are automatically decompressing the response:
HttpClientHandler handler = new HttpClientHandler()
{
AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
};
Example:
var cookieContainer = new CookieContainer();
var clientHandler = new HttpClientHandler {
AllowAutoRedirect = true,
UseCookies = true,
CookieContainer = cookieContainer,
AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate,
};
var httpClient = new HttpClient(clientHandler);
var httpRequest = new HttpRequestMessage(HttpMethod.Get, "https://example.com/");
var response = httpClient.Send(httpRequest);
response.Headers.TryGetValues("set-cookie", out var cookies);
string test = (new StreamReader(response.Content.ReadAsStream()).ReadToEnd());
Edit
This also populates my cookieContainer automatically. On both a httpClient.GetAsync("https://example.com/") or SendAsync(httpRequest)