I'm trying to transmit some data from my application to a specific web service using HttpClient. To do that I first have to login to the web service and receive the cookie (that's the authentication method used by the web service). I do it like that:
Uri uri = "login_uri";
CookieContainer CookieContainer_login = new CookieContainer();
HttpClientHandler ch = new HttpClientHandler
{
AllowAutoRedirect = true,
CookieContainer = CookieContainer_login,
UseCookies = true
};
HttpClient client = new HttpClient(ch);
List<KeyValuePair<string, string>> pairs = new List<KeyValuePair<string, string>>
{
new KeyValuePair<string, string>("user", "test"),
new KeyValuePair<string, string>("password", "test"),
new KeyValuePair<string, string>("loginSource", "0")
};
FormUrlEncodedContent content = new FormUrlEncodedContent(pairs);
System.Threading.Tasks.Task<HttpResponseMessage> response = client.PostAsync(uri, content);
It works, I receive the message about successful login via Fiddler. Now in order to use the web service (another Uri), for example to send a POST request, I have to pass my cookies (received during login process) to that request. As I am storing the cookies in the CookieContainer called CookieContainer_login I thought, that I can simply use the same client and only change the Uri in the PostAsync method or create a new client with the same HttpClientHandler and CookieContainer. Unfortunately it didn't work. Actually, I found out, that my CookieContainer is empty, even after the login process.
I tried to recreate that with HttpWebRequest like that:
string url_login = "login_uri";
string logparam = "user=test&password=test&loginSource=0";
HttpWebRequest loginRequest = (HttpWebRequest)WebRequest.Create(url_login);
loginRequest.ContentType = "application/x-www-form-urlencoded";
loginRequest.Accept = "text/xml";
loginRequest.Method = "POST";
loginRequest.CookieContainer = CookieContainer_login;
byte[] byteArray = Encoding.UTF8.GetBytes(logparam);
loginRequest.ContentLength = byteArray.Length;
Stream dataStream_login = loginRequest.GetRequestStream();
dataStream_login.Write(byteArray, 0, byteArray.Length);
It works, I also receive the successful login message, but also when I check the CookieContainer count, it shows 3 cookies that are being stored after login. Now my question is why with HttpClient there are no cookies in CookieContainer, but with the HttpWebRequest there are? How to get the cookies with the HttpClient as well?
Okay, I managed to solve my problem and hopefully my answer will be useful to someone with the similar issue. In my case the mistake was in the method PostAsync invocation. It's an asynchronous method so it needs an await operator that I was missing. The proper method invocation should look like this:
HttpResponseMessage response = new HttpResponseMessage();
response = await client.PostAsync(uri, content);
Now all the cookies are stored in my CookieContainer.
Related
At the moment I am using the following RestSharp request to get a website's content:
var client = new RestClient(productLink);
var request = new RestRequest(Method.GET);
request.AddHeader("Cookie", "insert-cookie-content");
IRestResponse response = client.Execute(request);
Console.WriteLine(response.Content);
I have tried converting it into HttpClient as i will need to use the AllowRedirect property later:
var client = new HttpClient();
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Add("Cookie", "insert-cookie-content");
var response = await client.GetAsync(productUrl);
Console.WriteLine(response);
The URL I am trying to get a response from is: https://www.nike.com/sg/launch/t/air-max-90-orange-duck-camo
My first problem is that the HttpClient request is giving me 403 Errors whereas the RestClient request was working fine. How can I fix this?
My second problem is that the cookie expires after a couple of uses, and I have to manually get a new one from postman and insert it. Is there anyway for the request to generate its own cookie?
Here is the two fiddler responses compared: https://imgur.com/a/bZo7d9F
In case of HttpClient if you want to pass the Cookies manually through the DefaultRequestHeaders then you have to tell this to the HttpClient to do NOT use CookieContainer. You have to use HttpClientHandler's UseCookie flag to indicate it.
var client = new HttpClient(new HttpClientHandler { UseCookies = false });
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Add("Cookie", "insert-cookie-content");
var response = await client.GetAsync(productUrl);
Console.WriteLine(response);
Trying to call an API from a controller using HttpClient and the API does not recognize the user as authenticated and logged in. When calling the API from JS I have no issue. I noticed the HttpClient was only sending via HTTP 1.1 and so I upgraded to 2.0 settings the DOTNET_SYSTEM_NET_HTTP_USESOCKETSHTTPHANDLER flag but this made no difference. I have tried all combinations of the HttpClientHandler properties including UseCookies and the request is never authenticated.
using (var handler = new HttpClientHandler {UseDefaultCredentials = true})
{
using (var httpClient = new HttpClient(handler))
{
var response = httpClient.GetStringAsync(new Uri($"https://localhost:64366/api/")).Result;
}
}
Will move to token based auth in the future but for now would like to understand why there is a difference between calling the API from C# vs JS. This is all HTTPS on localhost using asp net core 2.2.
Difference between JS and C# is that browsers attach cookies automatically to requests and you have to attach cookies manually in C# as juunas mentioned.
To obtain and use authentication cookie you may use the following pattern
CookieContainer cookies = new CookieContainer(); //this container saves cookies from responses and send them in requests
var handler = new HttpClientHandler
{
CookieContainer = cookies
};
var client = new HttpClient(handler);
string authUrl = ""; //your auth url
string anyUrl = ""; //any url that requires you to be authenticated
var authContent = new FormUrlEncodedContent(
new List<KeyValuePair<string, string>> {
new KeyValuePair<string, string>("login", "log_in"),
new KeyValuePair<string, string>("password", "pass_word")
}
);
//cookies will be set on this request
HttpResponseMessage auth = await client.PostAsync(authUrl, authContent);
auth.EnsureSuccessStatusCode(); //retrieving result is not required but you will know if something goes wrong on authentication
//and here retrieved cookies will be used
string result = await client.GetStringAsync(anyUrl);
Running a UWP app*
So I have an HttpClient and it's associated handler. I am making a request to a website, passing in specified headers, and using a specified CookieContainer, which is empty at the beginning of the request.
When I send the request, Fiddler shows extra cookies being sent that I have not added. Where are they coming from?
CookieContainer cookieJar = new CookieContainer();
HttpClientHandler handler = new HttpClientHandler( );
handler.UseDefaultCredentials = false;
handler.CookieContainer = cookieJar;
handler.Proxy = null;
using (var client = new HttpClient(handler as HttpClientHandler))
{
HttpResponseMessage response = new HttpResponseMessage();
String loginUrl = Const.BUNGIE_LOGIN_URI + (provider == Const.XBOX_PLATFORM ? Const.XBOX_LOGIN : Const.PS_LOGIN);
client.BaseAddress = new Uri(loginUrl);
//client.Timeout = new TimeSpan(0, 0, 0, 0, 450);
client.DefaultRequestHeaders.Add("Referer", "http://www.bungie.net");
client.DefaultRequestHeaders.Add("User-Agent", Const.USERAGENT);
client.DefaultRequestHeaders.Add("X-API-Key", Const.X_API_KEY);
client.DefaultRequestHeaders.Add("Connection", "Keep-Alive");
client.DefaultRequestHeaders.Add("Accept", "application/json");
client.DefaultRequestHeaders.Add("Accept-Language", "en-GB,en-US;q=0.8,en;q=0.6");
handler.AutomaticDecompression = DecompressionMethods.GZip;
response = client.GetAsync("").Result;
response.ReadCookies(); //adds cookies to cookieJar
}
What fiddler shows
Now, as the associated CookieContainer is empty before the request is made, where are these cookies coming from? Are they accessible? If I wanted the values from them, how would I obtain them?
Edit: Where are they being added to my HttpClient request from? Does HttpClient have a common CookieContainer / cache? I have two separate HttpClient instances, and when Client (A) makes a request, it received a "set-cookie" header back, setting a "bungled" cookie.
Later on, a separate instance of HttpClient, Client (B), makes a request to the same website, sending the cookie set within Client (A).
I did not explicitly append this cookie to Client (B)'s request, so how is it being added?
On Windows 10 build 10240, System.Net.Http.HttpClient is a wrapper on top of Windows.Web.Http.HttpClient (more info here), and so, cookies now work little bit different.
To delete those extra cookies, you will need to delete the cookies using a HttpCookieManager:
HttpBaseProtocolFilter filter = new HttpBaseProtocolFilter();
HttpCookieManager cookieManager = filter.CookieManager;
foreach (HttpCookie cookie in cookieManager.GetCookies(uri))
{
cookieManager.DeleteCookie();
}
I'm using VS2010 +.NET 4.0 + System.Net.Http (from Nuget).
For a reason which I don't manage to understand, the session cookie which I receive in my HttpResponseMessage is not automatically saved in the HttpClient CookieContainer.
Here is what my code looks like:
CookieContainer cookies = new CookieContainer();
HttpClientHandler handler = new HttpClientHandler();
handler.CookieContainer = cookies;
HttpClient client = new HttpClient(handler);
Uri site = new Uri("https://www.mywebsite.com");
var response1 = client.SendAsync(new HttpRequestMessage(HttpMethod.Get,site)).Result;
I can see in the response headers that I have the following:
Set-Cookie: JSESSIONID=FC8110E434C2C6DAB78B4E335024A639; Path=/member; Secure
However my cookie container remains empty ...why ?
Use this piece of code to retrieve cookies from response:
/// <summary>
/// Read web cookies
/// </summary>
public static CookieContainer ReadCookies(this HttpResponseMessage response)
{
var pageUri = response.RequestMessage.RequestUri;
var cookieContainer = new CookieContainer();
IEnumerable<string> cookies;
if (response.Headers.TryGetValues("set-cookie", out cookies))
{
foreach (var c in cookies)
{
cookieContainer.SetCookies(pageUri, c);
}
}
return cookieContainer;
}
I guess the problem is that your cookies are secure. The problem is that, CookieContainer won't send secure cookies back to the server in subsequent HTTP requests. It might be a bug, or maybe it has some reasons behind it.
A workaround is to re-add the cookie to CookieContainer manually. This way, cookie would be sent back in HTTP request header, as no secure would be defined when you send cookies back to the server.
See this article for more information.
I had problem with cookies because of case difference in path. I logged in to /mysite/login, and set cookie for mysite, but then redirect to /MySite, and HttpClient suddenly discard all the cookies! When I changed adress to /MySite/login all become ok.
Perhaps the problem is that your request Url path is to the root of the web site ("/"), but your Set-Cookie header indicates a Path of "/member".
Make sure you are automatically decompressing the response:
HttpClientHandler handler = new HttpClientHandler()
{
AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
};
Example:
var cookieContainer = new CookieContainer();
var clientHandler = new HttpClientHandler {
AllowAutoRedirect = true,
UseCookies = true,
CookieContainer = cookieContainer,
AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate,
};
var httpClient = new HttpClient(clientHandler);
var httpRequest = new HttpRequestMessage(HttpMethod.Get, "https://example.com/");
var response = httpClient.Send(httpRequest);
response.Headers.TryGetValues("set-cookie", out var cookies);
string test = (new StreamReader(response.Content.ReadAsStream()).ReadToEnd());
Edit
This also populates my cookieContainer automatically. On both a httpClient.GetAsync("https://example.com/") or SendAsync(httpRequest)
I'm trying to use an HttpClient for a third-party service that requires basic HTTP authentication. I am using the AuthenticationHeaderValue. Here is what I've come up with so far:
HttpRequestMessage<RequestType> request =
new HttpRequestMessage<RequestType>(
new RequestType("third-party-vendor-action"),
MediaTypeHeaderValue.Parse("application/xml"));
request.Headers.Authorization = new AuthenticationHeaderValue(
"Basic", Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes(
string.Format("{0}:{1}", "username", "password"))));
var task = client.PostAsync(Uri, request.Content);
ResponseType response = task.ContinueWith(
t =>
{
return t.Result.Content.ReadAsAsync<ResponseType>();
}).Unwrap().Result;
It looks like the POST action works fine, but I don't get back the data I expect. Through some trial and error, and ultimately using Fiddler to sniff the raw traffic, I discovered the authorization header isn't being sent.
I've seen this, but I think I've got the authentication scheme specified as a part of the AuthenticationHeaderValue constructor.
Is there something I've missed?
Your code looks like it should work - I remember running into a similar problem setting the Authorization headers and solved by doing a Headers.Add() instead of setting it:
request.Headers.Add("Authorization", "Basic " + Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes(string.Format("{0}:{1}", "username", "password"))));
UPDATE:
It looks like when you do a request.Content, not all headers are being reflected in the content object. You can see this by inspecting request.Headers vs request.Content.Headers. One thing you might want to try is to use SendAsync instead of PostAsync. For example:
HttpRequestMessage<RequestType> request =
new HttpRequestMessage<RequestType>(
new RequestType("third-party-vendor-action"),
MediaTypeHeaderValue.Parse("application/xml"));
request.Headers.Authorization =
new AuthenticationHeaderValue(
"Basic",
Convert.ToBase64String(
System.Text.ASCIIEncoding.ASCII.GetBytes(
string.Format("{0}:{1}", "username", "password"))));
request.Method = HttpMethod.Post;
request.RequestUri = Uri;
var task = client.SendAsync(request);
ResponseType response = task.ContinueWith(
t =>
{ return t.Result.Content.ReadAsAsync<ResponseType>(); })
.Unwrap().Result;
This would also work and you wouldn't have to deal with the base64 string conversions:
var handler = new HttpClientHandler();
handler.Credentials = new System.Net.NetworkCredential("username", "password");
var client = new HttpClient(handler);
...
Try setting the header on the client:
DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", Convert.ToBase64String(Encoding.ASCII.GetBytes(String.Format("{0}:{1}", userName, password))));
This works for me.
Also, consider that Redirect-Handler will clear the Authorization header if your request gets redirected.
So if you call an HTTP endpoint and it redirected to the HTTPS one, you will lose your authorization header.
request.Headers.Authorization = null;
Framework: .NET v6.0
Actually your problem is with PostAsync- you should use SendAsync. In your code - client.PostAsync(Uri, request.Content); sends only the content the request message headers are not included.
The proper way is:
HttpRequestMessage message = new HttpRequestMessage(HttpMethod.Post, url)
{
Content = content
};
message.Headers.Authorization = new AuthenticationHeaderValue("Basic", credentials);
httpClient.SendAsync(message);