We Keep Coding

Is it possible to generate headers automatically with HttpClient / RestSharp?

At the moment I am using the following RestSharp request to get a website's content:
var client = new RestClient(productLink);
var request = new RestRequest(Method.GET);
request.AddHeader("Cookie", "insert-cookie-content");
IRestResponse response = client.Execute(request);
Console.WriteLine(response.Content);
I have tried converting it into HttpClient as i will need to use the AllowRedirect property later:
var client = new HttpClient();
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Add("Cookie", "insert-cookie-content");
var response = await client.GetAsync(productUrl);
Console.WriteLine(response);
The URL I am trying to get a response from is: https://www.nike.com/sg/launch/t/air-max-90-orange-duck-camo
My first problem is that the HttpClient request is giving me 403 Errors whereas the RestClient request was working fine. How can I fix this?
My second problem is that the cookie expires after a couple of uses, and I have to manually get a new one from postman and insert it. Is there anyway for the request to generate its own cookie?
Here is the two fiddler responses compared: https://imgur.com/a/bZo7d9F

In case of HttpClient if you want to pass the Cookies manually through the DefaultRequestHeaders then you have to tell this to the HttpClient to do NOT use CookieContainer. You have to use HttpClientHandler's UseCookie flag to indicate it.
var client = new HttpClient(new HttpClientHandler { UseCookies = false });
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Add("Cookie", "insert-cookie-content");
var response = await client.GetAsync(productUrl);
Console.WriteLine(response);

How to store cookies using HttpClient in C#?

I'm trying to transmit some data from my application to a specific web service using HttpClient. To do that I first have to login to the web service and receive the cookie (that's the authentication method used by the web service). I do it like that:
Uri uri = "login_uri";
CookieContainer CookieContainer_login = new CookieContainer();
HttpClientHandler ch = new HttpClientHandler
{
AllowAutoRedirect = true,
CookieContainer = CookieContainer_login,
UseCookies = true
};
HttpClient client = new HttpClient(ch);
List<KeyValuePair<string, string>> pairs = new List<KeyValuePair<string, string>>
{
new KeyValuePair<string, string>("user", "test"),
new KeyValuePair<string, string>("password", "test"),
new KeyValuePair<string, string>("loginSource", "0")
};
FormUrlEncodedContent content = new FormUrlEncodedContent(pairs);
System.Threading.Tasks.Task<HttpResponseMessage> response = client.PostAsync(uri, content);
It works, I receive the message about successful login via Fiddler. Now in order to use the web service (another Uri), for example to send a POST request, I have to pass my cookies (received during login process) to that request. As I am storing the cookies in the CookieContainer called CookieContainer_login I thought, that I can simply use the same client and only change the Uri in the PostAsync method or create a new client with the same HttpClientHandler and CookieContainer. Unfortunately it didn't work. Actually, I found out, that my CookieContainer is empty, even after the login process.
I tried to recreate that with HttpWebRequest like that:
string url_login = "login_uri";
string logparam = "user=test&password=test&loginSource=0";
HttpWebRequest loginRequest = (HttpWebRequest)WebRequest.Create(url_login);
loginRequest.ContentType = "application/x-www-form-urlencoded";
loginRequest.Accept = "text/xml";
loginRequest.Method = "POST";
loginRequest.CookieContainer = CookieContainer_login;
byte[] byteArray = Encoding.UTF8.GetBytes(logparam);
loginRequest.ContentLength = byteArray.Length;
Stream dataStream_login = loginRequest.GetRequestStream();
dataStream_login.Write(byteArray, 0, byteArray.Length);
It works, I also receive the successful login message, but also when I check the CookieContainer count, it shows 3 cookies that are being stored after login. Now my question is why with HttpClient there are no cookies in CookieContainer, but with the HttpWebRequest there are? How to get the cookies with the HttpClient as well?

Okay, I managed to solve my problem and hopefully my answer will be useful to someone with the similar issue. In my case the mistake was in the method PostAsync invocation. It's an asynchronous method so it needs an await operator that I was missing. The proper method invocation should look like this:
HttpResponseMessage response = new HttpResponseMessage();
response = await client.PostAsync(uri, content);
Now all the cookies are stored in my CookieContainer.

HttpClient PCL Cookies Not Being Applied to Requests

I'm using .NET HttpClient for PCL (2.2.15) for a common library across Windows Phone 8, Windows 8, and .NET 4.5. However, I don't see cookies being applied to requests.
I'm setting up the HttpClient and HttpClientHandler using the following code. HttpClient is a property on the wrapping class.
CookieContainer = new CookieContainer();
var handler = new HttpClientHandler
{
AllowAutoRedirect = true,
CookieContainer = CookieContainer,
Credentials = client.CredentialCache,
UseCookies = true,
AutomaticDecompression =
DecompressionMethods.Deflate | DecompressionMethods.GZip
};
HttpClient = new HttpClient(handler, false);
Requests are being sent using the following.
var response = await HttpClient.SendAsync(httpRequestMessage);
I'm able to peek into the CookieContainer on the handler when stepping through in Debug and am able to see the CookieContainer for the domain I'm working against with the expected cookie.
The cookie has the following values:
Path = '/'
Domain = '.staging3.api.com'
Value = [authentication_id]
Name = 'API_AuthId'
The subsequent request is roughly: https://staging3.api.com/api/v3/Items
I've also tried setting the cookie myself, however CookieContainer.GetCookieHeader(requestUri) returns an empty string.
Additional information:
If I modify the requestUri to https://www.staging3.api.com/api/v3/Items I'm able to get the cookie header from CookieContainer.GetCookieHeader(requestUri)

HttpClient not storing cookies in CookieContainer

I'm using VS2010 +.NET 4.0 + System.Net.Http (from Nuget).
For a reason which I don't manage to understand, the session cookie which I receive in my HttpResponseMessage is not automatically saved in the HttpClient CookieContainer.
Here is what my code looks like:
CookieContainer cookies = new CookieContainer();
HttpClientHandler handler = new HttpClientHandler();
handler.CookieContainer = cookies;
HttpClient client = new HttpClient(handler);
Uri site = new Uri("https://www.mywebsite.com");
var response1 = client.SendAsync(new HttpRequestMessage(HttpMethod.Get,site)).Result;
I can see in the response headers that I have the following:
Set-Cookie: JSESSIONID=FC8110E434C2C6DAB78B4E335024A639; Path=/member; Secure
However my cookie container remains empty ...why ?

Use this piece of code to retrieve cookies from response:
/// <summary>
/// Read web cookies
/// </summary>
public static CookieContainer ReadCookies(this HttpResponseMessage response)
{
var pageUri = response.RequestMessage.RequestUri;
var cookieContainer = new CookieContainer();
IEnumerable<string> cookies;
if (response.Headers.TryGetValues("set-cookie", out cookies))
{
foreach (var c in cookies)
{
cookieContainer.SetCookies(pageUri, c);
}
}
return cookieContainer;
}

I guess the problem is that your cookies are secure. The problem is that, CookieContainer won't send secure cookies back to the server in subsequent HTTP requests. It might be a bug, or maybe it has some reasons behind it.
A workaround is to re-add the cookie to CookieContainer manually. This way, cookie would be sent back in HTTP request header, as no secure would be defined when you send cookies back to the server.
See this article for more information.

I had problem with cookies because of case difference in path. I logged in to /mysite/login, and set cookie for mysite, but then redirect to /MySite, and HttpClient suddenly discard all the cookies! When I changed adress to /MySite/login all become ok.

Perhaps the problem is that your request Url path is to the root of the web site ("/"), but your Set-Cookie header indicates a Path of "/member".

Make sure you are automatically decompressing the response:
HttpClientHandler handler = new HttpClientHandler()
{
AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
};
Example:
var cookieContainer = new CookieContainer();
var clientHandler = new HttpClientHandler {
AllowAutoRedirect = true,
UseCookies = true,
CookieContainer = cookieContainer,
AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate,
};
var httpClient = new HttpClient(clientHandler);
var httpRequest = new HttpRequestMessage(HttpMethod.Get, "https://example.com/");
var response = httpClient.Send(httpRequest);
response.Headers.TryGetValues("set-cookie", out var cookies);
string test = (new StreamReader(response.Content.ReadAsStream()).ReadToEnd());
Edit
This also populates my cookieContainer automatically. On both a httpClient.GetAsync("https://example.com/") or SendAsync(httpRequest)

HttpClient authentication header not getting sent

I'm trying to use an HttpClient for a third-party service that requires basic HTTP authentication. I am using the AuthenticationHeaderValue. Here is what I've come up with so far:
HttpRequestMessage<RequestType> request =
new HttpRequestMessage<RequestType>(
new RequestType("third-party-vendor-action"),
MediaTypeHeaderValue.Parse("application/xml"));
request.Headers.Authorization = new AuthenticationHeaderValue(
"Basic", Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes(
string.Format("{0}:{1}", "username", "password"))));
var task = client.PostAsync(Uri, request.Content);
ResponseType response = task.ContinueWith(
t =>
{
return t.Result.Content.ReadAsAsync<ResponseType>();
}).Unwrap().Result;
It looks like the POST action works fine, but I don't get back the data I expect. Through some trial and error, and ultimately using Fiddler to sniff the raw traffic, I discovered the authorization header isn't being sent.
I've seen this, but I think I've got the authentication scheme specified as a part of the AuthenticationHeaderValue constructor.
Is there something I've missed?

Your code looks like it should work - I remember running into a similar problem setting the Authorization headers and solved by doing a Headers.Add() instead of setting it:
request.Headers.Add("Authorization", "Basic " + Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes(string.Format("{0}:{1}", "username", "password"))));
UPDATE:
It looks like when you do a request.Content, not all headers are being reflected in the content object. You can see this by inspecting request.Headers vs request.Content.Headers. One thing you might want to try is to use SendAsync instead of PostAsync. For example:
HttpRequestMessage<RequestType> request =
new HttpRequestMessage<RequestType>(
new RequestType("third-party-vendor-action"),
MediaTypeHeaderValue.Parse("application/xml"));
request.Headers.Authorization =
new AuthenticationHeaderValue(
"Basic",
Convert.ToBase64String(
System.Text.ASCIIEncoding.ASCII.GetBytes(
string.Format("{0}:{1}", "username", "password"))));
request.Method = HttpMethod.Post;
request.RequestUri = Uri;
var task = client.SendAsync(request);
ResponseType response = task.ContinueWith(
t =>
{ return t.Result.Content.ReadAsAsync<ResponseType>(); })
.Unwrap().Result;

This would also work and you wouldn't have to deal with the base64 string conversions:
var handler = new HttpClientHandler();
handler.Credentials = new System.Net.NetworkCredential("username", "password");
var client = new HttpClient(handler);
...

Try setting the header on the client:
DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", Convert.ToBase64String(Encoding.ASCII.GetBytes(String.Format("{0}:{1}", userName, password))));
This works for me.

Also, consider that Redirect-Handler will clear the Authorization header if your request gets redirected.
So if you call an HTTP endpoint and it redirected to the HTTPS one, you will lose your authorization header.
request.Headers.Authorization = null;
Framework: .NET v6.0

Actually your problem is with PostAsync- you should use SendAsync. In your code - client.PostAsync(Uri, request.Content); sends only the content the request message headers are not included.
The proper way is:
HttpRequestMessage message = new HttpRequestMessage(HttpMethod.Post, url)
{
Content = content
};
message.Headers.Authorization = new AuthenticationHeaderValue("Basic", credentials);
httpClient.SendAsync(message);