In HttpClientHandler have propertie AllowAutoRedirect, but on build app for WindowsPhone throw exception:
HttpClientHandler.AllowAutoRedirect is not supported on this platform. Please check HttpClientHandler.SupportsRedirectConfiguration before using HttpClientHandler.AllowAutoRedirect.
I really want to prevent autoredirect.
I tried to use HttpWebRequest:
var client = (HttpWebRequest) WebRequest.Create(connectionUrl);
client.Headers["AllowAutoRedirect"] = "false";
client.Method = "GET";
client.Headers["UserAgent"] = #"Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.43 Safari/537.31";
client.ContentType = "application/json";
client.Headers["ContentLength"] = client.ToString().Length.ToString();
client.BeginGetResponse(Callback, client);
private void Callback(IAsyncResult ar)
{
var requestState =(HttpWebRequest) ar.AsyncState;
using (var postStream = requestState.EndGetRequestStream(ar))
{}
}
this code throw exception on EndGetRequestStream: "Value does not fall within the expected range"
I look forward to your help.
I think you are receiving the ArgumentException: Value does not fall within the expected range because you are initiating a BeginGetResponse() on the client but then doing a EndGetRequestStream() in your callback where instead you should call EndGetResponse(). Setting AllowAutoRedirect works fine you just need to fix your code. Try this:
var client = (HttpWebRequest)WebRequest.Create(connectionUrl);
client.AllowAutoRedirect = false;
client.Method = "GET";
client.BeginGetResponse(Callback, client);
private void Callback(IAsyncResult ar) {
var state = (HttpWebRequest)ar.AsyncState;
using (var response = state.EndGetResponse(ar)) {
var streamResponse = response.GetResponseStream();
var streamRead = new StreamReader(streamResponse);
var responseString = streamRead.ReadToEnd();
}
}
Why not use HttpClient with and HttpClientHandler??
HttpClientHandler clientHandler = new HttpClientHandler();
clientHandler.AllowAutoRedirect = false;
// Create an HttpClient using the HttpClientHandler
HttpClient client = new HttpClient(clientHandler);
More info: http://blogs.msdn.com/b/henrikn/archive/2012/08/07/httpclient-httpclienthandler-and-httpwebrequesthandler.aspx
Related
My method to get tokenKey is :
HttpWebRequest httpRequest = (HttpWebRequest)WebRequest.Create(url);
httpRequest.Method = "POST";
httpRequest.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36 OPR/52.0.2871.99";
string tokenResponse = null;
HttpClient client = new HttpClient();
HttpResponseMessage response = null;
try
{
client.DefaultRequestHeaders.Add("x-api-key", "key");
if (method.Equals("POST"))
{
httpRequest.Accept = "application/json";
httpRequest.ContentType = "application/json";
var data = #"{""username"":#"""+ login + #""",""password"" :#"""+ password + #"""}";
using (var streamWriter = new StreamWriter(httpRequest.GetRequestStream()))
{
streamWriter.Write(data);
}
var httpResponse = (HttpWebResponse)httpRequest.GetResponse();
using (var streamReader = new StreamReader(httpResponse.GetResponseStream()))
{
tokenResponse = streamReader.ReadToEnd();
}
}
}
catch (Exception ex)
{
throw new Exception(ex.Message);
}
return tokenResponse;
after var httpResponse = (HttpWebResponse)httpRequest.GetResponse();
this error msg: The underlying connection was closed: Unexpected error on a send.
x-api-key is okay but i don't put the original as the username and password
i do it easy with postman
postman request done
Can anyone help me understand where I'm going wrong?
Could this be a case of mismatch TLS versions? See this answer here:
C# HttpWebRequest The underlying connection was closed: An unexpected error occurred on a send
Also it looks like your HttpWebRequest won't have the correct headers, because you are setting them on the HttpClient object instead. The HttpClient also doesn't seem to be being used for this call.
See documentation, how to use HttpWebRequest and how to write body properly to request. I think the problem is with a request stream.
I'm getting the file size of remote urls and I just noticed the difference between HttpClient and httpWebRequest.
I compared and I noticed that httpclient is taking too much data.
this is a big issue for me because, in the Philippines we are only have limited data
Could you please tell me what's wrong with my httpclient class? I can't figure out what is causing the high data usage
HttpClient
HttpClientHandler handler = new HttpClientHandler()
{
Proxy = null,
UseProxy = false,
AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
};
var client = new HttpClient(handler);
client.DefaultRequestHeaders.Add("Method", "GET");
client.DefaultRequestHeaders.Add("Referer", uriPath.AbsoluteUri);
client.DefaultRequestHeaders.Add("Origin", "https://www.samplesite.com");
client.DefaultRequestHeaders.ConnectionClose = true;
client.DefaultRequestHeaders.UserAgent.ParseAdd("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36");
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
client.DefaultRequestHeaders.AcceptEncoding.Add(new StringWithQualityHeaderValue("gzip"));
client.DefaultRequestHeaders.AcceptEncoding.Add(new StringWithQualityHeaderValue("deflate"));
using (HttpResponseMessage response = await client.GetAsync(uriPath, HttpCompletionOption.ResponseHeadersRead, token).ConfigureAwait(false))
{
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
var resultTask = response.Content.ReadAsStringAsync();
var timeoutTask = Task.Delay(3000);
var completed = await Task.WhenAny(resultTask, timeoutTask).ConfigureAwait(false);
if (completed == timeoutTask)
return null;
return await resultTask;
}
HttpWebRequest
var webRequest = (HttpWebRequest)WebRequest.Create(uriPath);
webRequest.Method = "HEAD";
webRequest.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36";
using (var webResponse = await webRequest.GetResponseAsync())
{
return await Task.Run(() => webResponse.Headers.Get("Content-Length"), token);
}
You are using different HTTP methods GET in case of HttpClient & HEAD in case of WebRequest. To get file size you will enough HEAD method in both cases
The HTTP GET method requests a representation of the specified resource. Requests using GET should only retrieve data.
The HTTP HEAD method requests the headers that are returned if the specified resource would be requested with an HTTP GET method. Such a request can be done before deciding to download a large resource to save bandwidth, for example.
You need to change this code line
client.DefaultRequestHeaders.Add("Method", "GET");
it MUST BE
client.DefaultRequestHeaders.Add("Method", "HEAD");
A response to a HEAD method does not have a body in contradistinction to GET
UPD: use SendAsync method (not GetAsync)
HttpClientHandler handler = new HttpClientHandler();
using var client = new HttpClient(handler);
string requestUri = "enter request uri";
HttpRequestMessage message = new HttpRequestMessage(HttpMethod.Head, requestUri);
using var response = await client.SendAsync(message);
Finally it's solved Big thanks to #Dmitry
Here's the HttpClient updated code
public static async Task<string> GetTotalBytes(Uri uriPath)
{
HttpClientHandler handler = new HttpClientHandler();
handler.Proxy = null;
using (var client = new HttpClient(handler))
{
HttpRequestMessage message = new HttpRequestMessage(HttpMethod.Head, uriPath);
using (var response = await client.SendAsync(message))
{
response.EnsureSuccessStatusCode();
var lenght = response.Content.Headers.ContentLength;
return lenght.ToString();
}
}
}
RESULT (HttpClient):
Total URL: 355
Filsize: 1.14 GB
Elapsed Time: 0h 0m 2s 51.3ms
and here's HttpWebRequest (Maybe someone will need)
var webRequest = (HttpWebRequest)WebRequest.Create(uriPath);
webRequest.Method = "HEAD";
webRequest.Proxy = null;
using (var webResponse = await webRequest.GetResponseAsync())
{
return await Task.Run(() => webResponse.Headers.Get("Content-Length"), token);
}
This issue has kept me busy for days now, so I might just as well post it here... searching in other topics has brought me far, but I'm not there yet.
Background
My job is to automate tests in the browser: I make use of Selenium and .NET to handle this. One of our clients uses Active Directory for logging in on the web application. Logging in again for every test takes a lot of time (a new session is started for each test), so I'm trying to speed this phase up by writing a class that will login beforehand, without the browser.
By using Fiddler, I managed to reproduce most of the webrequests in code. Still, I don't work with custom web requests often, so I easily overlook possible issues. Something goes wrong in the last step, where I try to retrieve the AppServiceAuthSession cookie. I'll use a picture to illustrate:
On the left, there is the web request I make using my code. On the right, there is the web request I get when using the browser.
The Nonce cookie has its value, headers are correct... the only difference here is the "Expect" header instead of the "Connection" header, but that happens because I disabled the "AllowAutoRedirect" property for the request (I don't get to see this request in Fiddler if I keep it enabled).
Next, here comes the content of both requests, since that's hard to show by using the Fiddler interface.
Code request:
code=AQABAAIAAABHh4kmS_aKT5XrjzxRAtHzZ08Y3xv2A4HxumqfAbR5iIBFEbbm6oYUq5sdHF72ybOkXFz33XVW5uXvuq-Tiz43zQ7y08bAhGH-aR_3HEpugBMAX_Op4O0Z4RjMKOLyd5b-VNVhjqv_CRmgHTEhYnxubjNyaG-5Vsuo7YRa3jXFybBheb73Erp7CDYFCQ-dFjgNAEHEbzrfE64dGVvHCCr-rvpHCTJfW55eYE9l68d7rryVfqC8dX7wplr7A3G8ySS84Tb9_hYxA61L_fQEm22NehWGNKw_MwXWNsIONtwiRd5LQD95iXYwjCw9J1p-WoZw9p6228r8YOuBVJA0k22O_zjqvKGUkVEyVWvAZZvDSO26_m-uuz7ql52ZTfBhmOfF8iXfdgeUPUqSXmTmNtL0LAOgiYKt4DyH1TbgAN2ox-sVcFp0k7bmjE81h0yiY8RgR-a2aONvuIpSR1Dx7VG-1wkllQAkYGV8zMoahs0EVRiDdOEQwwvkegALZNaU_ROKkbxj-H71h7lffkLlZoeM1lQFwgcVyB1_kV04iZKg2x5-VxHRYCDCNdeaxuaZD5dOrxIh_Y6SfQK-VBWsDB1C_i76QQpFI2c-7oxFNn7RLCU5gw7iv8k31ry2-hKSfhcU6LtN--t-eW73ADu0cW9juh1zzgzXAPXz0FqNyh07SyRawPqir9jdZ4GlgU4r-Imm41n4b7LH_nYixE1q-JNTm7TlawW3Xe7i28QMx95wPGlqyYl0PAo0VMwMPgoG3Zi-5BiEFi31qOxCMJNbXWi-YOMMcV3W5-sNbYA44mOUoYnz0hny-PlvKFwTL9RLBxkgAA&
id_token=eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IkZTaW11RnJGTm9DMHNKWEdtdjEzbk5aY2VEYyIsImtpZCI6IkZTaW11RnJGTm9DMHNKWEdtdjEzbk5aY2VEYyJ9.eyJhdWQiOiI4NmJiNjljZC04OTQ4LTQ2ZmEtOWQ3NS05YWRiZWRmODI1NWYiLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC8xYWNhOGFmYy01YTZlLTRkYWUtOTRhNC0xODIwZDgwNGMxZWIvIiwiaWF0IjoxNTIxNzM2NTM1LCJuYmYiOjE1MjE3MzY1MzUsImV4cCI6MTUyMTc0MDQzNSwiYWlvIjoiQVRRQXkvOEdBQUFBNXhNTk10SUJybUI1Z2dTN0xRNWxmSUNBMnY3RGpBRXIyUzgwS3dBa0VlMlpacVhNQW1hMnE0eG5xeDJ5aGxNZSIsImFtciI6WyJ3aWEiXSwiY19oYXNoIjoiUlUxOHNDNjJldTUyVnd0ZG9CTEItUSIsImVtYWlsIjoiUnViZW4udGVyLk1hdGVuQG1hY2F3Lm5sIiwiaWRwIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvMDViYWVkNGQtZTA0OS00NWM3LWI0N2QtYzk5YzY2ZmRiNTg5LyIsImluX2NvcnAiOiJ0cnVlIiwiaXBhZGRyIjoiODIuMjAxLjExMi40IiwibmFtZSI6IlJ1YmVuIHRlciBNYXRlbiIsIm5vbmNlIjoiYmI2NDU1ZGQzODJkNDRmNGIyNDAwYThjYzY3NmNmNTZfMjAxODAzMjEwOTU3MzAiLCJvaWQiOiI2YjgzMGMyNi1hZGE4LTQyY2MtOTQ3MS1kMzU5NTc1N2IwNDciLCJyb2xlcyI6WyJDb250ZW50RWRpdG9yIl0sInN1YiI6InJIVlJfRFZsWjltN2V5LS02RVE4LUFGY1FVZ3VWa3VRZVdnR3ZVeGdmbE0iLCJ0aWQiOiIxYWNhOGFmYy01YTZlLTRkYWUtOTRhNC0xODIwZDgwNGMxZWIiLCJ1bmlxdWVfbmFtZSI6IlJ1YmVuLnRlci5NYXRlbkBtYWNhdy5ubCIsInV0aSI6InpoRTFLWVRhWGtxS1dWWThqQlFaQUEiLCJ2ZXIiOiIxLjAifQ.xcuSefb-pr21z53eoirQTj3nptW95rQ0RnsxkF6B436Z16bI826UK4wYLvjGl8NrXYV1-sJhlUTUuX31p90GxJthUV1tbzsvFUwNk8f9DJgKls-31X78fNbChkuNDIS9Tj6Lj0cPOlZaC6U8TXYaNsef8qURtvtXzPt7Td1t6rfvuJL29pnow0We7u-eqUR8_TDD9PzY82BdYSbfIn4X-cOw4F79O-OAce9zRebX3V1Cr53gizMUIKKDWlS8fdwKnpdluAgZup-r6RJa7c_ui0NB2REao-PxixkvO-PuCtEfLhRnbRAprVa6ggMlCrtN6Uf7EmLRGLNBGHbl379vgg&
state=redir%3d%252Flogin%252Fcheck&
session_state=4dcbbba5-e8a5-4ad7-9ba9-28f6af570ead
Browser request:
code=AQABAAIAAABHh4kmS_aKT5XrjzxRAtHzZT_hGoYPGfSq8uoVwxawNGgqfV8j3-WdO_yha1uD2rspKWZmKjbV4j3YQmg4IQOdNBKzWDYTAST-Shag2D2Ih9EHMThyu7ncE_XRSBDOb_f05q3WCKYtF5ki5VVFcd8Li029KFwEjjutGw4k-_GfPe1rym5CXRvqQzj0wGUN4Q3ndwqrcNXsdvGG6wEXbiTJ8_3aSyuAAsHBMr1l23fjJRe-CKfr7EqKDZLzm_2H6L0MODy_mWkrD715CjU2_mUb1YQ__6MZvhHss41BFPcL3zyK2oh7Z668dJMEK495MhKFt-NPUFUPXsUNwK1fP4F-tdmBIY39BL9vDxzynmcF5A63yb8q3m0CgJOZ0i1aobe1iEX75sNXjIa2385yFw08lmkVJFj9GccvIE5ak4Zi6sdjWhEtS9TyKXc5uBtKxJCYIy52avI6bUAyc5-Xq0amik0Q75wuLK0bSoeIwBofKuJnoHfl1o3_7N2uSZ_2dJx5zPW9VxRz4e6u-mkmDu9F5lK42zo3al8c03zKHhLkTVRx08JV7JZuqpLznz5JFEHp1QUq-KeYNdMXsngPZW3VEegSyu6VGYwvbmL07GYTZ3KWi-dGxej88KvobZF3O40rfISThS4gVpnFoH7UQs_RE_TEjRvUraFBUhjE-1EUd7WXl6NsxQwGO0Uw7W87ZqlY4G-JhN39MOdm4v9UZELmeZph87PWqqy9-6h1pK8wrH8S98HqV1q3DUR68Ue-zCSvV9hgxrkSnPtFDTfCgtOvpNj14GUesMi8L0XPa6DsO0hj27_A1T0STGSHMGg8QMcgAA&
id_token=eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IkZTaW11RnJGTm9DMHNKWEdtdjEzbk5aY2VEYyIsImtpZCI6IkZTaW11RnJGTm9DMHNKWEdtdjEzbk5aY2VEYyJ9.eyJhdWQiOiI4NmJiNjljZC04OTQ4LTQ2ZmEtOWQ3NS05YWRiZWRmODI1NWYiLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC8xYWNhOGFmYy01YTZlLTRkYWUtOTRhNC0xODIwZDgwNGMxZWIvIiwiaWF0IjoxNTIxNzExMTIxLCJuYmYiOjE1MjE3MTExMjEsImV4cCI6MTUyMTcxNTAyMSwiYWlvIjoiQVRRQXkvOEdBQUFBT1YreEZOcjAwR3R6TG9jdkZaMVZ0SjlIRkYyUjc2VWE4d1V0eGlISkpienU4em1nN2xtRzdPT3ZCelZ6VkpyeiIsImFtciI6WyJ3aWEiXSwiY19oYXNoIjoieThoZUw5RHh3b3ZkUV9IbmtTYzlqZyIsImVtYWlsIjoiUnViZW4udGVyLk1hdGVuQG1hY2F3Lm5sIiwiaWRwIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvMDViYWVkNGQtZTA0OS00NWM3LWI0N2QtYzk5YzY2ZmRiNTg5LyIsImluX2NvcnAiOiJ0cnVlIiwiaXBhZGRyIjoiODIuMjAxLjExMi40IiwibmFtZSI6IlJ1YmVuIHRlciBNYXRlbiIsIm5vbmNlIjoiMDk5MmE5NjQ1N2JiNDA3ZTgxZTYyZjE0YzA0ODg3ZTVfMjAxODAzMjIwOTQxNTIiLCJvaWQiOiI2YjgzMGMyNi1hZGE4LTQyY2MtOTQ3MS1kMzU5NTc1N2IwNDciLCJyb2xlcyI6WyJDb250ZW50RWRpdG9yIl0sInN1YiI6InJIVlJfRFZsWjltN2V5LS02RVE4LUFGY1FVZ3VWa3VRZVdnR3ZVeGdmbE0iLCJ0aWQiOiIxYWNhOGFmYy01YTZlLTRkYWUtOTRhNC0xODIwZDgwNGMxZWIiLCJ1bmlxdWVfbmFtZSI6IlJ1YmVuLnRlci5NYXRlbkBtYWNhdy5ubCIsInV0aSI6IlRFUGhmbG53dGs2YTFKTVJEcHNVQUEiLCJ2ZXIiOiIxLjAifQ.eT25sB0q7a5nEPBS5z2WVFHohBdzgioj6rd5DOMlb30dp1mIAVgvRZxSJhybB6RuHOl86LTkcyH4qzXl8uQ9fq8hapZUdJ2sDDqvk-PmcIWp_Ar6Gyo--4DtUxu7quIe-JaP_HyN_lsNbm0Gxut2VkZuHmCaiBxRriosP2FRg2GyKO75P6V7r0EPmQnS_4HA0WhEUmqNBzrZnbMV4rkc4FkLoMSLgJN69XdBMCQfsnhVcDWFPGj_Q79vlSbSq17Y7EPxn5phkrLGfZGMQj6CNHSzeT76LxR4txUu16QEQLLHckw2aSsESQ8SPFXwk794jEuyFIoFPrGg75hv4XxC7Q&
state=redir%3D%252Flogin%252Fcheck&
session_state=d897cb81-a5db-41f2-8852-8c18a82d36ab
Issue
As visible in the picture, the web request gets a response without AppServiceAuthSession and is redirected to the login.windows.net location. The normal browser request is different: it is successful at this point.
Question
Does anyone have a clue on what goes wrong here, or on how to debug it successfully? I triple-checked the input of the previous requests as well, but they all seem identical to the ones given to the browser. Even just a pointer in general might help... I'm getting a bit lost here. If more info is needed, just let me know. Thanks in advance!
Extra information
The code, in case it helps (censored where needed):
class Login
{
static CookieContainer cookiesNonce = new CookieContainer();
static CookieContainer cookiesA = new CookieContainer();
static CookieContainer cookiesB = new CookieContainer();
public static void DoLogin()
{
// CLIENTNAME-api, request for login with redirect
var url = #"https://CLIENTNAME-api.CLIENTNAME-dev.nl/.auth/login/aad?post_login_redirect_url=/login/check";
var request = WebRequest.Create(url) as HttpWebRequest;
request.Accept = #"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8";
request.UserAgent = #"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36";
request.Headers.Add("Upgrade-Insecure-Requests", "1");
request.CookieContainer = new CookieContainer();
request.KeepAlive = true;
request.AllowAutoRedirect = false;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
cookiesNonce.Add(response.Cookies);
cookiesA.Add(response.Cookies);
request = WebRequest.Create(response.Headers[HttpResponseHeader.Location]) as HttpWebRequest;
request.Accept = #"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8";
request.UserAgent = #"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36";
request.Headers.Add("Upgrade-Insecure-Requests", "1");
request.KeepAlive = true;
request.Host = #"login.windows.net";
request.CookieContainer = cookiesA;
}
string clientrequestid = "";
string authorizeUrl = "";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
cookiesA = new CookieContainer();
cookiesA.Add(response.Cookies);
var reader = new StreamReader(response.GetResponseStream());
string html = reader.ReadToEnd();
reader.Close();
var jsonRegEx = new Regex(#"\$Config=(.*?)(?<=\});");
var temp = jsonRegEx.Match(html).Groups[1].ToString() ;
var dict = JObject.Parse(temp);
var hpgrequestid = response.Headers["x-ms-request-id"];
var canary = GetValue(dict, "apiCanary");
clientrequestid = GetClientRequestId(dict);
authorizeUrl = response.Headers[HttpResponseHeader.Location];
// Request to microsoft online by credentials
url = #"https://login.microsoftonline.com/common/GetCredentialType";
request = WebRequest.Create(url) as HttpWebRequest;
request.Method = #"POST";
request.CookieContainer = cookiesB;
request.KeepAlive = true;
request.Headers.Add("hpgrequestid", hpgrequestid);
request.Headers.Add("canary", canary);
request.Headers.Add("client-request-id", clientrequestid);
request.Headers.Add("hpgact", "1800");
request.Headers.Add("hpgid", "1104");
request.Referer = response.ResponseUri.ToString(); ;
var reqStream = request.GetRequestStream();
var buffer = new ASCIIEncoding().GetBytes(#"{""username"":""user#COMPANYNAME.nl"",""isOtherIdpSupported"":true,""checkPhones"":false,""isRemoteNGCSupported"":false,""isCookieBannerShown"":true,""isFidoSupported"":false,""originalRequest"":""rQIIAZ2Qv08UQRzFd26PDaCJxBCD3RUQE83szM7N7s1eQoGCGjEhXAwQLch35wc33rK7zK6i_Ak2YGFjZSwvVlbGP4FGav8CYyOhsvTU-A_YvPeq93l5N_0ojPqLIsmyJJUKi5QLzBMDOFW9GKegMq2MYHFs3NXZOf7jvH3x5dX68Rs2OKYlHaPesGmquk_ISBeFw1DZ8G9S-nlY5CSEZ82Q5OWeLQiAIhLyPAM5-oTQGULfEBq3yITN41iprmCKc8MzxikFIWXSS6SJk11GI0G7LKJp3OvSr60rGyuTVvZbSmeP9EVrxpRuf7cq6-atf81pZd3yErv7BztxOdRyNPav_9u656Aahoe2UOVhHRa6-egvRiBBgJE4hkRjrkDjlAPHkWBUCcplpLNTf76sdGFVp3Klsbnu6H2w-VkbfW9fpn5_enp2zlvwOt7PNno_NfmrXrp08OHJ7dXX714ObnQD73SKkFuDxxUMDF2xxc7W_c2No3zdwc7q5kPxtLmztq22WP3i0ba992BtmfWjkwCdBMF54H2e-d-rfwE1""}");
reqStream.Write(buffer, 0, buffer.Length);
reqStream.Close();
}
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
cookiesB.Add(response.Cookies);
var reader = new StreamReader(response.GetResponseStream());
string html = reader.ReadToEnd();
reader.Close();
// Request to sts.COMPANYNAME.nl
url = #"https://sts.COMPANYNAME.nl/adfs/ls/wia?client-request-id=" + clientrequestid + #"&wa=wsignin1.0&wtrealm=urn%3afederation%3aMicrosoftOnline&wctx=LoginOptions%3D3%26estsredirect%3d2%26estsrequest%3drQIIAZ2QPU8UQRjHd26PDZwmEEOIdldATDSzOzs7uze7CYkoqBETwsUAwYLMzgs33t7uOrt4ykewAQsbK2N5sbIyfgQaqSmtjI2EytJF4xewef5P9X_53bJ9108WaZSmUcwFpDGhkESKwVj0QhgzkUqhKA5DZa515sjP8_bF19frR29x_wgVaAJ6g7ouq8TzhjLPDWSldv9-Qr5w88xz2UE98LJiX-ceY8LjLMtSxoefATgF4DsAk5bXZJMwFCKgWBCiSIoJQoxyHvUirsJoDyOfogD7KA57ATprzW6sNK748hRGH8qL1owqzGivLKr6nb1gpNBmeQnf_xPbKB9IPpzYN_513TesHLhjnYtiXLm5rD_Ziz7jjDLFYcgiCYlgEsaEEehTjARFhPsyPbHni1LmWnRLUyidya4cMZ19s2fNQSrz0Z1RYzJuZp-2wY_2VWQn09OdOeu61bV-tcGHqYZgtXTl-cend1ffvH_Vvxk41smU593u75asr9CKzne2Hm5uHGbrhu2sbj6mz-p7a9tiC1cvn2zrB4_WlnHiHzvg2HHOHevLzP_CP-vMXyKFKIDY76I4CXESkN3f0&cbcxt=&username=user%40COMPANYNAME.nl&mkt=&lc=";
request = WebRequest.Create(url) as HttpWebRequest;
cookiesB = new CookieContainer();
request.CookieContainer = cookiesB;
request.KeepAlive = true;
request.Headers.Add("Upgrade-Insecure-Requests", "1");
request.Referer = response.ResponseUri.ToString(); ;
request.UserAgent = #"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36";
request.Credentials = CredentialCache.DefaultNetworkCredentials;
request.AuthenticationLevel = System.Net.Security.AuthenticationLevel.MutualAuthRequested;
}
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
cookiesB.Add(response.Cookies);
var reader = new StreamReader(response.GetResponseStream());
string html = reader.ReadToEnd();
var doc = new HtmlDocument();
doc.LoadHtml(html);
var node = doc.DocumentNode.SelectSingleNode("//input[#name='wresult']");
var value = node.Attributes["value"].Value;
var decode = HttpUtility.HtmlDecode(value);
var wresult = HttpUtility.UrlEncode(decode);
node = doc.DocumentNode.SelectSingleNode("//input[#name='wctx']");
value = node.Attributes["value"].Value;
decode = HttpUtility.HtmlDecode(value);
var wctx = HttpUtility.UrlEncode(decode);
reader.Close();
// Logging in to microsoft online
url = #"https://login.microsoftonline.com/login.srf?client-request-id=" + clientrequestid;
request = WebRequest.Create(url) as HttpWebRequest;
request.CookieContainer = cookiesB;
request.Referer = response.ResponseUri.ToString(); ;
request.Method = #"POST";
request.Headers.Add("Upgrade-Insecure-Requests", "1");
request.KeepAlive = true;
request.UserAgent = #"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36";
request.Headers.Add("Origin", "https://sts.COMPANYNAME.nl");
request.Headers.Add("Cache-Control", "max-age=0");
request.ContentType = #"application/x-www-form-urlencoded";
var reqStream = request.GetRequestStream();
var buffer = new ASCIIEncoding().GetBytes(#"wa=wsignin1.0&wresult=" + wresult + #"&wctx=" + wctx);
reqStream.Write(buffer, 0, buffer.Length);
reqStream.Close();
}
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
cookiesB = new CookieContainer();
cookiesB.Add(response.Cookies);
var reader = new StreamReader(response.GetResponseStream());
string html = reader.ReadToEnd();
var doc = new HtmlDocument();
doc.LoadHtml(html);
reader.Close();
var state = GetValue(doc, "state");
var code = GetValue(doc, "code");
var idtoken = GetValue(doc, "id_token");
var session_state = GetValue(doc, "session_state");
// Callback to .auth/login
url = #"https://CLIENTNAME-api.CLIENTNAME-dev.nl/.auth/login/aad/callback";
request = WebRequest.Create(url) as HttpWebRequest;
request.Method = #"POST";
request.CookieContainer = new CookieContainer();
var cookie = cookiesNonce.GetCookies(new Uri("https://CLIENTNAME-api.CLIENTNAME-dev.nl"))["Nonce"];
request.CookieContainer.Add(cookie);
request.KeepAlive = true;
request.Headers.Add("Cache-Control", "max-age=0");
request.Headers.Add("Origin", "https://login.microsoftonline.com");
request.Headers.Add("Upgrade-Insecure-Requests", "1");
request.ContentType = #"application/x-www-form-urlencoded";
request.UserAgent = #"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36";
request.Accept = #"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8";
request.Referer = response.ResponseUri.ToString(); ;
request.Headers.Add("Accept-Encoding", #"gzip, deflate, br");
request.Headers.Add("Accept-Language", #"en-US,en;q=0.9");
request.AllowAutoRedirect = false;
var reqStream = request.GetRequestStream();
var buffer = new ASCIIEncoding().GetBytes(#"code=" + code + #"&id_token=" + idtoken + #"&state=" + state + "&session_state=" + session_state);
reqStream.Write(buffer, 0, buffer.Length);
reqStream.Close();
}
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
var appServiceAuthSession = response.Cookies["AppServiceAuthSession"].ToString();
var reader = new StreamReader(response.GetResponseStream());
string html = reader.ReadToEnd();
reader.Close();
}
}
private static string GetValue(HtmlDocument doc, string name)
{
var node = doc.DocumentNode.SelectSingleNode("//input[#name='" + name + #"']");
var value = node.Attributes["value"].Value;
var decode = HttpUtility.HtmlDecode(value);
var result = HttpUtility.UrlEncode(decode);
return result;
}
private static string GetValue(string key, string input, string delimiter)
{
var value = RegexHelper.GetValueForKey(input, key, delimiter);
return value;
}
private static string GetValue(JObject container, string key)
{
string value = (string)container[key];
return value;
}
private static string GetClientRequestId(JObject container)
{
var obj = container["desktopSsoConfig"];
var value = (string)obj["iwaEndpointUrlFormat"];
var content = value.Split('&');
var id = content[content.Length - 1].Replace("client-request-id=", "");
return id;
}
}
OpenID connect is intentionally unspecific about the nature of the user authentication. The premise is that the authentication service will render whatever authentication it needs to the user, and if this has been satisfactory, will issue a short lived code which is redirected to the application. The application then uses that code to obtain a token.
There are other flows for authenticating non-user facing application like services. That might not help you though if you need to emulate a user logon.
As you are on Selenium already, why not have it enter the test user credentials into the ui presented in a web browser window?
Not a real solution, but at least a useful workaround in our case:
Login by using the browser.
Navigate to the auth/me page.
Example: let's say the home url I had to use was https://company.company-dev.nl. I adjusted the url to https://company-api.company-dev.nl/.auth/me and navigated there.
Retrieve the access token from the html.
To read the token, I used the following code:
private string ConvertToken(string html)
{
var doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(html);
var timeout = DateTime.Now + TimeSpan.FromSeconds(5);
dynamic obj = null;
while (obj == null && DateTime.Now < timeout)
{
try
{
obj = (dynamic)JsonConvert.DeserializeObject(doc.DocumentNode.InnerText);
}
catch (Exception ex)
{
// Firefox occasionally bugs out if its JSON viewer kicks in too early, so use the according elements if needed
var element = Driver.FindElementById("rawdata-tab");
Driver.ClickElement(element);
var text = Driver.FindElementByClassName("data").Text;
obj = (dynamic)JsonConvert.DeserializeObject(text);
}
}
return obj[0]["access_token"].ToString();
}
Save the token.
When the next test starts, append the token to the home url.
{
NavigateToUrl(HomeUrl + #"login/" + Token);
}
Example: https://company.company-dev.nl/#/login/eyJ0eXAiOiandareallylonglineofcharactershere
This way, you can make only one test login, while the others will use the token generated by the first one. You have to play around with some other things as well (using the token while testing in parallel, dealing with token timeouts, etc.), but this is the core of the workaround.
I've created HttpClient that I'm using for sending requests:
public static void Initialize()
{
handler = new HttpClientHandler() { UseCookies = false, AllowAutoRedirect = true };
http = new HttpClient(handler) { BaseAddress = new Uri("http://csgolounge.com/mytrades") };
http.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36");
}
After that I'm creating instance of custom class that stores the cookies string for an account (something like id=xxxxxxxx; tkz=xxxxxxxxxx; token=xxxxxxxxxxx.
That's how I'm sending a post request:
public async Task Bump()
{
//if (Bumpable)
//{
var req = new HttpRequestMessage(HttpMethod.Post, "http://www.csgolounge.com/ajax/bumpTrade.php");
req.Headers.Add("Cookie", cookieString);
req.Headers.Add("X-Requested-With", "XMLHttpRequest");
req.Headers.Add("Referer", "http://csgolounge.com/mytrades"); //Not really sure if this does anything but I've run out of smart ideas long time ago
/*Dictionary<string, string> postData = new Dictionary<string, string>()
{
{"trade", offer_id}
};
var encoded = new FormUrlEncodedContent(postData);
*/
req.Content = new StringContent("&trade="+Offer_id, Encoding.UTF8, "application/x-www-form-urlencoded"); //Desperation.. decided to change the encoded dictionary to StringContent
var res = await SteamAccount.http.SendAsync(req);
var html = await res.Content.ReadAsStringAsync();
//}
}
I don't get what's wrong with this code. It seems correct to me.
Also, when I set AllowAutoRedirect = false it returns 301: Moved Permanently error, while normally it returns 200 with no HTML no matter what I pass as content.
What am I doing wrong?
Edit: Here's the JavaScript function I'm basing my request on:
function bumpTrade(trade) {
$.ajax({
type: "POST",
url: "ajax/bumpTrade.php",
data: "trade=" + trade
});
}
I've worked with more complex AJAX before, but this just doesn't seem to work no matter what I do.
Edit: I've lost my patience and switched to HttpWebRequest instead.
Now the method looks like this:
public async Task BumpLegacy()
{
while (true)
{
try
{
HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create("http://csgolounge.com/ajax/bumpTrade.php");
var cc = new CookieContainer();
MatchCollection mc = Regex.Matches(Account.CookieString, #"\s?([^=]+)=([^;]+);");
foreach (Match m in mc)
cc.Add(new Cookie(m.Groups[1].Value, m.Groups[2].Value, "/", "csgolounge.com"));
httpWebRequest.CookieContainer = cc;
byte[] bytes = Encoding.ASCII.GetBytes("trade=" + Offer_id);
httpWebRequest.Referer = "http://csgolounge.com/mytrades";
httpWebRequest.Headers.Add("X-Requested-With", "XMLHttpRequest");
httpWebRequest.Method = "POST";
httpWebRequest.ContentType = "application/x-www-form-urlencoded; charset=UTF-8";
httpWebRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36";
httpWebRequest.ContentLength = (long)bytes.Length;
var g = await httpWebRequest.GetRequestStreamAsync();
await g.WriteAsync(bytes, 0, bytes.Count());
g.Close();
var res = await httpWebRequest.GetResponseAsync();
res.Close();
break;
}
catch
{
}
}
}
Maybe I'm just dumb but for me it doesn't seem all that different. Are there some key differences that can be the cause?
Here is code from one of my working systems that submits a POST request through an HTTPClient.
[Route("resource")]
public async Task<dynamic> CreateResource([FromBody]Resource resource)
{
if (resource == null) return BadRequest();
dynamic response = null;
resource.Topic = GetDataFromSomewhereElse();
var message = new PostMessage(resource).BodyContent;
dynamic postRequest = new
{
Message = message
};
var post = JsonConvert.SerializeObject(postRequest);
HttpContent content = new StringContent(post, Encoding.UTF8, "application/json");
using (var client = new HttpClient())
{
client.Timeout = TimeSpan.FromMinutes(1);
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new System.Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json"));
try
{
client.BaseAddress = #"http://localhost:51145/";
HttpResponseMessage postResponse = await client.PostAsync("Resource", content); //"Resource" is a route exposed on the remote host
string json = await postResponse.Content.ReadAsStringAsync();
if (postResponse.StatusCode == HttpStatusCode.BadRequest) return BadRequest();
if (postResponse.StatusCode == HttpStatusCode.InternalServerError) return InternalServerError();
if (postResponse.StatusCode == HttpStatusCode.NotFound) return NotFound();
return json;
}
catch(Exception ex)
{
return InternalServerError(ex);
}
}
}
[Edit]
The "PostMessage" was modified to remove domain-specific details. Here is how BodyContent is defined inside the real "PostMessage" from my solution, to provide you enough context to understand what that "message" actually is and how it works into the sample.
public string BodyContent
{
get
{
string content = "";
Type type = this.GetType();
Assembly assembly = Assembly.GetExecutingAssembly();
string resource = String.Format("{0}.{1}", type.Namespace, this.EmbeddedResourceName);
Stream stream = assembly.GetManifestResourceStream(resource);
StreamReader reader = new StreamReader(stream);
content = reader.ReadToEnd();
return content;
}
}
...and here is PostRequest (again, with domain-specific details trimmed)
public class PostRequest
{
public string Message { get;set; }
}
I am trying to write some code to connect to an HTTPS site that uses Siteminder authentication.
I keep getting a 401. Any ideas?
I have read a few different things on here but none have really seemed all that helpful. I am also using Fiddler/Firefox Tamper to snoop what's going on.
Here is what I've got so far in regards to code:
try
{
Uri uri = new Uri("https://websiteaddresshere");
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri) as HttpWebRequest;
request.Accept = "text/html, application/xhtml+xml, */*";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko";
// request.Connection = "Keep-Alive";
// request.Method = "Get";
// request.Accept = "text";
request.AllowAutoRedirect = true;
request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
Cookie emersoncookie = new Cookie("SMCHALLENGE", "YES");
emersoncookie.Domain = "mydomain";
emersoncookie.Path = "/";
// authentication
var cache = new CredentialCache();
cache.Add(uri, "False", new NetworkCredential("myusername", "mypassword"));
request.Credentials = cache;
// response.
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (Stream stream = response.GetResponseStream())
{
XmlTextReader reader = new XmlTextReader(stream);
MessageBox.Show(stream.ToString());
}
}
}
catch (WebException exception)
{
string responseText;
using (var reader = new StreamReader(exception.Response.GetResponseStream()))
{
responseText = reader.ReadToEnd();
MessageBox.Show(responseText.ToString());
}
}
After doing some more reading on the MSDN website I decided to go a different route.
I ended up making this a service since it will need to be a service at the end of the day and did the following:
CookieContainer emersoncookie = new CookieContainer();
HttpWebRequest request = (HttpWebRequest)
WebRequest.Create("https://websiteaddress");
request.Credentials = new NetworkCredential("username", "password");
request.CookieContainer = emersoncookie;
request.Method = "GET";
HttpWebResponse response = (HttpWebResponse)
request.GetResponse();
Stream resStream = response.GetResponseStream();
using (Stream output = File.OpenWrite(#"c:\\somefolder\\somefile.someextention"))
using (Stream input = resStream)
{
input.CopyTo(output);
}
To anyone that might be running into Siteminder authentication issues, this piece of code works pretty well.
I couldn't get Jasen's code to work. Maybe your SM setup is different from mine. But with SiteMinder it's generally a two step authentication process. The code block below works for me:
//Make initial request
RestClient client = new RestClient("http://theResourceDomain/myApp");
client.CookieContainer = new CookieContainer();
IRestResponse response = client.Get(new RestRequest("someProduct/orders"));
//Now add credentials.
client.Authenticator = new HttpBasicAuthenticator("username", "password");
//Get resource from the SiteMinder URI which will redirect to the correct API URI upon authentication.
response = client.Get(new RestRequest(response.ResponseUri));
Although this uses RestSharp, it can be easily replicated using HttpClient or even HttpWebRequest.