We Keep Coding

Interface-less login to login.microsoftonline.com

This issue has kept me busy for days now, so I might just as well post it here... searching in other topics has brought me far, but I'm not there yet.
Background
My job is to automate tests in the browser: I make use of Selenium and .NET to handle this. One of our clients uses Active Directory for logging in on the web application. Logging in again for every test takes a lot of time (a new session is started for each test), so I'm trying to speed this phase up by writing a class that will login beforehand, without the browser.
By using Fiddler, I managed to reproduce most of the webrequests in code. Still, I don't work with custom web requests often, so I easily overlook possible issues. Something goes wrong in the last step, where I try to retrieve the AppServiceAuthSession cookie. I'll use a picture to illustrate:
On the left, there is the web request I make using my code. On the right, there is the web request I get when using the browser.
The Nonce cookie has its value, headers are correct... the only difference here is the "Expect" header instead of the "Connection" header, but that happens because I disabled the "AllowAutoRedirect" property for the request (I don't get to see this request in Fiddler if I keep it enabled).
Next, here comes the content of both requests, since that's hard to show by using the Fiddler interface.
Code request:
code=AQABAAIAAABHh4kmS_aKT5XrjzxRAtHzZ08Y3xv2A4HxumqfAbR5iIBFEbbm6oYUq5sdHF72ybOkXFz33XVW5uXvuq-Tiz43zQ7y08bAhGH-aR_3HEpugBMAX_Op4O0Z4RjMKOLyd5b-VNVhjqv_CRmgHTEhYnxubjNyaG-5Vsuo7YRa3jXFybBheb73Erp7CDYFCQ-dFjgNAEHEbzrfE64dGVvHCCr-rvpHCTJfW55eYE9l68d7rryVfqC8dX7wplr7A3G8ySS84Tb9_hYxA61L_fQEm22NehWGNKw_MwXWNsIONtwiRd5LQD95iXYwjCw9J1p-WoZw9p6228r8YOuBVJA0k22O_zjqvKGUkVEyVWvAZZvDSO26_m-uuz7ql52ZTfBhmOfF8iXfdgeUPUqSXmTmNtL0LAOgiYKt4DyH1TbgAN2ox-sVcFp0k7bmjE81h0yiY8RgR-a2aONvuIpSR1Dx7VG-1wkllQAkYGV8zMoahs0EVRiDdOEQwwvkegALZNaU_ROKkbxj-H71h7lffkLlZoeM1lQFwgcVyB1_kV04iZKg2x5-VxHRYCDCNdeaxuaZD5dOrxIh_Y6SfQK-VBWsDB1C_i76QQpFI2c-7oxFNn7RLCU5gw7iv8k31ry2-hKSfhcU6LtN--t-eW73ADu0cW9juh1zzgzXAPXz0FqNyh07SyRawPqir9jdZ4GlgU4r-Imm41n4b7LH_nYixE1q-JNTm7TlawW3Xe7i28QMx95wPGlqyYl0PAo0VMwMPgoG3Zi-5BiEFi31qOxCMJNbXWi-YOMMcV3W5-sNbYA44mOUoYnz0hny-PlvKFwTL9RLBxkgAA&
id_token=eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IkZTaW11RnJGTm9DMHNKWEdtdjEzbk5aY2VEYyIsImtpZCI6IkZTaW11RnJGTm9DMHNKWEdtdjEzbk5aY2VEYyJ9.eyJhdWQiOiI4NmJiNjljZC04OTQ4LTQ2ZmEtOWQ3NS05YWRiZWRmODI1NWYiLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC8xYWNhOGFmYy01YTZlLTRkYWUtOTRhNC0xODIwZDgwNGMxZWIvIiwiaWF0IjoxNTIxNzM2NTM1LCJuYmYiOjE1MjE3MzY1MzUsImV4cCI6MTUyMTc0MDQzNSwiYWlvIjoiQVRRQXkvOEdBQUFBNXhNTk10SUJybUI1Z2dTN0xRNWxmSUNBMnY3RGpBRXIyUzgwS3dBa0VlMlpacVhNQW1hMnE0eG5xeDJ5aGxNZSIsImFtciI6WyJ3aWEiXSwiY19oYXNoIjoiUlUxOHNDNjJldTUyVnd0ZG9CTEItUSIsImVtYWlsIjoiUnViZW4udGVyLk1hdGVuQG1hY2F3Lm5sIiwiaWRwIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvMDViYWVkNGQtZTA0OS00NWM3LWI0N2QtYzk5YzY2ZmRiNTg5LyIsImluX2NvcnAiOiJ0cnVlIiwiaXBhZGRyIjoiODIuMjAxLjExMi40IiwibmFtZSI6IlJ1YmVuIHRlciBNYXRlbiIsIm5vbmNlIjoiYmI2NDU1ZGQzODJkNDRmNGIyNDAwYThjYzY3NmNmNTZfMjAxODAzMjEwOTU3MzAiLCJvaWQiOiI2YjgzMGMyNi1hZGE4LTQyY2MtOTQ3MS1kMzU5NTc1N2IwNDciLCJyb2xlcyI6WyJDb250ZW50RWRpdG9yIl0sInN1YiI6InJIVlJfRFZsWjltN2V5LS02RVE4LUFGY1FVZ3VWa3VRZVdnR3ZVeGdmbE0iLCJ0aWQiOiIxYWNhOGFmYy01YTZlLTRkYWUtOTRhNC0xODIwZDgwNGMxZWIiLCJ1bmlxdWVfbmFtZSI6IlJ1YmVuLnRlci5NYXRlbkBtYWNhdy5ubCIsInV0aSI6InpoRTFLWVRhWGtxS1dWWThqQlFaQUEiLCJ2ZXIiOiIxLjAifQ.xcuSefb-pr21z53eoirQTj3nptW95rQ0RnsxkF6B436Z16bI826UK4wYLvjGl8NrXYV1-sJhlUTUuX31p90GxJthUV1tbzsvFUwNk8f9DJgKls-31X78fNbChkuNDIS9Tj6Lj0cPOlZaC6U8TXYaNsef8qURtvtXzPt7Td1t6rfvuJL29pnow0We7u-eqUR8_TDD9PzY82BdYSbfIn4X-cOw4F79O-OAce9zRebX3V1Cr53gizMUIKKDWlS8fdwKnpdluAgZup-r6RJa7c_ui0NB2REao-PxixkvO-PuCtEfLhRnbRAprVa6ggMlCrtN6Uf7EmLRGLNBGHbl379vgg&
state=redir%3d%252Flogin%252Fcheck&
session_state=4dcbbba5-e8a5-4ad7-9ba9-28f6af570ead
Browser request:
code=AQABAAIAAABHh4kmS_aKT5XrjzxRAtHzZT_hGoYPGfSq8uoVwxawNGgqfV8j3-WdO_yha1uD2rspKWZmKjbV4j3YQmg4IQOdNBKzWDYTAST-Shag2D2Ih9EHMThyu7ncE_XRSBDOb_f05q3WCKYtF5ki5VVFcd8Li029KFwEjjutGw4k-_GfPe1rym5CXRvqQzj0wGUN4Q3ndwqrcNXsdvGG6wEXbiTJ8_3aSyuAAsHBMr1l23fjJRe-CKfr7EqKDZLzm_2H6L0MODy_mWkrD715CjU2_mUb1YQ__6MZvhHss41BFPcL3zyK2oh7Z668dJMEK495MhKFt-NPUFUPXsUNwK1fP4F-tdmBIY39BL9vDxzynmcF5A63yb8q3m0CgJOZ0i1aobe1iEX75sNXjIa2385yFw08lmkVJFj9GccvIE5ak4Zi6sdjWhEtS9TyKXc5uBtKxJCYIy52avI6bUAyc5-Xq0amik0Q75wuLK0bSoeIwBofKuJnoHfl1o3_7N2uSZ_2dJx5zPW9VxRz4e6u-mkmDu9F5lK42zo3al8c03zKHhLkTVRx08JV7JZuqpLznz5JFEHp1QUq-KeYNdMXsngPZW3VEegSyu6VGYwvbmL07GYTZ3KWi-dGxej88KvobZF3O40rfISThS4gVpnFoH7UQs_RE_TEjRvUraFBUhjE-1EUd7WXl6NsxQwGO0Uw7W87ZqlY4G-JhN39MOdm4v9UZELmeZph87PWqqy9-6h1pK8wrH8S98HqV1q3DUR68Ue-zCSvV9hgxrkSnPtFDTfCgtOvpNj14GUesMi8L0XPa6DsO0hj27_A1T0STGSHMGg8QMcgAA&
id_token=eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IkZTaW11RnJGTm9DMHNKWEdtdjEzbk5aY2VEYyIsImtpZCI6IkZTaW11RnJGTm9DMHNKWEdtdjEzbk5aY2VEYyJ9.eyJhdWQiOiI4NmJiNjljZC04OTQ4LTQ2ZmEtOWQ3NS05YWRiZWRmODI1NWYiLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC8xYWNhOGFmYy01YTZlLTRkYWUtOTRhNC0xODIwZDgwNGMxZWIvIiwiaWF0IjoxNTIxNzExMTIxLCJuYmYiOjE1MjE3MTExMjEsImV4cCI6MTUyMTcxNTAyMSwiYWlvIjoiQVRRQXkvOEdBQUFBT1YreEZOcjAwR3R6TG9jdkZaMVZ0SjlIRkYyUjc2VWE4d1V0eGlISkpienU4em1nN2xtRzdPT3ZCelZ6VkpyeiIsImFtciI6WyJ3aWEiXSwiY19oYXNoIjoieThoZUw5RHh3b3ZkUV9IbmtTYzlqZyIsImVtYWlsIjoiUnViZW4udGVyLk1hdGVuQG1hY2F3Lm5sIiwiaWRwIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvMDViYWVkNGQtZTA0OS00NWM3LWI0N2QtYzk5YzY2ZmRiNTg5LyIsImluX2NvcnAiOiJ0cnVlIiwiaXBhZGRyIjoiODIuMjAxLjExMi40IiwibmFtZSI6IlJ1YmVuIHRlciBNYXRlbiIsIm5vbmNlIjoiMDk5MmE5NjQ1N2JiNDA3ZTgxZTYyZjE0YzA0ODg3ZTVfMjAxODAzMjIwOTQxNTIiLCJvaWQiOiI2YjgzMGMyNi1hZGE4LTQyY2MtOTQ3MS1kMzU5NTc1N2IwNDciLCJyb2xlcyI6WyJDb250ZW50RWRpdG9yIl0sInN1YiI6InJIVlJfRFZsWjltN2V5LS02RVE4LUFGY1FVZ3VWa3VRZVdnR3ZVeGdmbE0iLCJ0aWQiOiIxYWNhOGFmYy01YTZlLTRkYWUtOTRhNC0xODIwZDgwNGMxZWIiLCJ1bmlxdWVfbmFtZSI6IlJ1YmVuLnRlci5NYXRlbkBtYWNhdy5ubCIsInV0aSI6IlRFUGhmbG53dGs2YTFKTVJEcHNVQUEiLCJ2ZXIiOiIxLjAifQ.eT25sB0q7a5nEPBS5z2WVFHohBdzgioj6rd5DOMlb30dp1mIAVgvRZxSJhybB6RuHOl86LTkcyH4qzXl8uQ9fq8hapZUdJ2sDDqvk-PmcIWp_Ar6Gyo--4DtUxu7quIe-JaP_HyN_lsNbm0Gxut2VkZuHmCaiBxRriosP2FRg2GyKO75P6V7r0EPmQnS_4HA0WhEUmqNBzrZnbMV4rkc4FkLoMSLgJN69XdBMCQfsnhVcDWFPGj_Q79vlSbSq17Y7EPxn5phkrLGfZGMQj6CNHSzeT76LxR4txUu16QEQLLHckw2aSsESQ8SPFXwk794jEuyFIoFPrGg75hv4XxC7Q&
state=redir%3D%252Flogin%252Fcheck&
session_state=d897cb81-a5db-41f2-8852-8c18a82d36ab
Issue
As visible in the picture, the web request gets a response without AppServiceAuthSession and is redirected to the login.windows.net location. The normal browser request is different: it is successful at this point.
Question
Does anyone have a clue on what goes wrong here, or on how to debug it successfully? I triple-checked the input of the previous requests as well, but they all seem identical to the ones given to the browser. Even just a pointer in general might help... I'm getting a bit lost here. If more info is needed, just let me know. Thanks in advance!
Extra information
The code, in case it helps (censored where needed):
class Login
{
static CookieContainer cookiesNonce = new CookieContainer();
static CookieContainer cookiesA = new CookieContainer();
static CookieContainer cookiesB = new CookieContainer();
public static void DoLogin()
{
// CLIENTNAME-api, request for login with redirect
var url = #"https://CLIENTNAME-api.CLIENTNAME-dev.nl/.auth/login/aad?post_login_redirect_url=/login/check";
var request = WebRequest.Create(url) as HttpWebRequest;
request.Accept = #"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8";
request.UserAgent = #"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36";
request.Headers.Add("Upgrade-Insecure-Requests", "1");
request.CookieContainer = new CookieContainer();
request.KeepAlive = true;
request.AllowAutoRedirect = false;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
cookiesNonce.Add(response.Cookies);
cookiesA.Add(response.Cookies);
request = WebRequest.Create(response.Headers[HttpResponseHeader.Location]) as HttpWebRequest;
request.Accept = #"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8";
request.UserAgent = #"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36";
request.Headers.Add("Upgrade-Insecure-Requests", "1");
request.KeepAlive = true;
request.Host = #"login.windows.net";
request.CookieContainer = cookiesA;
}
string clientrequestid = "";
string authorizeUrl = "";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
cookiesA = new CookieContainer();
cookiesA.Add(response.Cookies);
var reader = new StreamReader(response.GetResponseStream());
string html = reader.ReadToEnd();
reader.Close();
var jsonRegEx = new Regex(#"\$Config=(.*?)(?<=\});");
var temp = jsonRegEx.Match(html).Groups[1].ToString() ;
var dict = JObject.Parse(temp);
var hpgrequestid = response.Headers["x-ms-request-id"];
var canary = GetValue(dict, "apiCanary");
clientrequestid = GetClientRequestId(dict);
authorizeUrl = response.Headers[HttpResponseHeader.Location];
// Request to microsoft online by credentials
url = #"https://login.microsoftonline.com/common/GetCredentialType";
request = WebRequest.Create(url) as HttpWebRequest;
request.Method = #"POST";
request.CookieContainer = cookiesB;
request.KeepAlive = true;
request.Headers.Add("hpgrequestid", hpgrequestid);
request.Headers.Add("canary", canary);
request.Headers.Add("client-request-id", clientrequestid);
request.Headers.Add("hpgact", "1800");
request.Headers.Add("hpgid", "1104");
request.Referer = response.ResponseUri.ToString(); ;
var reqStream = request.GetRequestStream();
var buffer = new ASCIIEncoding().GetBytes(#"{""username"":""user#COMPANYNAME.nl"",""isOtherIdpSupported"":true,""checkPhones"":false,""isRemoteNGCSupported"":false,""isCookieBannerShown"":true,""isFidoSupported"":false,""originalRequest"":""rQIIAZ2Qv08UQRzFd26PDaCJxBCD3RUQE83szM7N7s1eQoGCGjEhXAwQLch35wc33rK7zK6i_Ak2YGFjZSwvVlbGP4FGav8CYyOhsvTU-A_YvPeq93l5N_0ojPqLIsmyJJUKi5QLzBMDOFW9GKegMq2MYHFs3NXZOf7jvH3x5dX68Rs2OKYlHaPesGmquk_ISBeFw1DZ8G9S-nlY5CSEZ82Q5OWeLQiAIhLyPAM5-oTQGULfEBq3yITN41iprmCKc8MzxikFIWXSS6SJk11GI0G7LKJp3OvSr60rGyuTVvZbSmeP9EVrxpRuf7cq6-atf81pZd3yErv7BztxOdRyNPav_9u656Aahoe2UOVhHRa6-egvRiBBgJE4hkRjrkDjlAPHkWBUCcplpLNTf76sdGFVp3Klsbnu6H2w-VkbfW9fpn5_enp2zlvwOt7PNno_NfmrXrp08OHJ7dXX714ObnQD73SKkFuDxxUMDF2xxc7W_c2No3zdwc7q5kPxtLmztq22WP3i0ba992BtmfWjkwCdBMF54H2e-d-rfwE1""}");
reqStream.Write(buffer, 0, buffer.Length);
reqStream.Close();
}
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
cookiesB.Add(response.Cookies);
var reader = new StreamReader(response.GetResponseStream());
string html = reader.ReadToEnd();
reader.Close();
// Request to sts.COMPANYNAME.nl
url = #"https://sts.COMPANYNAME.nl/adfs/ls/wia?client-request-id=" + clientrequestid + #"&wa=wsignin1.0&wtrealm=urn%3afederation%3aMicrosoftOnline&wctx=LoginOptions%3D3%26estsredirect%3d2%26estsrequest%3drQIIAZ2QPU8UQRjHd26PDZwmEEOIdldATDSzOzs7uze7CYkoqBETwsUAwYLMzgs33t7uOrt4ykewAQsbK2N5sbIyfgQaqSmtjI2EytJF4xewef5P9X_53bJ9108WaZSmUcwFpDGhkESKwVj0QhgzkUqhKA5DZa515sjP8_bF19frR29x_wgVaAJ6g7ouq8TzhjLPDWSldv9-Qr5w88xz2UE98LJiX-ceY8LjLMtSxoefATgF4DsAk5bXZJMwFCKgWBCiSIoJQoxyHvUirsJoDyOfogD7KA57ATprzW6sNK748hRGH8qL1owqzGivLKr6nb1gpNBmeQnf_xPbKB9IPpzYN_513TesHLhjnYtiXLm5rD_Ziz7jjDLFYcgiCYlgEsaEEehTjARFhPsyPbHni1LmWnRLUyidya4cMZ19s2fNQSrz0Z1RYzJuZp-2wY_2VWQn09OdOeu61bV-tcGHqYZgtXTl-cend1ffvH_Vvxk41smU593u75asr9CKzne2Hm5uHGbrhu2sbj6mz-p7a9tiC1cvn2zrB4_WlnHiHzvg2HHOHevLzP_CP-vMXyKFKIDY76I4CXESkN3f0&cbcxt=&username=user%40COMPANYNAME.nl&mkt=&lc=";
request = WebRequest.Create(url) as HttpWebRequest;
cookiesB = new CookieContainer();
request.CookieContainer = cookiesB;
request.KeepAlive = true;
request.Headers.Add("Upgrade-Insecure-Requests", "1");
request.Referer = response.ResponseUri.ToString(); ;
request.UserAgent = #"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36";
request.Credentials = CredentialCache.DefaultNetworkCredentials;
request.AuthenticationLevel = System.Net.Security.AuthenticationLevel.MutualAuthRequested;
}
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
cookiesB.Add(response.Cookies);
var reader = new StreamReader(response.GetResponseStream());
string html = reader.ReadToEnd();
var doc = new HtmlDocument();
doc.LoadHtml(html);
var node = doc.DocumentNode.SelectSingleNode("//input[#name='wresult']");
var value = node.Attributes["value"].Value;
var decode = HttpUtility.HtmlDecode(value);
var wresult = HttpUtility.UrlEncode(decode);
node = doc.DocumentNode.SelectSingleNode("//input[#name='wctx']");
value = node.Attributes["value"].Value;
decode = HttpUtility.HtmlDecode(value);
var wctx = HttpUtility.UrlEncode(decode);
reader.Close();
// Logging in to microsoft online
url = #"https://login.microsoftonline.com/login.srf?client-request-id=" + clientrequestid;
request = WebRequest.Create(url) as HttpWebRequest;
request.CookieContainer = cookiesB;
request.Referer = response.ResponseUri.ToString(); ;
request.Method = #"POST";
request.Headers.Add("Upgrade-Insecure-Requests", "1");
request.KeepAlive = true;
request.UserAgent = #"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36";
request.Headers.Add("Origin", "https://sts.COMPANYNAME.nl");
request.Headers.Add("Cache-Control", "max-age=0");
request.ContentType = #"application/x-www-form-urlencoded";
var reqStream = request.GetRequestStream();
var buffer = new ASCIIEncoding().GetBytes(#"wa=wsignin1.0&wresult=" + wresult + #"&wctx=" + wctx);
reqStream.Write(buffer, 0, buffer.Length);
reqStream.Close();
}
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
cookiesB = new CookieContainer();
cookiesB.Add(response.Cookies);
var reader = new StreamReader(response.GetResponseStream());
string html = reader.ReadToEnd();
var doc = new HtmlDocument();
doc.LoadHtml(html);
reader.Close();
var state = GetValue(doc, "state");
var code = GetValue(doc, "code");
var idtoken = GetValue(doc, "id_token");
var session_state = GetValue(doc, "session_state");
// Callback to .auth/login
url = #"https://CLIENTNAME-api.CLIENTNAME-dev.nl/.auth/login/aad/callback";
request = WebRequest.Create(url) as HttpWebRequest;
request.Method = #"POST";
request.CookieContainer = new CookieContainer();
var cookie = cookiesNonce.GetCookies(new Uri("https://CLIENTNAME-api.CLIENTNAME-dev.nl"))["Nonce"];
request.CookieContainer.Add(cookie);
request.KeepAlive = true;
request.Headers.Add("Cache-Control", "max-age=0");
request.Headers.Add("Origin", "https://login.microsoftonline.com");
request.Headers.Add("Upgrade-Insecure-Requests", "1");
request.ContentType = #"application/x-www-form-urlencoded";
request.UserAgent = #"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36";
request.Accept = #"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8";
request.Referer = response.ResponseUri.ToString(); ;
request.Headers.Add("Accept-Encoding", #"gzip, deflate, br");
request.Headers.Add("Accept-Language", #"en-US,en;q=0.9");
request.AllowAutoRedirect = false;
var reqStream = request.GetRequestStream();
var buffer = new ASCIIEncoding().GetBytes(#"code=" + code + #"&id_token=" + idtoken + #"&state=" + state + "&session_state=" + session_state);
reqStream.Write(buffer, 0, buffer.Length);
reqStream.Close();
}
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
var appServiceAuthSession = response.Cookies["AppServiceAuthSession"].ToString();
var reader = new StreamReader(response.GetResponseStream());
string html = reader.ReadToEnd();
reader.Close();
}
}
private static string GetValue(HtmlDocument doc, string name)
{
var node = doc.DocumentNode.SelectSingleNode("//input[#name='" + name + #"']");
var value = node.Attributes["value"].Value;
var decode = HttpUtility.HtmlDecode(value);
var result = HttpUtility.UrlEncode(decode);
return result;
}
private static string GetValue(string key, string input, string delimiter)
{
var value = RegexHelper.GetValueForKey(input, key, delimiter);
return value;
}
private static string GetValue(JObject container, string key)
{
string value = (string)container[key];
return value;
}
private static string GetClientRequestId(JObject container)
{
var obj = container["desktopSsoConfig"];
var value = (string)obj["iwaEndpointUrlFormat"];
var content = value.Split('&');
var id = content[content.Length - 1].Replace("client-request-id=", "");
return id;
}
}

OpenID connect is intentionally unspecific about the nature of the user authentication. The premise is that the authentication service will render whatever authentication it needs to the user, and if this has been satisfactory, will issue a short lived code which is redirected to the application. The application then uses that code to obtain a token.
There are other flows for authenticating non-user facing application like services. That might not help you though if you need to emulate a user logon.
As you are on Selenium already, why not have it enter the test user credentials into the ui presented in a web browser window?

Not a real solution, but at least a useful workaround in our case:
Login by using the browser.
Navigate to the auth/me page.
Example: let's say the home url I had to use was https://company.company-dev.nl. I adjusted the url to https://company-api.company-dev.nl/.auth/me and navigated there.
Retrieve the access token from the html.
To read the token, I used the following code:
private string ConvertToken(string html)
{
var doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(html);
var timeout = DateTime.Now + TimeSpan.FromSeconds(5);
dynamic obj = null;
while (obj == null && DateTime.Now < timeout)
{
try
{
obj = (dynamic)JsonConvert.DeserializeObject(doc.DocumentNode.InnerText);
}
catch (Exception ex)
{
// Firefox occasionally bugs out if its JSON viewer kicks in too early, so use the according elements if needed
var element = Driver.FindElementById("rawdata-tab");
Driver.ClickElement(element);
var text = Driver.FindElementByClassName("data").Text;
obj = (dynamic)JsonConvert.DeserializeObject(text);
}
}
return obj[0]["access_token"].ToString();
}
Save the token.
When the next test starts, append the token to the home url.
{
NavigateToUrl(HomeUrl + #"login/" + Token);
}
Example: https://company.company-dev.nl/#/login/eyJ0eXAiOiandareallylonglineofcharactershere
This way, you can make only one test login, while the others will use the token generated by the first one. You have to play around with some other things as well (using the token while testing in parallel, dealing with token timeouts, etc.), but this is the core of the workaround.

C# UWP Login website using cookies

I'm trying to convert a WPF application to UWP but am struggeling on how to get the conenction to work.
The WPF application connects to a website, get cookies, read a token from html and then post data to login.
In WPF I had this class:
// CookieAwareWebClient.cs
public class CookieAwareWebClient : WebClient
{
public string Method;
public CookieContainer CookieContainer { get; set; }
public CookieAwareWebClient()
: this(new CookieContainer())
{
}
public CookieAwareWebClient(CookieContainer cookies)
{
this.CookieContainer = cookies;
}
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest request = base.GetWebRequest(address);
if (request is HttpWebRequest)
{
(request as HttpWebRequest).CookieContainer = this.CookieContainer;
(request as HttpWebRequest).ServicePoint.Expect100Continue = false;
(request as HttpWebRequest).UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:18.0) Gecko/20100101 Firefox/18.0";
(request as HttpWebRequest).Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
(request as HttpWebRequest).KeepAlive = true;
(request as HttpWebRequest).AutomaticDecompression = DecompressionMethods.Deflate |
DecompressionMethods.GZip;
if (Method == "POST")
{
(request as HttpWebRequest).ContentType = "application/x-www-form-urlencoded";
}
}
HttpWebRequest httpRequest = (HttpWebRequest)request;
httpRequest.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
return httpRequest;
}
protected override WebResponse GetWebResponse(WebRequest request)
{
WebResponse response = base.GetWebResponse(request);
String setCookieHeader = response.Headers[HttpResponseHeader.SetCookie];
if (setCookieHeader != null)
{
try
{
if (setCookieHeader != null)
{
Cookie cookie = new Cookie(); //create cookie
this.CookieContainer.Add(cookie);
}
}
catch (Exception)
{
}
}
return response;
}
}
,that is used like this:
// Program.cs
// ...
CookieAwareWebClient client = new CookieAwareWebClient(cookieJar);
string response = client.DownloadString("LINK TO WEBSITE TO GET COOKIES");
// Get the csrf token from response html
var tokenValue = Regex.Match(response, "name='csrf_token'
value='(.+?)'").Groups[1].Value;
// Prepare values to send
string token = $"csrf_token={tokenValue}&";
string email = $"email={email}&";
string password = $"password={password}&";
string rememberme = $"rememberme=on&";
string submit = $"submit=Login";
string postData = token + email + password + rememberme + submit;
client.Method = "POST";
response = client.UploadString("LINK TO LOGIN ACTION", postData);
This works flawless in WPF. How can I achieve someting like this in UWP?
I tried using HttpClient, but I either get bad response, javascript doesnt load, or I can't read the response.
So far, I tried to just connect to the normal site and read the html, to get the token. But I get a response error.
var handler = new HttpClientHandler { AllowAutoRedirect = true };
var client = new HttpClient(handler);
client.DefaultRequestHeaders.TryAddWithoutValidation("UserAgent", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:18.0) Gecko/20100101 Firefox/18.0");
client.DefaultRequestHeaders.TryAddWithoutValidation("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
client.DefaultRequestHeaders.TryAddWithoutValidation("KeepAlive", "true");
client.DefaultRequestHeaders.TryAddWithoutValidation("ServicePoint.Expect100Continue", "false");
var response = await client.GetAsync(new Uri("LINK"));
// ERROR => System.Net.Http.HttpRequestException: "Response status code does not indicate success: 503 ()."
response.EnsureSuccessStatusCode();
var html = await response.Content.ReadAsStringAsync();

You need to add a CookieContainer to the handler in order to be able to access the cookies return from the request
//have a cookie container before making the request
var cookies = new CookieContainer();
var handler = new HttpClientHandler() {
AllowAutoRedirect = true,
CookieContainer = cookies,
AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip,
};
//Handle TLS protocols (https)
System.Net.ServicePointManager.SecurityProtocol =
System.Net.SecurityProtocolType.Tls
| System.Net.SecurityProtocolType.Tls11
| System.Net.SecurityProtocolType.Tls12;
var client = new HttpClient(handler);
client.DefaultRequestHeaders.TryAddWithoutValidation("UserAgent", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:18.0) Gecko/20100101 Firefox/18.0");
client.DefaultRequestHeaders.TryAddWithoutValidation("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
client.DefaultRequestHeaders.TryAddWithoutValidation("KeepAlive", "true");
client.DefaultRequestHeaders.TryAddWithoutValidation("ServicePoint.Expect100Continue", "false");
var uri = new Uri("Link");
var response = await client.GetAsync(uri);
var html = await response.Content.ReadAsStringAsync();
// Get the csrf token from response html
var tokenValue = Regex.Match(html, "name='csrf_token' value='(.+?)'").Groups[1].Value;
//...Prepare values to send

C# - Connection: keep-alive Header is Not Being Sent During HttpWebRequest redirect

I have a big problem when I using HttpWebRequest to get content from my blog.
First let's see the code
The request is made by a C# console application:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://localhost/MyBlog2014/Security/Login");
request.ProtocolVersion = HttpVersion.Version10;
var data = "Email=myemail&Password=1234567&keepMeOn=false";
byte[] send = Encoding.Default.GetBytes(data);
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8";
request.Headers.Add("Accept-Language: ro-RO,ro;q=0.8,en-US;q=0.6,en;q=0.4");
request.Headers.Add("Cache-Control: max-age=0");
request.KeepAlive = true;
request.UnsafeAuthenticatedConnectionSharing = true;
request.ServicePoint.ConnectionLimit = 1;
request.ServicePoint.UseNagleAlgorithm = false;
request.ServicePoint.Expect100Continue = false;
request.Method = "POST";
request.UserAgent =
"Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = send.Length;
request.AllowAutoRedirect = true;
Stream sout = request.GetRequestStream();
sout.Write(send, 0, send.Length);
sout.Flush();
sout.Close();
Console.WriteLine("\nThe HTTP request Headers for the first request are: \n{0}", request.Headers);
IAsyncResult result = (IAsyncResult)request.BeginGetResponse(new AsyncCallback(RespCallback), request);
Callback method
static void RespCallback(IAsyncResult asynchronousResult)
{
try
{
HttpWebRequest request = (HttpWebRequest)asynchronousResult.AsyncState;
WebResponse res = request.GetResponse();
var stream = new StreamReader(res.GetResponseStream());
Console.WriteLine(stream.ReadToEnd());
stream.Close();
res.Close();
Console.WriteLine();
}
catch (WebException e)
{
//do something
}
catch (Exception e)
{
//do something
}
}
Now, this code don't work like I wish. It goes in my login method, save email on the session object and then makes a redirect to action index and here appears my big problem in the index method the object Session is null.
Let's see the code
Login method
[HttpPost]
public ActionResult Login(LoginDto login)
{
// if email and password are ok the save email on the sessio
// then reirect to index
//else show error messages(s) return View(login);
}
Index method
[AuthorizeUser(false)]
public ActionResult Index(int pag = 1)
{
//I built the model
return View(new HomePageModel
{
CUrrentPage = pag,
Articles = model,
MaxPage = totalpagesArticles
});
}
So, I will not receive the content from index method because I am not authorized.
The code for save email on the session object is this:
public void SaveDetaliiUser(SessionUserDetails userDetails)
{
HttpContext.Current.Session[SessionDetaliiUtilizator] = userDetails;
}
Before to write here I searched on the net a solution for my problem and the following links didn't help me
How to send KeepAlive header correctly in c#?
C# - Connection: keep-alive Header is Not Being Sent During HttpWebRequest
Keep a http connection alive in C#?
http://social.msdn.microsoft.com/Forums/en-US/87bc7029-ce23-438a-a767-f7c32dcc63a7/how-to-keep-connection-live-while-using-httpwebrequest?forum=netfxnetcom
Thank you in advance,
Marian

Finally I solved the problem. It seems to keep session during redirect I should set request.CookieContainer = new CookieContainer();

c# web browser cookies in httpwebrequest

I have a webbrowser control which loads a page.
I then hit a button to call this method:
public void get(Uri myUri)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(myUri);
CookieContainer cookieJar = new CookieContainer();
cookieJar.SetCookies(webBrowser1.Document.Url,webBrowser1.Document.Cookie.Replace(';', ','));
request.CookieContainer = cookieJar;
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
int cookieCount = cookieJar.Count;
Stream receiveStream = response.GetResponseStream();
StreamReader readStream = new StreamReader(receiveStream, Encoding.UTF8);
txt.Text = readStream.ReadToEnd();
txt2.Text = cookieCount.ToString();
}
As from the cookieCount int i can see that if i call the method before logging in on the page in the web browser control i would get 6 cookies, and after i log in i get 7.
However, even with the cookies the response i get is the same as if i wouldnt have been logged in.
So i am guessing that the cookies isnt being sent with the request?
Thanks!

You're recreating your CookieContainer every time you call this method, you need to use the same CookieContainer in all requests
you can use this code, to handle your requests:
static CookieContainer cookies = new CookieContainer();
static HttpWebRequest GetNewRequest(string targetUrl, CookieContainer SessionCookieContainer)
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(targetUrl);
request.CookieContainer = SessionCookieContainer;
request.AllowAutoRedirect = false;
return request;
}
public static HttpWebResponse MakeRequest(HttpWebRequest request, CookieContainer SessionCookieContainer, Dictionary<string, string> parameters = null)
{
request.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.52 Safari/536.5Accept: */*";
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
request.CookieContainer = SessionCookieContainer;
request.AllowAutoRedirect = false;
if (parameters != null)
{
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
string postData = "";
foreach (KeyValuePair<String, String> parametro in parameters)
{
if (postData.Length == 0)
{
postData += String.Format("{0}={1}", parametro.Key, parametro.Value);
}
else
{
postData += String.Format("&{0}={1}", parametro.Key, parametro.Value);
}
}
byte[] postBuffer = UTF8Encoding.UTF8.GetBytes(postData);
using (Stream postStream = request.GetRequestStream())
{
postStream.Write(postBuffer, 0, postBuffer.Length);
}
}
else
{
request.Method = "GET";
}
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
SessionCookieContainer.Add(response.Cookies);
while (response.StatusCode == HttpStatusCode.Found)
{
response.Close();
request = GetNewRequest(response.Headers["Location"], SessionCookieContainer);
response = (HttpWebResponse)request.GetResponse();
SessionCookieContainer.Add(response.Cookies);
}
return response;
}
and to request a page,
HttpWebRequest request = GetNewRequest("http://www.elitepvpers.com/forum/login.php?do=login", cookies);
Dictionary<string,string> parameters = new Dictionary<string,string>{{"your params","as key value"};
HttpWebResponse response = MakeRequest(request, cookies, parameters);
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
if(!reader.EndOfStream)
{
Console.Write(reader.ReadToEnd());
}
}

When matching the session, your web server may be taking into account some other HTTP request headers, besides cookies. To name a few: User-Agent, Authorization, Accept-Language.
Because WebBrowser control and WebRequest do not share sessions, you'd need to replicate all headers from the WebBrowser session. This would be hard thing to do, as you'd need to intercept WebBrowser trafic, in a way similar to what Fiddler does.
A more feasible solution might be to stay on the same session with WebBrowser by using Windows UrlMon APIs like URLOpenStream, URLDownloadToFile etc., instead of WebRequest. That works, because WebBrowser uses UrlMon library behind the scene.
I've recently answered some related questions:
The notorious yet unaswered issue of downloading a file when windows security is required
Upload a file to a website programmatically?

HttpWebrequests facebook login and request app token

I'm trying to login on facebook and retrive a token by using this link:
https://www.facebook.com/dialog/oauth?client_id=282892925078054&redirect_uri=https://www.facebook.com/&response_type=token
My code looks like this, but i get an invalid link when i'm requesting the link above.
CookieCollection cookies = new CookieCollection();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://www.facebook.com/login.php?login_attempt=1");
request.CookieContainer = new CookieContainer();
request.CookieContainer.Add(cookies);
//Get the response from the server and save the cookies from the first request..
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
cookies = response.Cookies;
string getUrl = "https://www.facebook.com/login.php?login_attempt=1";
string postData = String.Format("email={0}&pass={1}", email, pass);
HttpWebRequest getRequest = (HttpWebRequest)WebRequest.Create(getUrl);
getRequest.CookieContainer = new CookieContainer();
getRequest.CookieContainer.Add(cookies); //recover cookies First request
getRequest.Method = WebRequestMethods.Http.Post;
getRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
getRequest.AllowWriteStreamBuffering = true;
getRequest.ProtocolVersion = HttpVersion.Version11;
getRequest.AllowAutoRedirect = true;
getRequest.ContentType = "application/x-www-form-urlencoded";
byte[] byteArray = Encoding.ASCII.GetBytes(postData);
getRequest.ContentLength = byteArray.Length;
Stream newStream = getRequest.GetRequestStream(); //open connection
newStream.Write(byteArray, 0, byteArray.Length); // Send the data.
newStream.Close();
HttpWebResponse getResponse = (HttpWebResponse)getRequest.GetResponse();
cookies = getResponse.Cookies;
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
string sourceCode = sr.ReadToEnd();
}
//Get the token
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create("https://www.facebook.com/dialog/oauth?client_id=282892925078054&redirect_uri=https://www.facebook.com/&response_type=token");
getRequest.CookieContainer = new CookieContainer();
getRequest.CookieContainer.Add(cookies);
webRequest.AllowAutoRedirect = false;
HttpWebResponse rresponse = (HttpWebResponse)webRequest.GetResponse();
if (rresponse.StatusCode == HttpStatusCode.Redirect)
{
Console.WriteLine("redirected to: " + rresponse.GetResponseHeader("Location"));
}
Please help me. Thanks in advance.

https://www.facebook.com/dialog/oauth?client_id=282892925078054&redirect_uri=https://www.facebook.com/&response_type=token
The URL does not allow the application configuration.: The application settings do not allow one or more of the URLs. Internet Site URL and Canvas URL or domain URLs must be sub-domains of application domains.