We Keep Coding

Basic Auth not working when trying to get raw contents from BitBucket?

I'm trying to download file contents from Bit Bucket, but I keep getting a "Log In" page as a response. I'm providing credentials with Basic Auth. Is this a C# specific issue? Everything works fine if I try it via Postman. Code Below.
var url = "https://[BITBUCKET_DOMAIN]/projects/[ID]/repos/[REPO]/raw/[PATH_TO_MY_FILE]"
var uri = new Uri(url);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
string userName = "user";
string password = "pw";
request.PreAuthenticate = true;
request.Credentials = new NetworkCredential(userName, password);
request.Accept = "text/plain";
string result;
using (var response = request.GetResponse())
using (var stream = response.GetResponseStream())
using (var reader = new StreamReader(stream))
{
result = reader.ReadToEnd();
}
return result;

This top answer here worked for me.
HttpWebRequest using Basic authentication
String encoded = System.Convert.ToBase64String(System.Text.Encoding.GetEncoding("ISO-8859-1").GetBytes(userName + ":" + password));
request.Headers.Add("Authorization", "Basic " + encoded);

c# httpwebrequest for basic authentication

I've been trying to use a username, password and access key to authenticate on a server, so that I could use its API. I used to following code:
string encoded = Convert.ToBase64String(Encoding.Default.GetBytes("myusername:mypass"));
string api = "blahblahblahblabh";
string auth_string = "Basic " + encoded;
string _post_string = "username=myusername&password=mypass&grant_type=password";
HttpWebRequest request = WebRequest.Create(https_website) as HttpWebRequest;
request.Accept = "application/xml";
request.Method = "POST";
request.ContentType = "application/x-www-forms-urlencoded";
request.ContentLength = _post_string.Length;
CookieContainer cookies = new CookieContainer();
request.CookieContainer = cookies;
request.Headers["Authorization"] = auth_string;
request.Headers["api-key"] = api;
request.ServerCertificateValidationCallback += (sender1, certificate, chain, sslPolicyErrors) => { return true; };
StreamWriter writer = new StreamWriter(request.GetRequestStream());
writer.Write(_post_string);
writer.Close();
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
StreamReader reader = new StreamReader(response.GetResponseStream());
string xml_response = reader.ReadToEnd();
reader.Close();
So I sent a POST request with the username/password and access_key and tried to receive a response. I always get "The remote server returned an error: (404) Not Found". If I add more to the string of the 1st line, for example if instead of "myusername:mypass" I use "myusername:mypass:password" I get an error 401 (Unauthorized). I don't understand why this is happening...

Very weird error 401?

I am receiving a 401 error I am not anticipating. I am 100% sure the password and usernamne is correct. When I try the it on postman it works, and I get the data I expect. But in this code, the .downloadstring() method returns a 401 error. I created a new harvest account and tried get to that one with the same code, just changed the password and username and I got the API data I wanted. Is there any other reason then wrong password or username error 401 can be cached?
public List<Project> GetAllProjects()
{
uri = "https://bruh.harvestapp.com/projects";
jsonPath = Path.Combine(HostingEnvironment.MapPath("~/App_Data"), "projects.json");
using (WebClient webClient = new WebClient())
{
webClient.Headers[HttpRequestHeader.ContentType] = "application/json";
webClient.Headers[HttpRequestHeader.Accept] = "application/json";
webClient.Headers[HttpRequestHeader.Authorization] = "Basic " + Convert.ToBase64String(new ASCIIEncoding().GetBytes(usernamePassword));
string response = webClient.DownloadString(uri);
projectsList = JsonConvert.DeserializeObject<List<Wrapper>>(response).Select(p => p.project).ToList();
}
return projectsList;
}

According to this C# code sample from harvest, there are a few things that need to be changed:
static void Main(string[] args)
{
HttpWebRequest request;
HttpWebResponse response = null;
StreamReader reader;
StringBuilder sbSource;
// 1. Set some variables specific to your account.
string uri = "https://yoursubdomain.harvestapp.com/projects";
string username="youremail#somewhere.com";
string password="yourharvestpassword";
string usernamePassword = username + ":" + password;
ServicePointManager.ServerCertificateValidationCallback = Validator;
try
{
request = WebRequest.Create(uri) as HttpWebRequest;
request.MaximumAutomaticRedirections = 1;
request.AllowAutoRedirect = true;
// 2. It's important that both the Accept and ContentType headers are
// set in order for this to be interpreted as an API request.
request.Accept = "application/xml";
request.ContentType = "application/xml";
request.UserAgent = "harvest_api_sample.cs";
// 3. Add the Basic Authentication header with username/password string.
request.Headers.Add("Authorization", "Basic " + Convert.ToBase64String(new ASCIIEncoding().GetBytes(usernamePassword)));
using (response = request.GetResponse() as HttpWebResponse)
{
if (request.HaveResponse == true && response != null)
{
reader = new StreamReader(response.GetResponseStream(), Encoding.UTF8);
sbSource = new StringBuilder(reader.ReadToEnd());
// 4. Print out the XML of all projects for this account.
Console.WriteLine(sbSource.ToString());
}
}
}

The remote server returned an error: (401) Unauthorized when requesting RestApi data

I am triyng to get data from the Rest API. API wants 3 things to give authentication;
first one is "Accept:application/vnd.###.v1.0+json"
second one : "Content Type : application/json"
third one : Base64 encoded "userName:password" string
and I should pass these credentials for validation and authorization in custom header.I know there are a lot of thread on this site about this topic but I couldn't solve the problem from them.
Here is the code block :
public class McAfeeIPSManager
{
String URL = "https://serviceOfApi/sdkapi/session";
public void getWebRequest()
{
System.Net.ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };
String username = "user";
String password = "password1";
var request = HttpWebRequest.Create(URL) as HttpWebRequest;
request.Accept = "application/vnd.###.v2.0+json";
request.Method = "GET";
request.ContentType = "application/json";
String encoded = System.Convert.ToBase64String(System.Text.Encoding.GetEncoding("ISO-8859-1").GetBytes(username + ":" + password));
request.Headers.Add("Authorization","Basic "+encoded);
try
{
// Get response
using (var response = request.GetResponse() as HttpWebResponse)
{
// Get the response stream
using (var responseReader = new StreamReader(response.GetResponseStream()))
{
string responseBody = responseReader.ReadToEnd();
// Console application output
System.Diagnostics.Debug.Write("Response Body ---> " + responseBody);
//Console.WriteLine(responseBody);
}
}
}
catch (WebException ex)
{
System.Diagnostics.Debug.Write("Error : " + ex.Message);
Console.WriteLine("Error: {0}", ex.Message);
}
}
}
How can get data from WebAPI under these conditions?Can anybody help me?

You have no PreAuthenticate and credential ?
I have a code that may help you:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("https://pwmaffr2:8443/remote/system.delete?names=" + DeviceName + "");
request.Headers.Add("AUTHORIZATION", "Basic YTph");
request.ContentType = "text/html";
request.Credentials = new NetworkCredential(Username, Password);
ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };
request.PreAuthenticate = true;
request.Method = "GET";
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
);
StreamReader stream = new StreamReader(response.GetResponseStream());
string X = stream.ReadToEnd();

hmm in addition of what i post try deal with this it should work for you hope:
string credentials = String.Format("{0}:{1}", username, password);
byte[] bytes = Encoding.ASCII.GetBytes(credentials);
string base64 = Convert.ToBase64String(bytes);
string authorization = String.Concat("basic ", base64);
request.Headers.Add("Authorization", authorization);

Login to Https Website with WebRequest and WebResponse Not Working C#

I am working on a program that will log into a website and get certian data. However I am having trouble posting the login parameters and dealing with the cookies, as each time I get a page saying "You have logged out or Session has expired." So clearly I'm doing something wrong with posting the parameters or dealing with the cookies, but don't know which. I have been working on this for a while and just can't get my head around why this is not working correctly.
void Login2(string username, string password)
{
string pageSource;
string formUrl = "https://forUrl.com";
string formParams = string.Format("login={0}&sslProt={1}&pwd={2}&gru={3}", username, "", password, "115237091");
string cookieHeader;
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(formUrl);
req.AllowAutoRedirect = false;
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"];
string getUrl = "https://Urlbehindform.com";
HttpWebRequest getRequest = (HttpWebRequest)WebRequest.Create(getUrl);
getRequest.Method = "GET";
getRequest.AllowAutoRedirect = false;
getRequest.Headers.Add("Cookie", cookieHeader);
HttpWebResponse getResponse = (HttpWebResponse)getRequest.GetResponse();
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
Response.Redirect(getUrl);
}
I am getting the cookie when I do the POST and sending it back when I do the GET, but for some reason this doesn't seem to work. At first I thought it was the parameters, but after looking at the issue further using Tamper Data with Firefox the login parameters seem to be working fine. Any help would be great, as I have been working on this for a while and can't wrap my head around it. Thanks!
UPDATE:
After trying out a few suggestions I still can't get this to work. However Upon looking deeper into Data Tamper, It appears that there is a POST with the login parameters, then a GET to a different page and then finally the GET to the page after the login page (The one I'm trying to get to). After some further debugging I actually discovered that my login POST is not working as I thought, As the response header location is showing "/cv/scripts/A028/eng/logErr.asp". Meaning the rest of my code could have been fine all a long, it was that the POST wasn't giving me a valid login. Any Sugguestions as to why I am always getting the login error page? As always thanks for the help.
UPDATE:
After playing around further with Tamper Data is appears that the reason I am unable to get a successful login is that in order have a successful POST of the parameters there needs to be a cookie already obtained. How do I go about doing this?

Use a single CookieContainer for both requests. Then you don't have to copy cookies manually.
I [BMW1] added in a CookieContainer called cookies, but it still not working, Im not sure if im using the CookieContainer the right way. Here is an updated version of my code.
And edited by me [Hans Kesting], see comments with [HK]
void Login2(string username, string password)
{
string pageSource;
string formUrl = "https://server/cv/scripts/A028/eng/logProc.asp?ntry=0&dbg=";
string formParams = string.Format("login={0}&sslProt={1}&pwd={2}&gru={3}", username, "", password, "115237091");
// [HK] create a container for the cookies, where they are added automatically
CookieContainer cookies = new CookieContainer();
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(formUrl);
req.CookieContainer = cookies;
req.AllowAutoRedirect = false;
req.ContentType = "application/x-www-form-urlencoded";
req.Method = "POST";
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(formParams);
req.ContentLength = bytes.Length;
using (Stream os = req.GetRequestStream())
{
os.Write(bytes, 0, bytes.Length);
}
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
// [HK] no need to add cookies "by hand", that will happen automatically
//cookies.Add(resp.Cookies);
string getUrl = "https://server/cv/scripts/A028/eng/home.asp";
HttpWebRequest getRequest = (HttpWebRequest)WebRequest.Create(getUrl);
// [HK] use the same cookiecontainer as on the first request - correct
getRequest.CookieContainer = cookies;
getRequest.Method = "GET";
getRequest.AllowAutoRedirect = false;
HttpWebResponse getResponse = (HttpWebResponse)getRequest.GetResponse();
// [HK] no need to add cookies, they should be there already
//cookies.Add(getResponse.Cookies);
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
pageSource = sr.ReadToEnd();
}
// [HK] no need to add cookies, they should be there already
// cookies.Add(getResponse.Cookies);
Response.Redirect(getUrl);
}

You could use a Cookie aware web client,
public class CookieAwareWebClient : WebClient
{
public CookieContainer CookieContainer { get; set; }
public Uri Uri { get; set; }
public CookieAwareWebClient() : this (new CookieContainer())
{
}
public CookieAwareWebClient(CookieContainer cookies)
{
this.CookieContainer = cookies;
}
protected override WebRequest GetWebRequest(Uri address)
{
WebRequest request = base.GetWebRequest(address);
if (request is HttpWebRequest)
{
(request as HttpWebRequest).CookieContainer = this.CookieContainer;
}
HttpWebRequest httpRequest = (HttpWebRequest) request;
httpRequest.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
return httpRequest;
}
protected override WebResponse GetWebResponse(WebRequest request)
{
WebResponse response = base.GetWebResponse(request);
String setCookieHeader = response.Headers[HttpResponseHeader.SetCookie];
if (setCookieHeader != null)
{
//do something if needed to parse out the cookie.
if (setCookieHeader != null)
{
Cookie cookie = new Cookie(); //create cookie
this.CookieContainer.Add(cookie);
}
}
return response;
}
}
Example usage:
var wc = new CookieAwareWebClient ();
wc.Headers["Content-type"] = "application/x-www-form-urlencoded";
string HtmlResult = wc.UploadString(URI, myParameters);