We Keep Coding

CredentialCache and HttpWebRequest in .NET

I'm having difficulty understanding how web requests and credentials work in .NET.
I have the following method that is executing a request to a SOAP endpoint.
public WebResponse Execute(NetworkCredential Credentials)
{
HttpWebRequest webRequest = CreateWebRequest(_url, actionUrl);
webRequest.AllowAutoRedirect = true;
webRequest.PreAuthenticate = true;
webRequest.Credentials = Credentials;
// Add headers and content into the requestStream
asyncResult.AsyncWaitHandle.WaitOne();
return webRequest.EndGetResponse(asyncResult);
}
It works well enough. However, users of my applications may have to execute dozens of these requests in short succession. Hundreds over the course of the day. My goal is to implement some of the recommendations I've read about, namely using an HttpClient that exists for the entire lifetime of the application, and to use the CredentialCache to store user's credentials, instead of passing them in to each request.
So I'm starting with the CredentialCache.
Following the example linked above, I instantiated a CredentialCache and added my network credentials to it. Note that this is the exact same NetworkCredential object that I was passing to the request earlier.
NetworkCredential credential = new NetworkCredential();
credential.UserName = Name;
credential.Password = PW;
Program.CredCache.Add(new Uri("https://blah.com/"), "Basic", credential);
Then, when I go to send my HTTP request, I get the credentials from the cache, instead of providing the credentials object directly.
public WebResponse Execute(NetworkCredential Credentials)
{
HttpWebRequest webRequest = CreateWebRequest(_url, actionUrl);
webRequest.AllowAutoRedirect = true;
webRequest.PreAuthenticate = true;
webRequest.Credentials = Program.CredCache;
// more stuff down here
}
The request now fails with a 401 error.
I am failing to understand this on several levels. For starters, I can't seem to figure out whether or not the CredentialCache has indeed passed the proper credentials to the HTTP request.
I suspect part of the problem might be that I'm trying to use "Basic" authentication. I tried "Digest" as well just as a shot in the dark (which also failed), but I'm sure there must be a way to see what kind of authentication the server is expecting.
I have been combing StackOverflow and MDN trying to read up as much as possible about this, but I am having a difficult time separating the relevant information from the outdated and irrelevant information.
If anyone can help me solve the problem that would be most appreciated, but even links to proper educational resources would be helpful.

According to the documentation the CredentialCache class is only for SMTP, it explicitly says that it is not for HTTP or FTP requests:
https://msdn.microsoft.com/en-us/library/system.net.credentialcache(v=vs.110).aspx
Which directly contradicts the info in the later api docs. Which one is right I don't know.
You could try using the HttpClient class. The methods and return types are different, so you would need to tweak your other a code a bit, but it would look a bit like this:
public class CommsClass
{
private HttpClient _httpClient;
public CommsClass(NetworkCredential credentials)
{
var handler = new HttpClientHandler { Credentials = credentials };
_httpclient = new HttpClient(handler);
}
public HttpResponseMessage Execute(HttpRequestMessage message)
{
var response = _httpClient.SendAsync(message).Result;
return response;
}
}
You can do all sorts of other things with the handler, and the client like set request headers or set a base address.

.NET https authentication (pingdom api)

I have to receive the API from Pingdom. The address is https://api.pingdom.com
How can I in .NET do a http get when it is https? Google gives me nothing to work with :(
Best regards
UPDATE::
Thanks for help.. Trying with PowerShell:
$NC = New-Object System.Net.NetworkCredential("USER", "PASS")
$CC = New-Object System.Net.CredentialCache
$CC.Add("api.pingdom.com", 443, "Basic", $NC)
$webclient = [System.Net.WebRequest]::Create("https://api.pingdom.com")
$webclient.Credentials = $CC
$webclient.PreAuthenticate = $true
$webclient.Method = "POST"
$webclient.GetResponse()
I get the error: Exception calling "GetResponse" with "0" argument(s): "The remote server returned an error: (401) Unauthorized."
Any good advice?

Basically,
http://www.pingdom.com/services/api-documentation-rest/
The authentication method for user credentials is HTTP Basic Access Authentication (encrypted over HTTPS). This means you will provide your credentials every time you make a request. No sessions are used.
HTTP Basic Access Authentication is well documented both here and on MSDN.
This answer together with the API docs should get you started down the right path.
https://stackoverflow.com/a/1127295/64976
Assuming you use a WebRequest, you attach a CredentialCache to your request:
NetworkCredential nc = new NetworkCredential("user", "password");
CredentialCache cc = new CredentialCache();
cc.Add("www.site.com", 443, "Basic", nc);
The CredentialCache is used to be able to set Basic authentication.

You should be able to set the credentials of the webclient and then any time a login is needed, it will supply what you gave it.
WebClient webClient = new WebClient();
webClient.Credentials = new System.Net.NetworkCredential("UserName", "Password", "Domain");

C# Network credentials not being passed to server?

Edit: Using:
byte[] authBytes = System.Text.Encoding.UTF8.GetBytes(user + ":" + password);
wr.Headers["Authorization"] = "Basic " + Convert.ToBase64String(authBytes);
Seems to work fine.
I have an application that communicates with a CMS. I'm currently trying to get this client to be able to upload text/xml data to the CMS using a "POST" method.
I can pass this through using curl perfectly fine:
curl -u user:password -H "Content-Type:text/xml" -d "<element>myXML</element>" serverURL
However, trying to use the HttpWebRequest in C# I can't get the server to return what I want it to. So I fired up Wireshark and had a look at what was actually being passed through, and it's pretty much identical except for the fact that when using curl I can see:
Authorization: Basic <a bunch of hex>=\r\n
Credentials: user:password
In the HTTP header fields, while in the output from my client, these header fields are simply not present. ("Credentials:" isn't actually there in plain text, it's a subtree of "Authorization:" - so I'm not sure where it's getting it from, but the username and password are correct.)
The C# code I'm trying to use to set the credentials for the webrequest is something like this:
NetworkCredential myCred = new NetworkCredential(
user, password, serverURL);
CredentialCache myCache = new CredentialCache();
myCache.Add(new Uri(serverURL), "Basic", myCred);
HttpWebRequest wr = (HttpWebRequest) HttpWebRequest.Create(serverURL);
wr.Credentials = myCache;
I've tried just setting the credentials like this too (and without specifying serverURL):
wr.Credentials = new NetworkCredential(user,password,serverURL);
But that still doesn't make it show up in wireshark. Does anyone have any idea if:
A) That authorization information should actually be in the HTTP header for this to work, and
B) If it is - then how do I make C# put it in? I only seem to be able to find decent examples using the default credentials, which doesn't apply to what I'm doing.
Thanks in advance.

.NET's WebRequest has an infuriating default behavior where it only sends credentials after receiving an HTTP 401 Not Authorized response.
Manually adding the credentials header (as you've done) seems to be the best solution available.
More details in this post

How can I know from a web application whether another web application is authenticated or not without using redirects?

I have two asp.net web applications AppClient and AppServer. How can I know from AppClient whether AppServer is Authenticated or not?
I do not want to redirect to the server then redirect to the client. I hope if I can do that in background.
What I've tried so far:
In the server (AppServer): I created a page called IsAuthenticated.aspx in the AppServer and made it write in the Response whether AppServer is authenticated or not.
Response.Write(User.Identity.IsAuthenticated);
And in the client, I wrote the following code:
string url = "http://appServer/isauthenticated.aspx";
System.Net.HttpWebResponse response;
System.Net.HttpWebRequest request;
request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "GET";
request.ContentType = "text/html";
request.Proxy.Credentials = CredentialCache.DefaultCredentials;
request.Credentials = CredentialCache.DefaultCredentials;
request.ServicePoint.ConnectionLimit = 25;
response = (HttpWebResponse)request.GetResponse();
StreamReader sr = new StreamReader(response.GetResponseStream());
string result = sr.ReadToEnd();
This code above works well but it does not do the expected result. The response returns always false even the AppServer is Authenticated.
Actually I understand what is happenning and seems my implementation is not true. Do you have any other suggestion ?

You mean that you have two websites (say, www.client.com and www.server.com) and your user is authenticated on www.server.com, and you want to check if he's authenticated there when he comes to www.client.com. Did I understand you correctly?
In this case your code obviously won't work (unless you use Windows authentication and impersonation).
Please, elaborate what you want to do. I see two cases, and the solution differs for each of them.
1) You don't want to check username and password on your www.client.com site, and you want to rely on www.server.com's account database. This is similar to OpenID, Google SSO etc.
In this case you need to implement a remote authentication function on www.server.com. Here's a draft:
[ServiceContract]
public interface IRemoteAuth
{
[OperationContract]
public bool CredentialsValid(string login, string hashedPassword)
{
return MyDatabaseHelper.IsPasswordValid(login, hashedPassword);
}
}
Typically, these server methods are implemented as ASMX web services or WCF services (the second one seems the preferred way).
2) You want www.client.com to know if the user is online and authenticated RIGHT NOW on www.server.com. In this case, you need to keep track of the currently online users on www.server.com (ASP.NET doesn't provide this information automatically IMO). Then you should design a service (again, ASMX or WCF) that answers a question 'is the user called XXX currently online or not?'.
[ServiceContract]
public interface IOnlineUsersInfo
{
[OperationContract]
public bool IsUserOnline(string login)
{
return MyOnlineUserList.Instance.Any(user => user.Login == login);
}
}

Is the AppClient setup to use impersonation? If not the AppClient will always send the credentials of the user the application is running under which usually will be the network service account.

Difficulty with BugzScout.net from behind a proxy

I'm attempting to use Fogbugz's BugzScout in order to automatically submit unhanded application exceptions to my Fogbugz on demand Account. I've written up a wrapper class for it and everything appears to be just groovy - on my box. Testing the same code in the production environment, behind a Proxy that requires authentication, I have had nothing but issues.
I went to work modifying the BugzScout code in order to get it to authenticate with the Proxy, and after trying many different methods suggested via a Google search, found one that works! But now I'm getting an "Connection actively refused" error from Fogbugz itself, and I don't know what to do.
Here is the code where the BugzScout connects via a .net WebClient to submit a new case, with my modifications to deal with our Proxy. What am I doing that would cause Fogbugz to refuse my request? I've removed all non web-client related code from the procedure for ease of reading.
public string Submit(){
WebClient client = new WebClient();
WebProxy proxy = new WebProxy();
proxy.UseDefaultCredentials = true;
client.Proxy = proxy;
Byte[] response = client.DownloadData(fogBugzUrl);
string responseText = System.Text.Encoding.UTF8.GetString(response);
return (responseText == "") ? this.defaultMsg : responseText;
}
The url is correct and the case is filled in properly- this has been verified.
EDIT: Additional info.
Using Fogbugz on Demand.
Using FogBugz.net code in it's entirety, with only these additions
WebProxy proxy = new WebProxy();
proxy.UseDefaultCredentials = true;
client.Proxy = proxy;
Error occurs when attempting to connect to both https://oursite.fogbugz.com/scoutsubmit.asp and http://oursite.fogbugz.com//scoutsubmit.asp (except one says port 443, and the other port 80, obviously)
I don't know anything about web authentication so I can't tell you what kind I'm using- if you tell me where to look I'd be happy to answer that for you.

Got the fix from Fogbugz- this is the appropriate network code to get though the proxy authentication and not mis-authenticate with Bugzscout.
WebRequest.DefaultWebProxy.Credentials = CredentialCache.DefaultNetworkCredentials;
WebRequest request = WebRequest.Create(fogBugzUrl);
request.ContentType = "application/x-www-form-urlencoded";
request.Method = "POST";
request.Proxy.Credentials = CredentialCache.DefaultNetworkCredentials;
Stream requestStream = request.GetRequestStream();
requestStream.Write(bytes, 0, bytes.Length);
requestStream.Close();

Is your fogbugzUrl using HTTP Basic Authentication? Is it SSL (hosted on On Demand?)
The connection actively refused message would be coming from the web server itself, not really FogBugz.
Can you post the HTTP Status Code?
One thing to note if you are using FogBugz On Demand is you HAVE to use the https:// url (not the http url).