We Keep Coding

How to submit a CSR to Windows CA for signing from C#?

My scenario:
I have an EC2 instance running Windows Server 2016 DataCenter OS in it. This EC2 instance is configured to work as CA and IIS server (I am testing cloudhsm so it is okay for me to have multiple services in the same server). My CA is configured with RSA#Cavium Key Storage Provider (as it supports AWS CloudHSM Key Storage Provider). Now from my sample .Net WebAPI app, I can create a CSR using below code:
private async Task<CertificateSigningResponse> ActualSigningAsync(CertificateSigningRequest csr)
{
CertificateSigningResponse certificateSigningResponse;
try
{
using (RSA rsa = RSA.Create(csr.KeySize))
{
CertificateRequest request = new CertificateRequest("CN=servername-CA2", rsa, HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1);
OidCollection oidCollection = new OidCollection();
oidCollection.Add(new Oid("1.3.6.1.5.5.7.3.8"));
request.CertificateExtensions.Add(new X509BasicConstraintsExtension(true, false, 0, true));
request.CertificateExtensions.Add(new X509EnhancedKeyUsageExtension(oidCollection, true));
request.CertificateExtensions.Add(new X509SubjectKeyIdentifierExtension(request.PublicKey, false));
var serialNumberInBytes = System.Text.Encoding.UTF8.GetBytes(Guid.NewGuid().ToString());
/*
// below line creates a self signed certificate
var selfSignedCertificate = request.Create(
new X500DistinguishedName("CN=servername-CA2"),
X509SignatureGenerator.CreateForRSA(rsa, RSASignaturePadding.Pkcs1),
DateTimeOffset.Now,
DateTimeOffset.Now.AddMonths(1),
serialNumberInBytes);
*/
var certSigningRequest = request.CreateSigningRequest(X509SignatureGenerator.CreateForRSA(rsa, RSASignaturePadding.Pkcs1));
if (certSigningRequest != null)
{
certificateSigningResponse = PopulateCertificateSigningResponse(JsonConvert.SerializeObject(certSigningRequest), $"Sample Cert with Sl#: {certSigningRequest}");
}
else
{
certificateSigningResponse = PopulateCertificateSigningResponse("", $"Error: {request.PublicKey}");
}
}
}
catch (Exception ex)
{
certificateSigningResponse = PopulateCertificateSigningResponse(string.Empty, ex.Message);
}
return certificateSigningResponse;
}
Now, I want to send this CSR to my CA (which is running on EC2) and get it signed. Once signed, I want to send the signed certificate back to the user. Currently, I have hosted my sample webapi app on the same EC2 instance. This app actually exposes an endpoint, which I can hit via postman and pass in some data as request body (POST).
So, ideally, the app receives my POST request and extracts data from it, then using that data, it is creating a csr. And now I want to send this csr to my CA, get it signed and finally return the signed certificate to user. Below pseudo code may give you some idea about my requirements:
//pseudo code; not existed
// request is generated in above code
var caService = new CAService();//can use my ca info i.e. name, public dns, ip
var response = await caService.SignAsync(request);//response should be a cert
return response;
I have seen few posts using CertUtilLib/CertEnrollLib but I want to achieve this by fully managed code. Also if it is not possible, please suggest me some alternatives. Thanks.
EDIT 1 (after Michal's comment)
Below are some info for your interest:
Current Scenario:
we are using a 3rd party API to sign some critical data.
In this way, we make a POST request to that particular endpoint with the data we wanted to be signed.
The 3rd party API handles everything for us behind the scene. We know that they are using some HSM service at their end to sign data.
My intention:
I am trying to achieve the same functionality using AWS CloudHSM.
The calling process should be same, i.e. we should still call an endpoint with data and get the signed copy of the data.
For this, my understanding was to create the CSR at server end and sign it using CloudHSM from within the server.
For that, I have created a sample .Net API app that exposes an endpoint where I can post my data to be signed.
Then it is my app's responsibility to create the CSR (I did it already), and then send the CSR to a CA (this is my EC2 Windows
Server 2016 instance which has the CA feature enabled on it) for
signing (I am stuck here at the moment).

HttpClient certificate when hosted in IIS/Asp.NET Application not work

I have a strange problem.
I need to perform an SSL request using a CER client certificate, to a server that requires authentication by that certificate.
I am using the code below:
var cert = X509Certificate.CreateFromCertFile("cert.cer");
var handler = new WebRequestHandler();
handler.ClientCertificates.Add(cert);
var http_client = new HttpClient(handler);
http_client.BaseAddress = new Uri("https://service.com/");
var str_json = JsonConvert.SerializeObject(new
{
Field = "Value1",
Fiesl2 = "Value2"
});
var byteContent = new ByteArrayContent(Encoding.UTF8.GetBytes(str_json));
byteContent.Headers.Remove("Content-Type");
byteContent.Headers.Add("Content-Type", "application/json");
var res = http_client.PostAsync("ResourcePath", byteContent).Result;
res.EnsureSuccessStatusCode(); //THe error 401 ocurrs here
var res_body = res.Content.ReadAsStringAsync().Result;
This code works perfectly when I squeeze into a ConsoleApplicaiton or a WebApplication in IIS Express.
But when I squeeze exactly the same code in Local IIS or IIS Server, I get the 401-Unauthorized error. The strange thing is that using Fiddler, in this case I can not even see the request attempt.
I've already checked that path is not the problem.
The problem occours in .NET 4.5, 4.5.1, 4.5.2, 4.6, 4.6.1 and etc..
Can anyone help me out, is it any configuration that should be performed in IIS. I've researched a lot, but I did not find that specific error.

A .cer file at client side does not contain private key, so usually it won't work if mutual SSL/TLS is required by the server. You need to get a valid certificate with private key (usually a .pfx file).

Why might the server not receive a certificate attached to a request using HttpClient?

We're trying to connect to an endpoint that does nothing more than verify whether a certificate has been correctly attached.
Our code is straightforward:
ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
using (var handler = new HttpClientHandler())
{
var rawcert = File.ReadAllText(#"C:\OpenSSL\bin\cert.pem");
var rawkey = File.ReadAllText(#"C:\OpenSSL\bin\private.key");
var provider = new CertificateFromFileProvider(rawcert, rawkey);
var certificate = provider.Certificate;
handler.ClientCertificateOptions = ClientCertificateOption.Manual;
handler.ClientCertificates.Add(certificate);
handler.ServerCertificateCustomValidationCallback +=
(HttpRequestMessage req, X509Certificate2 cert2, X509Chain chain, SslPolicyErrors err) =>
{
Trace.WriteLine($"Sender: {req}");
Trace.WriteLine($"cert: {cert2}");
Trace.WriteLine($"chain: {chain}");
Trace.WriteLine($"sslPolicyErrors: {err}");
return true;
};
using (var client = new HttpClient(handler))
{
var response = await client.GetStringAsync("https://{uri}/mtlsTest");
return response;
}
}
(This is a sandbox environment for a proof of concept, so we're comfortable shortcutting the ValidationCallback for now.)
We can see the certificate is attached appears well-formed in the callback (origin and subject are as-expected), but we're getting a response from the server that indicates no certificate was attached. Why might that be?
We have also tried exporting the certificate as a pfx and attaching that instead of the original pem file, with an identical result.
Update
Thanks to useful comments below from #bartonjs and #Crypt32, we've tried adding the private key (using the handy Nuget Package from #stef-heyenrath), but although this results in the certificate that we add to the handler showing HasPrivateKey=true:
... when the ValidationCallback fires, the X509Certificate2 traces HasPrivateKey=false:
... and we get the same error as before - the server never acknowledges receiving the certificate.
If we package the .pem and the private key into a .pfx using open ssl:
pkcs12 -inkey private.key -in cert.pem -export -out cert.pfx
... then we get the same result again, and the server can find no certificate. Likewise if we install the certificate in the personal store and load it from there (same result). An inspection with Wireshark suggests the certificate is simply never being attached.
Why would this be? Full revised code above.

The initial problem here was a badly-formed certificate provided by our test host.
A further problem (once we had a good certificate to work with) was with the output of the CertificateFromFileProvider method of the OpenSSL.X509Certificate2Provider NuGet package.
For whatever reason, the certificate created would not work correctly. Taking the same input files and generating a .pfx with OpenSSL resulted in an X509Certificate2 that worked fine.

UWP app HttpClient HTTPS client certificate problems

I'm writing a UWP app in C# that is eventually destined for IoT, but right now I've only been debugging locally. I'm using Windows.Web.Http.HttpClient to connect to a self-hosted WCF REST web service that I've also written and have running as a Console app on the same machine for testing. The service requires mutual authentication with certificates, so I have a CA cert, service cert, and client cert.
My UWP code works like this:
Check app cert store for client cert and CA cert installed.
If not, install from PFX file and CER file, respectively.
Attach the Certificate to the HttpBaseProtocolFilter and add the filter to the HttpClient
Call the HttpClient.PostAsync
After I call PostAsync I get the following error: An Error Occurred in the Secure Channel Support. After plenty of searching online, and by common sense, I'm pretty sure HttpClient is barfing because of a problem establishing the mutually-authenticated SSL connection. But based on my troubleshooting I can't figure why.
To troublshoot further, I've written a plain old Console app using System.Net.Http.HttpClient, attached the client certificate to the request and everything works great. Sadly, System.Net isn't fully supported on UWP. I've also tried NOT attaching the certificate to the UWP HttpClient and the app prompts me with a UI to select an installed certificate. I select the correct cert and still get the same exception (this at least lets me know the cert is installed correctly and validating properly with the CA from the app's perspective). In additon, I hit the GET on the web service from a browser, select the client cert when prompted, and am able to download a file.
I've tried using Fiddler and, I assume because of the way it proxies traffic, it seems to work a little bit further, except my web service rejects the request as Forbidden (presumably because Fiddler is not including the correct client cert in the request). I haven't hit up Wireshark yet because it's a pain to get Wireshark to work using localhost on Windows.
My next step is to start changing the web service to not require client authentication and see if that is the problem.
Two questions: Why is Windows.Web.Http.HttClient not working in this case? And, less important, any recommendations on good HTTP monitoring tools to help me debug this further?

This MSDN post proved to have the answer. Seems like an oversight on MS part requiring a separate, meaningless call to the API beforehand. Oh well.
http://blogs.msdn.com/b/wsdevsol/archive/2015/03/26/how-to-use-a-shared-user-certificate-for-https-authentication-in-an-enterprise-application.aspx
Excerpt from the article:
However, the security subsystem requires user confirmation before allowing access to a certificates private key of a certificate stored in the shared user certificates store. To complicate matters, if a client certificate is specified in code then the lower level network functions assume the application has already taken care of this and will not prompt the user for confirmation.
If you look at the Windows Runtime classes related to certificates you won’t find any method to explicitly request access to the certificate private key, so what is the app developer to do?
The solution is to use the selected certificate to 'Sign' some small bit of data. When an application calls CryptographicEngine.SignAsync, the underlying code requests access to the private key to do the signing at which point the user is asked if they want to allow the application to access the certificate private key. Note that you must call 'Async' version of this function because the synchronous version of the function: Sign, uses an option that blocks the display of the confirmation dialog.
For example:
public static async Task<bool> VerifyCertificateKeyAccess(Certificate selectedCertificate)
{
bool VerifyResult = false; // default to access failure
CryptographicKey keyPair = await PersistedKeyProvider.OpenKeyPairFromCertificateAsync(
selectedCertificate, HashAlgorithmNames.Sha1,
CryptographicPadding.RsaPkcs1V15);
String buffer = "Data to sign";
IBuffer Data = CryptographicBuffer.ConvertStringToBinary(buffer, BinaryStringEncoding.Utf16BE);
try
{
//sign the data by using the key
IBuffer Signed = await CryptographicEngine.SignAsync(keyPair, Data);
VerifyResult = CryptographicEngine.VerifySignature(keyPair, Data, Signed);
}
catch (Exception exp)
{
System.Diagnostics.Debug.WriteLine("Verification Failed. Exception Occurred : {0}", exp.Message);
// default result is false so drop through to exit.
}
return VerifyResult;
}
You can then modify the earlier code example to call this function prior to using the client certificate in order to ensure the application has access to the certificate private key.

Add the Certificate file your Project
Add the Certificate to the Manifested file (give file path in attachment)
the Frist Service Call of in Ur Project use to ignore the certificate validation Following Code is most Suitable for Login Function.
try
{
var filter = new HttpBaseProtocolFilter();
filter.IgnorableServerCertificateErrors.Add(ChainValidationResult.Expired);
filter.IgnorableServerCertificateErrors.Add(ChainValidationResult.Untrusted);
filter.IgnorableServerCertificateErrors.Add(ChainValidationResult.InvalidName);
filter.IgnorableServerCertificateErrors.Add(ChainValidationResult.RevocationFailure);
filter.IgnorableServerCertificateErrors.Add(ChainValidationResult.RevocationInformationMissing);
filter.IgnorableServerCertificateErrors.Add(ChainValidationResult.WrongUsage);
filter.IgnorableServerCertificateErrors.Add(ChainValidationResult.IncompleteChain);
Windows.Web.Http.HttpClient client = new Windows.Web.Http.HttpClient(filter);
TimeSpan span = new TimeSpan(0, 0, 60);
var cts = new CancellationTokenSource();
cts.CancelAfter(span);
var request = new Windows.Web.Http.HttpRequestMessage()
{
RequestUri = new Uri(App.URL + "/oauth/token"),
Method = Windows.Web.Http.HttpMethod.Post,
};
//request.Properties. = span;
string encoded = System.Convert.ToBase64String(System.Text.Encoding.GetEncoding("ISO-8859-1").GetBytes(Server_Username + ":" + Server_Password));
var values = new Dictionary<string, string>
{ { "grant_type", "password" },{ "username", Uname}, { "password", Pwd }};
var content = new HttpFormUrlEncodedContent(values);
request.Headers.Add("Authorization", "Basic " + encoded);
request.Content = content;
User root = new User();
using (Windows.Web.Http.HttpResponseMessage response = await client.SendRequestAsync(request).AsTask(cts.Token))
{
HttpStatusCode = (int)response.StatusCode;
if (HttpStatusCode == (int)HttpCode.OK)
{
using (IHttpContent content1 = response.Content)
{
var jsonString = await content1.ReadAsStringAsync();
root = JsonConvert.DeserializeObject<User>(jsonString);
App.localSettings.Values["access_token"] = root.Access_token;
App.localSettings.Values["refresh_token"] = root.Refresh_token;
App.localSettings.Values["expires_in"] = root.Expires_in;
var json = JsonConvert.SerializeObject(root.Locations);
App.localSettings.Values["LocationList"] = json;
App.localSettings.Values["LoginUser"] = Uname;
}
}
}
}
catch (Exception ex)
{
ex.ToString();
}

Error connecting with client certificate from smart card / etoken to WCF Service

We have a WCF service configured to work with client certificates.
Our client is a .net 3.5 WPF application.
While testing, it works perfectly with certificates generated by Microsoft CA and others.
While testing it with a physical token (Aladdin/Safenet eToken Pro 64k) it also works well.
(On first try connecting to server the “Token Logon” window pops up.
After successful authentication to the token the server request works, and the next try to connect to server succeed without the token logon message showing)
Now, if we remove the token and re-insert it, when trying to connect with the same certificate we get an error “The request was aborted: Could not create SSL/TLS secure channel”. HResult 0x80131509
The only way to make it work again is restart the application.
Certificates are retrieved with System.Security.Cryptogragpy:
var store = new X509Store("My", StoreLocation.CurrentUser);
store.Open(OpenFlags.ReadOnly);
//Getting the right certificate and then:
store.Close();
The connection to server is with this code:
string uriStr = txtUrl.Text + "/rest/auth/strong/Ping";
var client = WebRequest.Create(new Uri(uriStr)) as HttpWebRequest;
client.Method = "POST";
client.ContentType = "application/text; charset=unicode;";
client.ContentLength = 0;
client.PreAuthenticate = false;
client.KeepAlive = false;
client.ClientCertificates.Add(certificate); //Cert attached to the request
using (var res = client.GetResponse() as HttpWebResponse)
{
using (var responseStream = res.GetResponseStream())
{
using (var reader = new StreamReader(responseStream))
{
string s = reader.ReadToEnd();
ShowAndWriteToFile(s);
}
}
}
I looked at Verbose logging and Wireshark captures, Couldn't find the problem from there.
Other stuff I tried:
Clearing pin code cache – when clearing it before removing token, then on next attempt to connect to server the “Token logon” window pops up.
Clearing it after remove and re-insert didn’t change anything.
Restarting the WCF Service – after the token re-inserted, still same result.
Checking how other operations on the certificate behaves before and after token remove and re-insert.
For instance, if I Sign and verify some message with the certificate after the removal and re-insert of the token then the “Token logon” window pops up again and it continues working.
Trying to connect with self-created certificate put in the store – Worked, then deleting it from store and re-adding it – still worked.