We Keep Coding

C# m2mqtt to connect to AWS broker using Root CA, key, and certificate

I am trying to use M2MQtt library to connect to AWS MQTT broker using a root CA, client certificate and key. I am using the following C# client connection code
MqttClient client = new MqttClient(
endPoint,
MqttSettings.MQTT_BROKER_DEFAULT_SSL_PORT,
true,
new X509Certificate2(#"ca.pem"),
new X509Certificate2(#"certificate.pem"),
MqttSslProtocols.TLSv1_2
);
client.Connect(Guid.NewGuid().ToString());
however, this fails with a FormatException error. It's probably related to the fact that I don't know where to pass in the private key for this connection. This is something that I already have working, prototyped in Python using AWSIoTPythonSDK (see below)
from AWSIoTPythonSDK.MQTTLib import AWSIoTMQTTClient
f = open('mqttEndpoint.txt', 'r')
awsHost = f.read()
f.close()
myAWSIoTMQTTClient = AWSIoTMQTTClient('foo')
myAWSIoTMQTTClient.configureEndpoint(awsHost, 8883)
myAWSIoTMQTTClient.configureCredentials('ca.pem', 'id_rsa', 'certificate.pem')
Does anyone know how this is supposed to work?

I figured out my problem. The clue was the fact that to properly authenticate against AWS, you need to provide both the certificate (a PEM in my case) as well as the private key, which I could not figure out how to pass into MqttClient() constructor, because it takes only one "certificate".
The solution is to use a PFX/P12 certificate, which includes inside it both a PEM and a private key (thank you, Microsoft, for being different). There are many resources that explain how to create a PFX from a PEM+key (i.e. here, here, here, here, etc). Then you have to use a X509Certificate2() class to pull in the PFX file (that '2' is
MqttClient client = new MqttClient(
endPoint,
MqttSettings.MQTT_BROKER_DEFAULT_SSL_PORT,
true,
rootCa,
new X509Certificate2(#"certificate.pfx", #""); // My PFX was created with a blank password, hence empty string as 2nd arg
MqttSslProtocols.TLSv1_2
);
client.Connect(Guid.NewGuid().ToString());

UWP app HttpClient HTTPS client certificate problems

I'm writing a UWP app in C# that is eventually destined for IoT, but right now I've only been debugging locally. I'm using Windows.Web.Http.HttpClient to connect to a self-hosted WCF REST web service that I've also written and have running as a Console app on the same machine for testing. The service requires mutual authentication with certificates, so I have a CA cert, service cert, and client cert.
My UWP code works like this:
Check app cert store for client cert and CA cert installed.
If not, install from PFX file and CER file, respectively.
Attach the Certificate to the HttpBaseProtocolFilter and add the filter to the HttpClient
Call the HttpClient.PostAsync
After I call PostAsync I get the following error: An Error Occurred in the Secure Channel Support. After plenty of searching online, and by common sense, I'm pretty sure HttpClient is barfing because of a problem establishing the mutually-authenticated SSL connection. But based on my troubleshooting I can't figure why.
To troublshoot further, I've written a plain old Console app using System.Net.Http.HttpClient, attached the client certificate to the request and everything works great. Sadly, System.Net isn't fully supported on UWP. I've also tried NOT attaching the certificate to the UWP HttpClient and the app prompts me with a UI to select an installed certificate. I select the correct cert and still get the same exception (this at least lets me know the cert is installed correctly and validating properly with the CA from the app's perspective). In additon, I hit the GET on the web service from a browser, select the client cert when prompted, and am able to download a file.
I've tried using Fiddler and, I assume because of the way it proxies traffic, it seems to work a little bit further, except my web service rejects the request as Forbidden (presumably because Fiddler is not including the correct client cert in the request). I haven't hit up Wireshark yet because it's a pain to get Wireshark to work using localhost on Windows.
My next step is to start changing the web service to not require client authentication and see if that is the problem.
Two questions: Why is Windows.Web.Http.HttClient not working in this case? And, less important, any recommendations on good HTTP monitoring tools to help me debug this further?

This MSDN post proved to have the answer. Seems like an oversight on MS part requiring a separate, meaningless call to the API beforehand. Oh well.
http://blogs.msdn.com/b/wsdevsol/archive/2015/03/26/how-to-use-a-shared-user-certificate-for-https-authentication-in-an-enterprise-application.aspx
Excerpt from the article:
However, the security subsystem requires user confirmation before allowing access to a certificates private key of a certificate stored in the shared user certificates store. To complicate matters, if a client certificate is specified in code then the lower level network functions assume the application has already taken care of this and will not prompt the user for confirmation.
If you look at the Windows Runtime classes related to certificates you won’t find any method to explicitly request access to the certificate private key, so what is the app developer to do?
The solution is to use the selected certificate to 'Sign' some small bit of data. When an application calls CryptographicEngine.SignAsync, the underlying code requests access to the private key to do the signing at which point the user is asked if they want to allow the application to access the certificate private key. Note that you must call 'Async' version of this function because the synchronous version of the function: Sign, uses an option that blocks the display of the confirmation dialog.
For example:
public static async Task<bool> VerifyCertificateKeyAccess(Certificate selectedCertificate)
{
bool VerifyResult = false; // default to access failure
CryptographicKey keyPair = await PersistedKeyProvider.OpenKeyPairFromCertificateAsync(
selectedCertificate, HashAlgorithmNames.Sha1,
CryptographicPadding.RsaPkcs1V15);
String buffer = "Data to sign";
IBuffer Data = CryptographicBuffer.ConvertStringToBinary(buffer, BinaryStringEncoding.Utf16BE);
try
{
//sign the data by using the key
IBuffer Signed = await CryptographicEngine.SignAsync(keyPair, Data);
VerifyResult = CryptographicEngine.VerifySignature(keyPair, Data, Signed);
}
catch (Exception exp)
{
System.Diagnostics.Debug.WriteLine("Verification Failed. Exception Occurred : {0}", exp.Message);
// default result is false so drop through to exit.
}
return VerifyResult;
}
You can then modify the earlier code example to call this function prior to using the client certificate in order to ensure the application has access to the certificate private key.

Add the Certificate file your Project
Add the Certificate to the Manifested file (give file path in attachment)
the Frist Service Call of in Ur Project use to ignore the certificate validation Following Code is most Suitable for Login Function.
try
{
var filter = new HttpBaseProtocolFilter();
filter.IgnorableServerCertificateErrors.Add(ChainValidationResult.Expired);
filter.IgnorableServerCertificateErrors.Add(ChainValidationResult.Untrusted);
filter.IgnorableServerCertificateErrors.Add(ChainValidationResult.InvalidName);
filter.IgnorableServerCertificateErrors.Add(ChainValidationResult.RevocationFailure);
filter.IgnorableServerCertificateErrors.Add(ChainValidationResult.RevocationInformationMissing);
filter.IgnorableServerCertificateErrors.Add(ChainValidationResult.WrongUsage);
filter.IgnorableServerCertificateErrors.Add(ChainValidationResult.IncompleteChain);
Windows.Web.Http.HttpClient client = new Windows.Web.Http.HttpClient(filter);
TimeSpan span = new TimeSpan(0, 0, 60);
var cts = new CancellationTokenSource();
cts.CancelAfter(span);
var request = new Windows.Web.Http.HttpRequestMessage()
{
RequestUri = new Uri(App.URL + "/oauth/token"),
Method = Windows.Web.Http.HttpMethod.Post,
};
//request.Properties. = span;
string encoded = System.Convert.ToBase64String(System.Text.Encoding.GetEncoding("ISO-8859-1").GetBytes(Server_Username + ":" + Server_Password));
var values = new Dictionary<string, string>
{ { "grant_type", "password" },{ "username", Uname}, { "password", Pwd }};
var content = new HttpFormUrlEncodedContent(values);
request.Headers.Add("Authorization", "Basic " + encoded);
request.Content = content;
User root = new User();
using (Windows.Web.Http.HttpResponseMessage response = await client.SendRequestAsync(request).AsTask(cts.Token))
{
HttpStatusCode = (int)response.StatusCode;
if (HttpStatusCode == (int)HttpCode.OK)
{
using (IHttpContent content1 = response.Content)
{
var jsonString = await content1.ReadAsStringAsync();
root = JsonConvert.DeserializeObject<User>(jsonString);
App.localSettings.Values["access_token"] = root.Access_token;
App.localSettings.Values["refresh_token"] = root.Refresh_token;
App.localSettings.Values["expires_in"] = root.Expires_in;
var json = JsonConvert.SerializeObject(root.Locations);
App.localSettings.Values["LocationList"] = json;
App.localSettings.Values["LoginUser"] = Uname;
}
}
}
}
catch (Exception ex)
{
ex.ToString();
}

Access denied when using Net.Tcp Port Sharing Service

This question is related to Starting processes under specific credentials from a Windows service, but it's a different problem.
I've started a process from a Windows service in the System session (0) under specific credentials, but it is unable to listen to a port sharing URL. It's using a "Worker" domain account on a Windows Server 2008 machine.
My SMSvcHost.exe.config file: http://pastie.org/private/jxed8bdft0eir5uc371pq
I've restarted the services and the machine as well, but it's still giving me this exception:
System.ServiceModel.CommunicationException: The service endpoint failed to listen on the URI 'net.tcp://localhost:5400/Agent/384' because access was denied. Verify that the current user is granted access in the appropriate allowAccounts section of SMSvcHost.exe.config. ---> System.ComponentModel.Win32Exception: Access is denied
at System.ServiceModel.Activation.SharedMemory.Read(String name, String& content)
at System.ServiceModel.Channels.SharedConnectionListener.SharedListenerProxy.ReadEndpoint(String sharedMemoryName, String& listenerEndpoint)
My ProcessHelper code that starts the process: http://pastie.org/private/iytqehsdfujrgil1decda. I'm calling the StartAsUserFromService method.
I suppose the link between the SID in the config and the account the process is running under is somehow not being made. But why?
EDIT:
I've double-checked that the config I'm editing is used by the service. I've tried adding the System account and Everyone explicitly, but it's still giving me an access denied error. It's like it's not looking at that config at all.
How can I find where the missing permission is?
EDIT:
I reinstalled .NET 4.5.1 on the machine and all the Windows updates, still no luck.
EDIT:
Is this the correct way of duplicating a user token to allow it to use port sharing? Specifically the SecurityDescriptor bit?
private static ImpersonationResult ImpersonateUser(string domain, string username, string password)
{
IntPtr token = IntPtr.Zero;
IntPtr primaryToken = IntPtr.Zero;
try
{
// Get token
bool bImpersonated = LogonUser(
username,
domain,
password,
(int)LogonType.NetworkClearText,
(int)LogonProvider.Default,
ref token);
if (!bImpersonated)
{
throw new Exception(string.Format("Failed to impersonate identity. Error code: {0}", Marshal.GetLastWin32Error()));
}
SecurityDescriptor sd = new SecurityDescriptor();
IntPtr ptr = Marshal.AllocCoTaskMem(Marshal.SizeOf(sd));
Marshal.StructureToPtr(sd, ptr, false);
InitializeSecurityDescriptor(ptr, 1);
sd = (SecurityDescriptor)Marshal.PtrToStructure(ptr, typeof(SecurityDescriptor));
// Set up security
bool bDecriptorSet = SetSecurityDescriptorDacl(
ref sd,
true,
IntPtr.Zero,
false);
if (!bDecriptorSet)
{
throw new Exception(string.Format("Failed to set security descriptor. Error code: {0}", Marshal.GetLastWin32Error()));
}
SecurityAttributes processAttributes = new SecurityAttributes();
processAttributes.lpSecurityDescriptor = ptr;
processAttributes.nLength = (uint)Marshal.SizeOf(sd);
processAttributes.bInheritHandle = true;
// Duplicate token
bool bTokenDuplicated = DuplicateTokenEx(
token,
0,
ref processAttributes,
(int)SecurityImpersonationLevel.SecurityImpersonation,
(int)TokenType.TokenPrimary,
ref primaryToken);
if (!bTokenDuplicated)
{
throw new Exception(string.Format("Failed to duplicate identity token. Error code: {0}", Marshal.GetLastWin32Error()));
}
SecurityAttributes threadAttributes = new SecurityAttributes();
threadAttributes.lpSecurityDescriptor = IntPtr.Zero;
threadAttributes.nLength = 0;
threadAttributes.bInheritHandle = false;
// Got the token
return new ImpersonationResult()
{
Token = primaryToken,
ProcessAttributes = processAttributes,
ThreadAttributes = threadAttributes
};
}
finally
{
FreeToken(token);
}
}
private static void FreeToken(IntPtr token)
{
if (token != IntPtr.Zero)
{
CloseHandle(token);
}
}
EDIT:
Here's the app.config bit of my process that enables port sharing: http://pastie.org/private/8ekqeps4d7rmo7hnktsw
Here's an app.config bit of the service that starts the process: http://pastie.org/private/nqqcwz8bvjb5fzp48yavbw. It has no problems using Port Sharing because it's running under the System account.
The Port Sharing service itself is enabled, and I already mentioned that I've restarted the machine several times.
The "Application Server" role is not installed, but when I go to add it, the TCP Port Sharing Role is already ticked and greyed out, so something else must have installed it. Does it come with .NET 4.5.1?

Preface: Please use for one question one thread...
PortSharing: WHERE have you enabled port sharing? We cannot see this in your configuration file. For more infos see: How to: Configure a Windows Communication Foundation Service to Use Port Sharing
Have you installed the "Application Server" role on your server? See also: Checklist: Use TCP Port Sharing to Allow Multiple WCF Applications to Use the Same TCP Port
Is port sharing enabled on the system? See: How to: Enable the Net.TCP Port Sharing Service
Also, have you restarted your server? Sometimes this is needed (or at least all services which uses this port), see: http://blogs.msdn.com/b/joncole/archive/2010/06/10/tcp-port-sharing-access-is-denied.aspx
A comprehensive description about port-sharing is: http://blogs.msdn.com/b/andreal/archive/2009/04/05/net-tcp-ip-port-sharing.aspx
Also be aware that you must add some accounts for activation, if this is needed:
Configuring the Net.TCP Port Sharing Service
Are you sure that the SmSvcHost.exe.config allows access from your account, your process is running under?
<configuration>
<system.serviceModel.activation>
<net.tcp listenBacklog="16" <!—16 * # of processors -->
maxPendingAccepts="4"<!— 4 * # of processors -->
maxPendingConnections="100"
receiveTimeout="00:00:30" <!—30 seconds -->
teredoEnabled="false">
<allowAccounts>
<!-- LocalSystem account -->
<add securityIdentifier="S-1-5-18"/>
<!-- LocalService account -->
<add securityIdentifier="S-1-5-19"/>
<!-- Administrators account -->
<add securityIdentifier="S-1-5-20"/>
<!-- Network Service account -->
<add securityIdentifier="S-1-5-32-544" />
<!-- IIS_IUSRS account (Vista only) -->
<add securityIdentifier="S-1-5-32-568"/>
</allowAccounts>
</net.tcp>
</system.serviceModel.activation>

It turns out that the logon type was causing the permissions to not work correctly with the Port Sharing Service. I changed it to LogonType.Batch and it started working.
Full code:
ProcessHelper: http://pastie.org/private/dlkytj8rbigs8ixwtg
TokenImpersonationContext: http://pastie.org/private/nu3pvpghoea6pwwlvjuq

(Just another answer that may help someone)
Turns out that in my case, logged in user did not have Administrative privilege.
Changing the account type to Administrator solved the problem.
I also did not change anything in SmSvcHost.exe.config

EWS Exchange Web service API AutodiscoverUrl exception

I get an error when I try to create an appointment:
The expected XML node type was XmlDeclaration, but the actual type is
Element.
This Exception occurs when I call AutodiscoverUrl.
I created a web service to do this.
[webMethod]
CreateAppointment()
{
var service = new ExchangeService(ExchangeVersion.Exchange2007_SP1)
{
Credentials = new WebCredentials("myAcount#gmail.com", "mypassowrd")
};
service.AutodiscoverUrl("myAcount#gmail.com");
//----------------------------------------------------------------------
var app = new Appointment(service)
{
Subject = "Meet George",
Body = "You need to meet George",
Location = "1st Floor Boardroom",
Start = DateTime.Now.AddHours(2),
End = DateTime.Now.AddHours(3),
IsReminderSet = true,
ReminderMinutesBeforeStart = 15
};
app.RequiredAttendees.Add(new Attendee("any#gmail.com"));
app.Save(SendInvitationsMode.SendToAllAndSaveCopy);
}

Some potential answers.
Passing in the wrong url or domain.
Passing in a bad email address.
Rebuilding the Windows Profile can sometimes help. (Warning: have an IT Admin do this). And it might be overkill.
A user could have an old, bad, or multiple outlook profiles set up. The email server name could be bad in the outlook profile. (See Control Panel > Mail)

Autodiscover depends on two things:
DNS entries that point from the users mail domain to the Autodiscover data on the Exchange server. Typically you would have a DNS entry with the name autodiscover.domain.com, but there's more than one way of setting this up for different versions of Exchange. If the correct DNS entry doesn't exist, auto-discovery will fail.
Autodiscover data hosted on the Exchange server (I believe it's an XML file) and accessed over HTTP. If this isn't accessible (perhaps it's behind a firewall) then auto-discovery will fail.
Check the appropriate DNS entries and autodiscover information is accessible to your client.

WCF security issue

Some background:
in order to provide authentication I'm using certificates on client and server side (WCF) and use one certificate for all clients (manually loading it from application directory - not the safest way, but it doesn't require to manage certificate storage and making installation more difficult):
AddressHeader hostHdr = AddressHeader.CreateAddressHeader(ServiceFactory.CLIENT_HOST_HEADER, ServiceFactory.NAMESPACE, hostName);
builder.Headers.Add(hostHdr);
builder.Identity = new X509CertificateEndpointIdentity(GetServiceCertificate(name));
_factory = new ChannelFactory<T>(name, builder.ToEndpointAddress());
_factory.Credentials.ClientCertificate.Certificate = GetClientCertificate(name);
X509ServiceCertificateAuthentication auth = _factory.Credentials.ServiceCertificate.Authentication;
auth.CertificateValidationMode =X509CertificateValidationMode.Custom;
auth.CustomCertificateValidator = new CustomCertificateValidator(new[] {GetServiceCertificate(name)});
This is client side, and serverside host setting up looks like this:
private void CertificateSetup(ServiceHost host)
{
if (ServiceCertificate != null)
host.Credentials.ServiceCertificate.Certificate = ServiceCertificate;
X509ClientCertificateAuthentication authentication =
host.Credentials.ClientCertificate.Authentication;
authentication.CertificateValidationMode =
X509CertificateValidationMode.Custom;
authentication.CustomCertificateValidator =
new CustomCertificateValidator(new [] {ClientCertificate});
}
That works fine and allows to sign messages, but as far as security mode set in following way:
<security mode="Message">
<message clientCredentialType="Certificate" />
</security>
But i need
string name = OperationContext.Current.ServiceSecurityContext.WindowsIdentity.Name;
somehow to obtain WindowsIdentity in ServiceSecurityContext.
Mixed (Transport and Message) security mode is not helpful, because I don't know why but even if i set Windows clientCredentials in config for Transport part mode infrastructure tries to establish SSL connection.
Any ideas ????
Certificates are used for message signing,i.e. proving that other side is either real client or service (man-in-the-middle). But authorization in system being developed partially relies on WindowsIdentity in ServiceSecurityContect. All i want-include WindowsIdentity in sec.context, meanwhile PrimaryIdentity is X509CertIdentity.
So i need to know on serverside which domain user requested service operation.

Try this link re: Impersonation:
http://msdn.microsoft.com/en-us/library/ms730088.aspx
section on - "Mapping a Client Certificate to a Windows Account", seems like what you're after.
with config on client:
authentication mapClientCertificateToWindowsAccount="true"
Client code:
' Create a binding that sets a certificate as the client credential type.
Dim b As WSHttpBinding = New WSHttpBinding()
b.Security.Message.ClientCredentialType = MessageCredentialType.Certificate
' Create a service host that maps the certificate to a Windows account.
Dim httpUri As Uri = New Uri("http://localhost/Calculator")
Dim sh As ServiceHost = New ServiceHost(GetType(HelloService), httpUri)
sh.Credentials.ClientCertificate.Authentication.MapClientCertificateToWindowsAccount = True
Hope that helps

If you're using message security with certificates at each end securing it, I'm not quite sure why you would need a windows identity? What's the reasoning behind that?
Maybe this link will be of use - sections 6 & 7 have details of config settings for certificate authentiaction, which works similarly to SSL:
http://www.codeproject.com/KB/WCF/wcf_certificates.aspx