I've got a communication system based on TcpClient, and it works great except for when it's doing HTTPS to a particular IP. Then it starts to fail.
By using a browser or HttpWebRequest, I have no problems doing HTTPS to that IP.
I've created a test program to narrow my problem down to its basic essence, you can have a look at it here if you want: TestViaTcp
That test program works perfectly for basic HTTP to the same IP, it always produces a successful response to the request. I put it in a loop, trigger it with a keypress, it will continue to succeed all day long. As soon as I toggle the HTTPS, I get a recurring pattern. It'll work, then it won't, success followed by failure followed by success back and forth all day long.
The particular failure I keep getting is this one:
{"Authentication failed because the remote party has closed the transport stream."}
[System.IO.IOException]: {"Authentication failed because the remote party has closed the transport stream."}
Data: {System.Collections.ListDictionaryInternal}
HelpLink: null
InnerException: null
Message: "Authentication failed because the remote party has closed the transport stream."
Source: "System"
TargetSite: {Void StartReadFrame(Byte[], Int32, System.Net.AsyncProtocolRequest)}
And here's the stack trace attached to that:
at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult)
at System.Net.Security.SslStream.AuthenticateAsClient(String targetHost, X509CertificateCollection clientCertificates, SslProtocols enabledSslProtocols, Boolean checkCertificateRevocation)
at DeriveClassNameSpace.Services.Web.TcpMessaging.TestViaTcp(IPEndPoint endpoint, String auth, Boolean useSSL)
HttpWebRequest and the browser are both (IIRC) using the Win32 libraries to handle the back-and-forth communication, while TcpClient is (AFAIK) using the managed .net Socket class, so I'm sure there's a large difference between them. I do need to do this with TcpClient, so unfortunately I can't just "use HttpWebRequest since I know I can make it work".
The biggest hint as to what the problem is here is likely the "works, doesn't, works, doesn't" pattern, what's causing that? What can I do to avoid the IOException I'm getting? Is there some way to get the "always works" behaviour that I can see when I do the HTTPS with HttpWebRequest?
There should be something I can do with TcpClient to get it to act and react just like HttpWebRequest does, but I'm not there yet. Any ideas?
Note: The server I'm communicating with is configurable as to what port it listens on and what protocol it expects, but is otherwise completely unmodifiable.
Also note: I've read that .net 3.5 had this particular issue with SslStream before SP1, but I've got SP1 and my program is built targetting 3.5 so I'm assuming that this isn't a 'known bug' I'm running into here.
Wouldn't you know it, after I spend the time forming the question is when I stumble upon the answer.
Here's the relevant documentation: jpsanders blog entry
The important part was this:
If the stack from the exception includes something similar to this: System.IO.IOException: Authentication failed because the remote party has closed the transport stream. It is possible that the server is an older server does not understand TLS and so you need to change this as specified in the 915599 kb to something like this: ServicePointManager.SecurityProtocol= SecurityProtocolType.Ssl3;Before you make any HTTPS calls in your application.
So I change my accepted protocols to this: (removing the possibility of TLS)
SslProtocols protocol = SslProtocols.Ssl2 | SslProtocols.Ssl3;
And everything works great.
I was allowing TLS, so it tries that first, and the server doesn't support it, so the stream is closed. The next time it uses Ssl2 or Ssl3 and everything's fine. Or something like that. It works, I'm a happy panda.
Related
I created an angular universal app and followed the following tutorial to use SSL for local development with SSR:
https://medium.com/#dnlcyan/local-angular-ssr-with-https-b9d305dc620d
The certificate is trusted in the browser and also the requests work on browser level. But for request coming from the node server the API throws the following exception:
Microsoft.AspNetCore.Server.Kestrel.Https.Internal.HttpsConnectionMiddleware
Failed to authenticate HTTPS connection. System.IO.IOException:
Received an unexpected EOF or 0 bytes from the transport stream. at
System.Net.Security.SslStream.g__InternalFillHandshakeBufferAsync|189_0[TIOAdapter](TIOAdapter
adap, ValueTask`1 task, Int32 minSize) at
System.Net.Security.SslStream.ReceiveBlobAsync[TIOAdapter](TIOAdapter
adapter) at
System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter
adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean
isApm) at
Microsoft.AspNetCore.Server.Kestrel.Https.Internal.HttpsConnectionMiddleware.OnConnectionAsync(ConnectionContext
context)
I read something about that the certificate might not be trusted, but I added it to windows and also the browser indicates that it's correct. I tried to create a new certificate according to this tutorial but it doesn't work as well: https://fmoralesdev.com/2020/01/03/serve-angular-app-over-https-using-angular-cli/
Does anyone know what the issue might be here?
I need to connect to an endpoint and pass the certification for validation (on their end). Following this gives me this code:
using var cert = new X509Certificate2(pathToCert);
var handler = new HttpClientHandler();
handler.ClientCertificates.Add(cobCert);
var client = new HttpClient(handler);
var response = await client.PostAsync(url, new StringContent(json, Encoding.UTF8, "application/json"));
This returns the following error:
System.Net.Http.HttpRequestException: The SSL connection could not be
established, see inner exception. --->
System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken
message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo
exception) at
System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken
message, AsyncProtocolRequest asyncRequest) at
System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32
count, AsyncProtocolRequest asyncRequest) at
System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32
count, AsyncProtocolRequest asyncRequest) at
System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32
readBytes, AsyncProtocolRequest asyncRequest) at
System.Net.Security.SslStream.PartialFrameCallback(AsyncProtocolRequest
asyncRequest)
What is the correct way to handle this?
Note: The actual app will use IHttpClientFactory but currently creating the HttpClient for testing.
Edit: Sorry forgot to say, that this endpoint with this cert works in Postman
The error complains that the server certificate is invalid. Perhaps it has expired or revoked. Sometimes the root authority certificate itself is compromised and revoked. This is rare but it has happened in the past.
Perhaps a test server uses a self-signed certificate for development. Self-signed certificates are invalid by definition because they aren't signed by a trusted authority.
HTTPS isn't used to encrypt the connection, it's used to ensure that no other server gets "in the middle" of the client and server. It does that by ensuring the remote server is who its certificate says it is. Obviously, the certificate needs to be valid - which means it needs to be signed by a trusted source. Without validation someone could add a malicious proxy to your network that would pose as the remote server, intercept your calls and send them to the remote server.
If the server service has expired or been revoked, the server admin will have to renew it.
To use a self-signed certificate without compromising security is to trust it. The .NET Core SDK will add the self-signed certificate used by .NET Core projects to the trusted certificates of the local machine. On other machines you can navigate to the API URL with a browser, click on the warning that appears at the left of the address bar and trust the certificate
I am new to Amazon Web Services Lambda functions, and I decided to write a small function to ping a website. We have a VPC subnet and security group already set up, which I applied to my function when I published it.
When I run this small function in debug mode through Visual Studio it works okay, but once I publish it to AWS Lambda it fails and I get an exception:
System.Net.NetworkInformation.PingException: An exception occurred during a Ping request. ---> System.PlatformNotSupportedException: The system's ping utility could not be found.\n at System.Net.NetworkInformation.Ping.SendWithPingUtility(IPAddress address, Byte[] buffer, Int32 timeout, PingOptions options)\n at System.Net.NetworkInformation.Ping.SendPingAsyncCore(IPAddress address, Byte[] buffer, Int32 timeout, PingOptions options)\n at System.Net.NetworkInformation.Ping.GetAddressAndSendAsync(String hostNameOrAddress, Int32 timeout, Byte[] buffer, PingOptions options)\n --- End of inner exception stack trace ---\n at System.Net.NetworkInformation.Ping.GetAddressAndSendAsync(String hostNameOrAddress, Int32 timeout, Byte[] buffer, PingOptions options)\n at TestPingWebsite.Function.FunctionHandler(String input, ILambdaContext context) in C:\Users\GWhite\source\TestingWebsite\TestPingWebsite\TestPingWebsite\Function.cs:line 26
Can anybody help me understand why this error occurs please?
AWS lambda does not support outbound ICMP (the protocol that drives ping) traffic. In addition to this, the underlying infrastructure to create the ping messages isnt supported either.
From the AWS docs (https://aws.amazon.com/lambda/faqs/):
Q: What restrictions apply to AWS Lambda function code?
Lambda attempts to impose as few restrictions as possible on normal
language and operating system activities, but there are a few
activities that are disabled: Inbound network connections are blocked
by AWS Lambda, and for outbound connections only TCP/IP and UDP/IP
sockets are supported, and ptrace (debugging) system calls are
blocked. TCP port 25 traffic is also blocked as an anti-spam measure.
I run the following websocket client code on windows and everything works fine - like expected. But if the code is published for linux-arm and copied to a RaspberryPi3 (runs under Raspian) it will end up in an AuthenticationException.
csproj file content:
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp2.0</TargetFramework>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Newtonsoft.Json" Version="10.0.3" />
<PackageReference Include="System.Net.WebSockets.Client" Version="4.3.1" />
</ItemGroup>
The connection attempt: (the point where the exception is thrown)
private readonly ClientWebSocket _socket;
public ApiConnection()
{
_socket = new ClientWebSocket();
}
public async Task Connect()
{
// the uri is like: wss://example.com/ws
await _socket.ConnectAsync(new Uri(_settings.WebSocketUrl), CancellationToken.None);
if (_socket.State == WebSocketState.Open)
Console.WriteLine("connected.");
}
Exception stack:
System.Net.WebSockets.WebSocketException (0x80004005): Unable to connect to the remote server ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Net.Security.SslState.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)
at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.PartialFrameCallback(AsyncProtocolRequest asyncRequest)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Net.Security.SslState.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result)
at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Net.WebSockets.WebSocketHandle.<ConnectAsyncCore>d__24.MoveNext()
at System.Net.WebSockets.WebSocketHandle.<ConnectAsyncCore>d__24.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Net.WebSockets.ClientWebSocket.<ConnectAsyncCore>d__16.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
The target websocket server is running behind a nginx proxy on Ubuntu. I think the problem relies on the client because if the code is executed on windows everything works fine.
I tried also importing the CA certifacte into Raspians "certificate store". With no luck.
UPDATE:
A http connection (ws://) works also on linux. It seems, the WebSocketClient didn't trust my LetsEncrypt cert?
This happens when the browser / client does not trust the SSL cert that the server is throwing at it.
To test, load the same url / a url on the same site in your browser, you should get a warning.
When the cert issue is resolved the warning will go away.
The exact process for resolving SSL cert issues is dependent on a lot of things like ...
OS, Web Server, Cert Authority, Cert Providers Portal so it's near impossible for anyone on here to give you specifics about fixing cert issues, but that said ...
There is however a bit of generic advice on this here on the SE network ...
https://unix.stackexchange.com/questions/90450/adding-a-self-signed-certificate-to-the-trusted-list
https://unix.stackexchange.com/questions/17748/trust-a-self-signed-pem-certificate
In your case, as rasbian is based on debian, some standard debian advice might help ...
In Debian, the certificates stash is located in /etc/ssl/certs/. This directory contains by default a series of symlinks that points to the certificates installed by the ca-certificates package (including the needed symlinks generated by c_rehash(1)) and a ca-certificates.crt which is a concatenation of all these certificates. Everything managed by the update-ca-certificates(8) command which is taking care of updating the symlinks and the ca-certificates.crt file.
Adding a new (CA) certificate to the stash is quite easy as update-ca-certificates(8) is also looking for files in /usr/local/share/ca-certificates/, the administrator just has to place the new certificate in the PEM format in this directory (with the .crt extension) and run update-ca-certificates(8) as root. All the applications on the system (wget, …) should now trust it.
The other possible solution might be "I trust my code to not request bad url's so i'll ignore SSL cert errors" which you could do with something like this ...
C# Ignore certificate errors?
... but that's not ideal, at least it gives you a work around until you can resolve the issue, worst case you could still check but by coding your own check instead of just a blanket return true.
final point:
I often find no matter what the OS, doing something as simple as a reboot or two in between tests / checks can clear something out that you wouldn't normally consider to be an issue.
The certificates that are validated on windows, won't necessarily validate on Linux.
Each of the operating systems are using different certificates and different methods to validate them, furthermore there are certificates known to Linux which are not supported by windows.
There can be a situation, where your LetsEncrypt cert is recognized by windows but Linux did not recognize this and thus, threw and exception of AuthenticationException stating clearly
"The remote certificate is invalid according to the validation procedure"
Meaning the Linux tried to validate the certificate, but failed, as it was not recognized to the Linux at all but your windows recognized it and acted as expected.
I don't know much as to which certificates will work on which Linux, but I would recommend to research this thing in order to find a way to use a certificate, that both the windows and Linux can recognize, validate and work with.
I tried something similar recently (although I used Mono instead of .Net Core), and in my case it was simply that the system time on the Raspberry was off by a couple of days(!), thereby running outside the certificate's "Valid From" to "Valid To" timestamps. This can happen if the Raspi has no internet connection to synch it's time via NTP. Raspberries do not contain a hardware clock with a buffer battery, so they lose track of time when not powered on.
The first step would be to log onto the Pi and run date, to see if the system clock is correct.
IF this is your problem, you have several possible fixes:
Enable internet access to the Raspberry Pi, so it can synch its clock via NTP
Set the correct time yourself, then keep the Raspi powered on at all times, making sure to manually set the time whenever you power it on
Install a hardware clock ($3 - $5, about ten minutes worth of work if you follow instructions), and be done with the problem (as long as the battery lasts)
Another idea could be to check where .Net Core expects the CA-certs to be installed. At least using Mono this differs from the Linux defaults. I used the X509Store C# API to install the certificate instead of the (Debian-)Linux system tools.
I am using version 2.1.2 of PushSharp. The app is .NET 4.5.1 (although I have also tried targeting .NET 4.5 and .NET 4)
I am trying but not succeeding to send push messages through the sandbox Apple APNS.
I am successfully sending messages using a PHP script provided here on Ray Wenderlich's walkthrough using the same certificate and sending to the same device ID as for my PushSharp app.
I have tested by exporting the completed cert as p12 from the key chain. Exporting the completed cert and key. Exporting the private key. Also by the method used here. When I combine the cert and key for use in the PHP script, I have no problems.
I have imported the p12 certificate onto the machines I have tested from - seems to make no difference.
I have tried changing the IsProduction flag when registering the apple push service to the push broker. There is no error when it is set as production (even though this is a sandbox cert) however it obviously doesn't get through to the device in that case either.
None of my messages will go through, all get a service exception which looks like the following:
System.Security.Authentication.AuthenticationException: A call to SSPI failed, see inner exception. ---> System.ComponentModel.Win32Exception: The message received was unexpected or badly formatted
--- End of inner exception stack trace ---
at System.Net.Security.SslState.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, Exception exception)
at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult)
at PushSharp.Apple.FeedbackService.Run(ApplePushChannelSettings settings, CancellationToken cancelToken)
at PushSharp.Apple.ApplePushService.<>c__DisplayClass4.<.ctor>b__1(Object state)
This is basically what my code looks like:
var push = new PushBroker();
// register event handlers for channel create/destroy/exception, notificationrequeue, serviceexception, notification sent
var appleCert = File.ReadAllBytes(ConfigurationManager.AppSettings["CertAddress"]);
push.RegisterAppleService(new ApplePushChannelSettings(false, appleCert, ConfigurationManager.AppSettings["CertPassword"]));
var pn = new AppleNotification().ForDeviceToken(item.SendToDeviceIdentifier).WithAlert(item.AlertMessage).WithBadge(item.Badges);
push.QueueNotification(pn);
I get the channel up event called, and then the service exception.
Some of the related questions mention that this error can be related to a firewall issue - I have tested my app in 2 different networks which are able to send push notifications (1 of which is using a PushSharp app currently).
Any insight would be much appreciated.
We were having the same issue using the now deprecated APNS-Sharp library (ancestor to PushSharp). I submitted a pull request for APNS-Sharp that fixes the issue based on my tests.
The modification was to change (in ApplePushChannel.cs)
stream.AuthenticateAsClient(this.appleSettings.Host, this.certificates, System.Security.Authentication.SslProtocols.Ssl3, false);
to
stream.AuthenticateAsClient(this.appleSettings.Host, this.certificates, System.Security.Authentication.SslProtocols.Tls, false);
I didn't find a confirmation on this, but it looked like the SSL3 protocol was no longer supported by the Sandbox APNS. Like others that reported the issue, my notifications against the Production APNS were still working.
You can find the pull request here:
https://github.com/Redth/PushSharp/pull/369/files
Update
There is a thread on the Apple Developer web site on this topic:
https://devforums.apple.com/thread/224320?tstart=0
However, some of the people there are also on this thread or on the github thread. So the information is biased for sure. A contact that I have at Apple is saying:
While there is no official documentation out yet, it seems like APNS is moving towards TLS rather than SSL (solely based on seeing this change - I have not heard anything official).
If any of you are running this on Windows Server 2003 (I know, I know) you will have to run this patch or you will still get strange errors even after you implement the fix. I spent a few hours wondering why my 2008 server worked and my 2003 server didn't.
http://support.microsoft.com/kb/948963