Tip: OpenSSL SSL_connect: SSL_ERROR_SYSCALL

 Symptoms:

We use curl -v https://<domain> to test if the network traffic is allowed
The expected result would be like 

*  Trying 12.12.12.12:443...

* TCP_NODELAY set

* Connected to ***  port 443 (#0)

* ALPN, offering h2

* ALPN, offering http/1.1

* successfully set certificate verify locations:

*   CAfile: /etc/ssl/certs/ca-certificates.crt

  CApath: /etc/ssl/certs

* TLSv1.3 (OUT), TLS handshake, Client hello (1):

* TLSv1.3 (IN), TLS handshake, Server hello (2):

* TLSv1.2 (IN), TLS handshake, Certificate (11):

* TLSv1.2 (OUT), TLS alert, unknown CA (560):

However, we see this error :

*  Trying 12.12.12.12:443...

* TCP_NODELAY set

* Connected to *** port 443 (#0)

* ALPN, offering h2

* ALPN, offering http/1.1

* successfully set certificate verify locations:

*   CAfile: /etc/ssl/certs/ca-certificates.crt

  CApath: /etc/ssl/certs

* TLSv1.3 (OUT), TLS handshake, Client hello (1):

* OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to *** :443

Solution:

From the output, we see 443 is open but the TLS handshake, Server hello is missing. We have mid-tiers to handle TLS certificates. So it is very likely that the network is interrupted between  LB and mid-tiers where TLS is being handled.  It would be a good approach to double-check firewall ports between them. :)

Another reason is ingress controller pods were stuck may bounce or scale up to workaround it