TLS Support
TLS interception and unwrapping is not enabled by default, but can be done so with annotations to your workload.
When using the annotation examples below, be sure to add them to any existing annotations on your workload. It's common to delete existing annotations or add the annotations in the wrong place (like on the pod instead of the deployment) in this step.
How Speedscale reads TLS requests from Speedscale on Vimeo.
TLS Inbound Interception
The sidecar will be listening for incoming requests, and must present to the client the correct certificate. Because you already have TLS configured, the cert files you are using must be provided to the sidecar. There are three available settings:
tls-in-secret
(required) is the name of the Kubernetes secrettls-in-private
(optional) is the filename of the private key inside the secret (default:tls.key
)tls-in-public
(optional) is the filename of the public cert inside the secret (default:tls.crt
)
When your deployment is injected, the sidecar will have an extra environment variable TLS_IN_UNWRAP=true
,
TLS_IN_PUBLIC_KEY
, TLS_IN_PRIVATE_KEY
and a volume mount to access the files from the provided secret.
annotations:
sidecar.speedscale.com/inject: "true"
sidecar.speedscale.com/tls-out: "true"
sidecar.speedscale.com/tls-in-secret: "my-tls-secret"
sidecar.speedscale.com/tls-in-private: "tls.key"
sidecar.speedscale.com/tls-in-public: "tls.crt"
TLS Outbound Interception
To unwrap outbound TLS calls there are multiple steps required:
- Configure the sidecar to enable outbound TLS interception
- Configure your application to trust the new TLS Certificates
When your deployment is injected, the sidecar will have an extra environment variable TLS_OUT_UNWRAP=true
and a volume mount to access the files from the speedscale-certs
secret. The operator will automatically
create a secret named speedscale-certs
and put into the namespace. All that is required is to add this
annotation to your deployment:
annotations:
sidecar.speedscale.com/inject: "true"
sidecar.speedscale.com/tls-out: "true"
Mutual Authentication for Outbound Calls
If your backend system requires [Mutual Authentication](https://tools.ietf. Org/html/rfc8120) (aka Mutual TLS or 2-Way TLS), this requires configuring the sidecar with an additional X509 key pair. During the TLS handshake, the backend system will request a Client Certificate. This is the certificate that goproxy will present. There are three available settings:
tls-mutual-secret
(required) is the name of the Kubernetes secrettls-mutual-private
(optional) is the filename of the private key inside the secret (default:tls.key
)tls-mutual-public
(optional) is the filename of the public cert inside the secret (default:tls.crt
)
When your deployment is injected, the sidecar will have extra environment variables TLS_MUTUAL_PUBLIC_KEY
and TLS_MUTUAL_PRIVATE_KEY
and a volume mount to access the files from the provided secret. You must
provide a Kubernetes secret that has the TLS private key and public cert. The name of the secret and the names
of the files can be provided to operator
to inject automatically.
annotations:
sidecar.speedscale.com/inject: "true"
sidecar.speedscale.com/tls-out: "true"
sidecar.speedscale.com/tls-mutual-secret: "my-tls-secret"
sidecar.speedscale.com/tls-mutual-private: "tls.key"
sidecar.speedscale.com/tls-mutual-public: "tls.crt"
Trusting TLS Certificates
Now that TLS outbound calls are intercepted by goproxy, you must configure your application to trust the speedscale-certs from your cluster or goproxy will report errors and TLS traffic will not be decrypted.
Speedscale will attempt to use the Root CA Cert in the speedscale-certs
secret. Intercepted TLS calls will
have a new cert that is generated from this Root CA. So your application needs to trust this Root CA for TLS
calls to be handled automatically.
Configuring TLS trust is, in most cases, language specific. Every runtime language has different characteristics and if you do not configure this correctly you will get TLS/SSL errors. Always configure this in a controlled environment first.
- Go
- Node.js
- Ruby
- .NET
- Java
export SSL_CERT_FILE="${HOME}/.speedscale/certs/tls.crt"
export NODE_EXTRA_CA_CERTS="${HOME}/.speedscale/certs/tls.crt"
For Node.js applications newer than v7.3.0
.
export SSL_CERT_FILE="${HOME}/.speedscale/certs/tls.crt"
export SSL_CERT_FILE="${HOME}/.speedscale/certs/tls.crt"
.NET Core uses OpenSSL on Linux and Mac which respects default settings. The default Microsoft .NET Docker base images are Linux based which means these settings apply, however running Windows based workloads may require additional configuration.
Java applications utilize a truststore to specify certificates to be trusted.
On desktop:
Create the keystore with Speedscale certs.
proxymock certs --jks
Then apply these flags when running your app:
java \
-Djavax.net.ssl.trustStore=/etc/ssl/speedscale/jks/cacerts.jks \
-Djavax.net.ssl.trustStorePassword=changeit \
-jar app.jar
In cluster:
During Operator installation a secret called speedscale-jks
will be created that contains the speedscale-certs
root CA
along with a standard set of CA certs used by openjdk
. This secret is automatically mounted when the
tls-out
setting is configured as shown below. The Java app itself needs to be configured to use this secret
as well which requires configuring your JVM to use the truststore with these settings:
These can be automatically applied by adding to your JVM by setting JAVA_TOOL_OPTIONS
. This can be set
on your workload by adding the sidecar.speedscale.com/tls-java-tool-options: "true"
annotation. Read more
about this setting here.
When running in-cluster, these flags are also surfaced as an environment variable SPEEDSCALE_JAVA_OPTS
if you need to merge with your own existing sets of Java flags.
apiVersion: apps/v1
kind: Deployment
metadata:
name: spring-boot-app
annotations:
sidecar.speedscale.com/inject: "true"
sidecar.speedscale.com/tls-out: "true"
sidecar.speedscale.com/tls-java-tool-options: "true"
How Does It Work?
The following explanations are provided for engineers seeking a deeper understanding of how TLS unwrapping takes place. Tools like envoy follow an almost identical procedure with their sidecars and this is not Speedscale-specific knowledge. Also keep in mind that Speedscale supports alternative ingest mechanisms including HTTP file and Postman collection import. See the integrations section for more information.
Inbound
Your application is already presenting a public TLS certificate to clients, and the clients must trust this certificate. That means they are either “real” certificates signed by a public CA (certificate authority) or they are otherwise managed certificates using a tool like cert-manger in Kubernetes. Since the certificates are already present in the cluster and your app presents them to clients as trustworthy, Speedscale simply needs to know where the certificates are located so it can decode a copy of the traffic in the same way the service does. Conceptually, it looks like this:
In this mode, the sidecar is acting as a passthrough. It copies the traffic to the Speedscale forwarder, but does not modify the traffic itself.
Outbound
- The application makes it’s call as normal for instance to
https://example.com:443
- The iptables rule modifies the intended destination and changes the port to
4140
where the sidecar is listening, and the sidecar initiates the outbound call transparently to the application (no code change required). - The sidecar checks the application client request to determine if the TLS handshake is starting and finds the intended destination.
- The sidecar generates a TLS certificate for the SNI hostname (
example.com
in our case) that will expire in 1 hour and signs with the Cluster Root CA that was created as part of the operator installation. The certificate is stored in the sidecar local cache which it will use for the next hour. - As long as the application is properly configured to trust the Cluster Root CA, the certificate will match both the domain being accessed as well as be a trusted certificate.
- After this successful handshake, the sidecar has access to all of the details of the request and the response. Note that because this is at the transport layer it works for non-HTTP protocols like databases and queue systems as well.
In this mode, the sidecar is actively re-signing traffic so that your service is interacting with the Speedscale sidecar and the sidecar is then interacting with the external service. This is similar to tools like mitmproxy.
Sidecar proxies are slowly being replaced by eBPF-based collections but platform support is not wide enough for enterprise full adoption yet. If you're interesting in an eBPF-based collector please reach out to support.