Istio
Please make sure the Speedscale Operator is installed before configuring Istio support.
Istio is a service mesh offering that modifies a cluster to provide, among other things, traffic and network management.
External Networking Requirements
Speedscale pods in the the speedscale
namespace, as well as the generator
and responder
pods that are
deployed during traffic replays require external internet access. If your istio installation is configured
with an outbound traffic policy of REGISTRY_ONLY
rather than ALLOW_ANY
, meaning that only whitelisted
services can be accessed from within the cluster (see
OutboundTrafficPolicy),
you will to configure ServiceEntry
resources that allow the following MESH_EXTERNAL
access:
Host | Protocol | Direction |
---|---|---|
app.speedscale.com | HTTPS | Outbound |
firehose.us-east-1.amazonaws.com | HTTPS | Outbound |
sqs.us-east-1.amazonaws.com | HTTPS | Outbound |
sns.us-east-1.amazonaws.com | HTTPS | Outbound |
s3.us-east-1.amazonaws.com | HTTPS | Outbound |
*.s3.us-east-1.amazonaws.com | HTTPS | Outbound |
sts.amazonaws.com | HTTPS | Outbound |
sts.us-east-1.amazonaws.com | HTTPS | Outbound |
monitoring.us-east-1.amazonaws.com | HTTPS | Outbound |
gcr.io | HTTPS | Outbound |
These hosts are subject to change and security via TLS is recommended as opposed to IP whitelisting. If you require a list of IPs, they can be programmatically accessed as shown here for AWS and here for GCR.
For example:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: speedscale-external-svc-https
namespace: speedscale
spec:
hosts:
- app.speedscale.com
- firehose.us-east-1.amazonaws.com
- sqs.us-east-1.amazonaws.com
- sns.us-east-1.amazonaws.com
- s3.us-east-1.amazonaws.com
- "*.s3.us-east-1.amazonaws.com"
- sts.amazonaws.com
- sts.us-east-1.amazonaws.com
location: MESH_EXTERNAL
ports:
- name: https
number: 443
protocol: TLS
resolution: NONE
Speedscale Sidecar Configuration
Istio makes use of a proxy known as Envoy, which it adds as a sidecar to
workloads that reside within the mesh. Both the Istio and Speedscale sidecars act as transparent proxies: each
must modify iptables
routing rules in order to intercept both ingress and egress traffic. Unfortunately,
they cannot coexist when operating in this mode since both are attempting to intercept and manage workload
traffic.
Istio is supported by Speedscale despite this conflict, but requires a few extra steps.
Within an Istio mesh, the Speedscale sidecar must operate as a non-transparent proxy; a reverse proxy for inbound traffic and a forward proxy for outbound traffic. This requires two things:
- Envoy must be configured to send ingress traffic to the Speedscale reverse proxy, which is done automatically by the Speedscale Operator via an Istio Sidecar resource.
- Your application must be configured to use an outbound proxy
Add Workload Annotations
Begin by adding the following annotations to your Kubernetes workload along with any other sidecar annotations:
sidecar.speedscale.com/inject: "true"
sidecar.speedscale.com/proxy-type: dual
sidecar.speedscale.com/proxy-protocol: tcp:http
Note: the proxy-protocol
annotation shown above will operate the outbound, forward proxy as an
HTTP proxy. If your application needs so use a SOCKS4 or SOCKS5 proxy, use tcp:socks
. See
proxy modes for more information.
Configure Outbound TLS Support
Outbound TLS support for the Speedscale sidecar can be enabled with the annotation
sidecar.speedscale.com/tls-out: "true"
. You may be required to perform additional steps if your
application and not Envoy is originating TLS requests. See
Trusting TLS Certificates for more information.
Configuring Your Application Proxy Server
Every language has it's own nuances for how it works with a forward proxy server for outbound traffic. Select your language to see well-known patterns for that language.
- Go
- Python
- Node.js
- Java
- .NET
- Ruby
Golang supports HTTP_PROXY
and HTTPS_PROXY
environment variables to configure outbound http or https
requests.
- Kubernetes
- Outside of Kubernetes
spec:
template:
spec:
containers:
- env:
- name: HTTP_PROXY
value: http://127.0.0.1:4140
- name: HTTPS_PROXY
value: http://127.0.0.1:4140
export HTTP_PROXY='socks5://127.0.0.1:4140'
export HTTPS_PROXY='socks5://127.0.0.1:4140'
Or alternatively as an http forward proxy:
export HTTP_PROXY='http://127.0.0.1:4140'
export HTTPS_PROXY='http://127.0.0.1:4140'
Python supports HTTP_PROXY
and HTTPS_PROXY
as well as the lowercase alternatives http_proxy
and
https_proxy
. This is true if you are using either the standard library urllib.request
module or the
popular requests
module. Socks proxies may require additional dependencies. For example, with the requests
module:
https://requests.readthedocs.io/en/latest/user/advanced/?highlight=proxy#socks
- Kubernetes
- Outside of Kubernetes
spec:
template:
spec:
containers:
- env:
- name: HTTP_PROXY
value: http://127.0.0.1:4140
- name: HTTPS_PROXY
value: http://127.0.0.1:4140
These two options can also support specifying proxies directly with a dictionary:
urllib.request.urlopen(
'http://example.com',
proxies={
'http': 'http://localhost:4140',
'https': 'http://localhost:4140',
},
)
When using Node, you need to set proxy-protocol
to http
or tcp:http
. In addition, the NodeJS app needs
to be configured with global-agent:
npm install --save global-agent
Then add global-agent to your code:
import 'global-agent/bootstrap';
Set these environment variables in the NodeJS runtime environment to configure the global-agent proxy:
- Kubernetes
- Outside of Kubernetes
spec:
template:
spec:
containers:
- env:
- name: GLOBAL_AGENT_HTTP_PROXY
value: http://127.0.0.1:4140
- name: GLOBAL_AGENT_HTTPS_PROXY
value: http://127.0.0.1:4140
- name: GLOBAL_AGENT_NO_PROXY
value: "*127.0.0.1:12557"
export GLOBAL_AGENT_HTTP_PROXY='http://127.0.0.1:4140'
export GLOBAL_AGENT_HTTPS_PROXY='http://127.0.0.1:4140'
export GLOBAL_AGENT_NO_PROXY='*127.0.0.1:12557'
export NODE_EXTRA_CA_CERTS=${HOME}/.speedscale/certs/tls.crt
When using Java, you need to set proxy-protocol
to socks
or tcp:socks
. Java has built-in system
properties for configuring the socks proxy server, add the following -D
system property flags:
- Kubernetes
- Outside of Kubernetes
spec:
template:
spec:
containers:
- env:
- name: JAVA_OPTS
value: >-
-Dhttp.proxyHost=127.0.0.1
-Dhttp.proxyPort=4140
-Dhttps.proxyHost=127.0.0.1
-Dhttps.proxyPort=4140
-Djavax.net.ssl.trustStore=/etc/ssl/speedscale/jks/cacerts.jks
-Djavax.net.ssl.trustStorePassword=changeit
-DsocksProxyHost=127.0.0.1
-DsocksProxyPort=4140
.NET supports HTTP_PROXY
and HTTPS_PROXY
environment variables to configure outbound http or https
requests.
- Kubernetes
- Outside of Kubernetes
spec:
template:
spec:
containers:
- env:
- name: HTTP_PROXY
value: http://127.0.0.1:4140
- name: HTTPS_PROXY
value: http://127.0.0.1:4140
export HTTP_PROXY='socks5://127.0.0.1:4140'
export HTTPS_PROXY='socks5://127.0.0.1:4140'
Or alternatively as an http forward proxy:
export HTTP_PROXY='http://127.0.0.1:4140'
export HTTPS_PROXY='http://127.0.0.1:4140'
Ruby supports HTTP_PROXY
and HTTPS_PROXY
environment variables to configure outbound http or https
requests.
- Kubernetes
- Outside of Kubernetes
spec:
template:
spec:
containers:
- env:
- name: HTTP_PROXY
value: http://127.0.0.1:4140
- name: HTTPS_PROXY
value: http://127.0.0.1:4140
export HTTP_PROXY='http://127.0.0.1:4140'
export HTTPS_PROXY='http://127.0.0.1:4140'
Allow Egress Speedscale Traffic (Optional)
If your Istio installation and sidecar control which subset of egress traffic is allowable, you may
need to add the speedscale
namespace to the sidecar's egress configuration. This step is not
necessary if you do not have custom sidecar configuration. Here is an example from the Istio
docs:
apiVersion: networking.istio.io/v1beta1
kind: Sidecar
metadata:
name: default
namespace: prod-us1
spec:
egress:
- hosts:
- "speedscale/*"
Ensure VirtualService Contains Host (Optional)
If your service is accessible both outside and inside the cluster, make sure the Istio
VirtualService
contains the same host. See the Istio
documentation for more
information.
Get In Touch
Istio allows for numerous different networking configuration options that can become difficult to navigate. Please be sure to consult the Istio documentation or reach out to Speedscale directly for more information or assistance.