Skip to main content

Proxy Modes

The default proxy mode of the sidecar in a Kubernetes environment is known as a transparent proxy, meaning it is transparent to the workload and requires no additional proxy configuration. This is done with iptables routing for both inbound and outbound traffic. While this is the preferred traffic capture mode, there are instances when this approach cannot be used. This might include workloads utilizing Kubernetes host networking, or workloads that perform their own iptables modifications.

In these instances, the Speedscale sidecar must be run as an inline proxy that explicitly recieves requests to proxy rather than transparently intercepting them. There are two halves to this inline proxy operation: a reverse proxy for handling inbound traffic, and a forward proxy for handling outbound traffic.

warning

forward and dual modes do not automatically reconfigure your application. Sidecar injection can succeed while outbound capture stays empty if the workload runtime does not send traffic to the forward proxy on 127.0.0.1:4140 or your configured proxy-out-port.

Remember

When using the annotation examples below, be sure to add them to any existing annotations on your workload.

Sidecar capture is deprecated

For new Kubernetes installations, use the eBPF collector. The proxy modes below are retained for existing sidecar deployments and workloads that do not meet the eBPF requirements.

Proxy Configuration​

Configuration for this operation mode is managed with workload annotations. You must specify the type of proxy the sidecar should operate as: reverse, forward, or dual (both reverse and forward):

  • sidecar.speedscale.com/proxy-type: "reverse"
  • sidecar.speedscale.com/proxy-type: "forward"
  • sidecar.speedscale.com/proxy-type: "dual"

Sidecar reverse and forward proxies support the following proxy protocols: http, socks4, and socks5. Authentication is not required.

Operator And Runtime Responsibilities​

Switching away from transparent mode introduces two separate pieces of configuration:

  • reverse: the operator configures the sidecar to accept inbound traffic and forward it to your app.
  • forward: your workload runtime must explicitly use the sidecar as its outbound proxy.
  • dual: both of the above apply. Inbound traffic reaches the sidecar, and outbound traffic must still be configured in the application runtime.

Reverse proxies can be configured to require no specific proxy protocol, but instead operate as a simple TCP proxy. When operating in this mode, the sidecar treats incoming requests are treated as opaque TCP connections but must also be aware of their destination. This is managed with the following annotations:

  • sidecar.speedscale.com/proxy-host: "localhost"
  • sidecar.speedscale.com/proxy-port: "80"

These values default to localhost and 80 if the annotations are omitted from the workload.

Forward proxies must always utilize a proxy protocol and do not support simple TCP passthrough since they could support any number of destinations.

The underlying protocol either the reverse, forward, or both proxies use is configured with the following annotation:

  • sidecar.speedscale.com/proxy-protocol: "socks"

This annotation supports the following values:

  • tcp: applies only to reverse (inbound) proxies
  • http: operate all proxies (reverse, forward, or dual) as an http proxy
  • socks: operate all proxies (reverse, forward, or dual) as a socks4/socks5 compatible proxy
  • tcp:http: dual mode only, operate a tcp reverse proxy and an http forward proxy
  • tcp:socks: dual mode only, operate a tcp reverse proxy and a socks4/socks5 forward proxy

Additional Customization​

By default, the Speedscale sidecar listens on port 4143 for all inbound reverse proxy requests. This value can be modified if desired with the annotation:

  • sidecar.speedscale.com/proxy-in-port: "10000"

For outbound forward proxy requests, the sidecar will listen on port 4140, which can also be configured with an annotation:

  • sidecar.speedscale.com/proxy-out-port: "11000"

Runtime Configuration For Outbound Traffic​

For forward and dual modes, the workload must be configured to use the sidecar's forward proxy.

Common patterns:

  • HTTP_PROXY=http://127.0.0.1:4140
  • HTTPS_PROXY=http://127.0.0.1:4140
  • JAVA_TOOL_OPTIONS=-Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=4140 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=4140

If you changed sidecar.speedscale.com/proxy-out-port, use that port in the runtime configuration too.

Language-specific notes:

Examples​

Run a reverse (tcp) and forward (http) proxy and send any inbound requests to local port 8080:

annotations:
  sidecar.speedscale.com/inject: "true"
  sidecar.speedscale.com/proxy-type: "dual"
  sidecar.speedscale.com/proxy-protocol: "tcp:http"
  sidecar.speedscale.com/proxy-port: "8080"

Run an outbound forward (socks) proxy only:

annotations:
  sidecar.speedscale.com/inject: "true"
  sidecar.speedscale.com/proxy-type: "forward"
  sidecar.speedscale.com/proxy-protocol: "socks"

Run an inbound reverse (http) proxy only and send requests to local port 8888:

annotations:
  sidecar.speedscale.com/inject: "true"
  sidecar.speedscale.com/proxy-type: "reverse"
  sidecar.speedscale.com/proxy-protocol: "http"
  sidecar.speedscale.com/proxy-port: "8888"

Run both a reverse (tcp) and forward (http) proxy, but change the sidecar ports to 10000 for inbound requests and 10001 for outbound requests:

annotations:
  sidecar.speedscale.com/inject: "true"
  sidecar.speedscale.com/proxy-type: "dual"
  sidecar.speedscale.com/proxy-protocol: "tcp:http"
  sidecar.speedscale.com/proxy-port: "8080"
  sidecar.speedscale.com/proxy-in-port: "10000"
  sidecar.speedscale.com/proxy-out-port: "10001"

After applying the workload annotations above, the application still needs matching runtime settings. For example, if the forward proxy remains on 4140, many workloads will need either:

export HTTP_PROXY=http://127.0.0.1:4140
export HTTPS_PROXY=http://127.0.0.1:4140

or for Java:

export JAVA_TOOL_OPTIONS="-Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=4140 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=4140"

Kerberos authentication for Speedscale Cloud​

When your outbound proxy responds with 407 Proxy-Authenticate: Negotiate, enable Kerberos authentication for Speedscale cloud connections. The client authenticates to the proxy before establishing TLS to the cloud. The Speedscale API key still authorizes the tenant inside that TLS connection.

This configuration covers external REST calls, including API-key validation, and cloud gRPC connections used by forwarder and inspector. It also applies to cloud calls from speedctl and proxymock. Application traffic captured through a local proxy uses the separate application proxy configuration.

Requirements​

Use released speedctl, proxymock, and operator-managed forwarder/inspector images at version v2.5.1145 or later. Upgrade the CLI and operator before enabling this setting.

Ask your network or identity administrator for:

  • An HTTP or HTTPS CONNECT proxy URL that offers Negotiate authentication.
  • A realm configuration in krb5.conf and either a FILE credential cache or a keytab with its client principal.
  • The proxy's service principal name (SPN), normally HTTP/<proxy hostname>.
  • Direct access to the realm's Key Distribution Centers (KDCs) and any DNS discovery they require.
  • A proxy policy that permits the cloud destination, HTTP/2 and long-lived gRPC tunnels.

KDC and Kerberos DNS requests bypass the HTTP proxy. Configure an explicit KDC in krb5.conf when DNS discovery is unavailable. Keep the client and KDC clocks synchronized.

Configuration reference​

Set these variables in the process making the cloud connection. For the classic Kubernetes operator install, use the override ConfigMap and credential mount below.

VariableBehavior
HTTPS_PROXYHTTP or HTTPS proxy URL for TLS cloud destinations, such as http://proxy.example.com:8080.
NO_PROXYHosts and domains that should connect directly. Keep local and in-cluster destinations in this list.
SPEEDSCALE_KERBEROS_PROXYSet to true to enable Kerberos CONNECT authentication. Unset or false keeps the existing proxy behavior.
KRB5_CONFIGPath to a readable krb5.conf. Defaults to /etc/krb5.conf.
KRB5CCNAMEFILE credential cache, such as FILE:/path/to/cache. Takes precedence over a keytab when both are set.
KRB5_CLIENT_KTNAMEFILE keytab, such as FILE:/path/to/client.keytab. Used when KRB5CCNAME is unset.
SPEEDSCALE_KERBEROS_PRINCIPALClient principal in user@REALM form. Required for keytab mode.
SPEEDSCALE_KERBEROS_PROXY_SPNOverride the default HTTP/<proxy hostname> SPN, for example when a load balancer uses a different registered name.
SPEEDSCALE_KERBEROS_DISABLE_FASTSet to true to disable Kerberos FAST preauthentication when required by the realm administrator. Defaults to false.
SPEEDSCALE_KERBEROS_SECRETKubernetes Secret name that the operator mounts on forwarder and inspector at /etc/speedscale/kerberos. The operator itself needs the mount patch below.

Credentials in HTTPS_PROXY, such as http://user:password@proxy.example.com:8080, explicitly select Basic authentication. A rejected Basic attempt reports Basic and does not fall back to Kerberos. Remove the URL credentials to negotiate Kerberos.

Local CLI setup​

Set the proxy and realm configuration before running speedctl or proxymock cloud commands. Replace the example paths and realm with values supplied by your administrator.

export HTTPS_PROXY=http://proxy.example.com:8080
export NO_PROXY=localhost,127.0.0.1,.svc,.cluster.local
export SPEEDSCALE_KERBEROS_PROXY=true
export KRB5_CONFIG=/path/to/krb5.conf

Option A: FILE credential cache​

Create a FILE cache with kinit:

unset KRB5_CLIENT_KTNAME
export KRB5CCNAME=FILE:/path/to/cache
kinit -c "$KRB5CCNAME" user@EXAMPLE.COM

The cache must contain a valid ticket-granting ticket (TGT), even when a service ticket is already cached. Renew or replace the cache before expiry using your external credential provider. The client does not manage cache renewal.

Option B: Keytab​

For noninteractive login, unset the cache variable and supply a keytab and principal:

unset KRB5CCNAME
export KRB5_CLIENT_KTNAME=FILE:/path/to/client.keytab
export SPEEDSCALE_KERBEROS_PRINCIPAL=user@EXAMPLE.COM

The client obtains a fresh TGT from the keytab when opening an authenticated tunnel. An existing KRB5CCNAME value selects cache mode even when the keytab settings are present.

File paths and rotation​

Bare file paths and FILE: prefixes are supported. Windows drive-letter paths work with either form, for example FILE:C:\Speedscale\cache or C:\Speedscale\client.keytab. Native macOS API caches and Linux KEYRING/KCM caches are unsupported; create a FILE cache explicitly.

The process user must be able to read the configuration and credential files. Each new authenticated tunnel reloads them. Existing REST keepalive connections and gRPC streams continue through their established tunnels after a file update.

Kubernetes operator setup​

This example uses the classic operator install in the speedscale namespace. Adjust the namespace to match your installation. The current operator Helm chart has no Kerberos credential-mount value; keep the operator patch in your Helm post-renderer or GitOps configuration so upgrades retain it.

1. Provision the credential Secret​

Create a Secret containing krb5.conf and the keytab:

kubectl -n speedscale create secret generic speedscale-kerberos \
  --from-file=krb5.conf=/path/to/krb5.conf \
  --from-file=client.keytab=/path/to/client.keytab

Use your existing Secret provisioning system for ongoing credential management. Restrict access to the administrators and workloads that need these credentials. Keep credential contents in the Secret; the ConfigMap below contains file paths and settings.

2. Configure the operator​

Add these entries to speedscale-operator-override in the same namespace. Preserve any other override entries you already use. If the ConfigMap does not exist, create it with this manifest:

kerberos-override.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: speedscale-operator-override
  namespace: speedscale
data:
  HTTPS_PROXY: http://proxy.example.com:8080
  NO_PROXY: localhost,127.0.0.1,.svc,.cluster.local
  SPEEDSCALE_KERBEROS_PROXY: "true"
  SPEEDSCALE_KERBEROS_SECRET: speedscale-kerberos
  KRB5_CONFIG: /etc/speedscale/kerberos/krb5.conf
  KRB5_CLIENT_KTNAME: FILE:/etc/speedscale/kerberos/client.keytab
  SPEEDSCALE_KERBEROS_PRINCIPAL: user@EXAMPLE.COM
kubectl apply -f kerberos-override.yaml

For cache mode, provision a Secret file named cache, remove KRB5_CLIENT_KTNAME and SPEEDSCALE_KERBEROS_PRINCIPAL from the override, and set KRB5CCNAME: FILE:/etc/speedscale/kerberos/cache. An external process must refresh the cache and update the Secret before expiry.

3. Mount the Secret on the operator​

Apply this strategic merge patch to the operator Deployment through your deployment tooling:

kerberos-operator-patch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: speedscale-operator
  namespace: speedscale
spec:
  template:
    spec:
      containers:
        - name: operator
          volumeMounts:
            - name: kerberos-proxy
              mountPath: /etc/speedscale/kerberos
              readOnly: true
      volumes:
        - name: kerberos-proxy
          secret:
            secretName: speedscale-kerberos

For an existing installation, apply the same patch and restart the operator after updating the ConfigMap:

kubectl -n speedscale patch deployment speedscale-operator \
  --type=strategic --patch-file=kerberos-operator-patch.yaml
kubectl -n speedscale rollout restart deployment/speedscale-operator
kubectl -n speedscale rollout status deployment/speedscale-operator

The operator propagates its Kerberos environment settings and read-only Secret mount to the forwarder and inspector Deployments it reconciles. Captured application sidecars do not receive these credentials. The files must be readable by each workload's configured user.

Mount the entire Secret directory to receive credential rotations. Kubernetes updates mounted Secret files after a delay; subPath mounts do not receive those updates. Allow for that delay when refreshing a cache before expiry. Environment-setting changes require an operator restart, while updated mounted files take effect on new authenticated tunnels.

Namespaced chart setup​

The namespaced installation runs the forwarder, inspector and replay coordinator in the release namespace. Its Kerberos settings mount a pre-existing Secret read-only on those three components. They do not grant Secret read permissions to an application sidecar or replay runtime Pod.

Chart versions through 2.5.1149 do not contain the namespaced Kerberos values below. Before using this procedure, run helm show values speedscale/speedscale-operator and confirm the published chart has a namespaced.kerberos section.

Create the credential Secret in the release namespace:

kubectl -n banking-app create secret generic speedscale-kerberos \
  --from-file=krb5.conf=/path/to/krb5.conf \
  --from-file=client.keytab=/path/to/client.keytab

Add these values to the same file used for the namespaced install:

values.yaml
https_proxy: http://proxy.example.com:8080
no_proxy: localhost,127.0.0.1,.svc,.cluster.local
namespaced:
  enabled: true
  kerberos:
    enabled: true
    secretName: speedscale-kerberos
    credentialSource: keytab
    principal: user@EXAMPLE.COM
    # Set proxySPN only when the registered SPN differs from HTTP/<proxy hostname>.
    # proxySPN: HTTP/proxy.example.com

For a FILE cache, create the Secret with --from-file=cache=/path/to/cache, set credentialSource: cache, and omit principal. Refresh the cache and update the Secret before its TGT expires. The chart requires https_proxy, the Secret name, and a principal for keytab mode. It mounts only krb5.conf and the selected credential file, so either missing Secret key prevents the Pods from starting.

The installation guide covers the namespace permissions and the public chart. Keep https_proxy and no_proxy at the chart root, and permit direct access to the realm's KDC from these three Pods in addition to the authenticated proxy connection. The chart's optional NetworkPolicy does not add egress for the proxy port or KDC; supply that access in your CNI policy before enabling the chart policy.

Validate the connection​

Test the actual proxy and realm in your dev environment before rollout. A local Kerberos fixture or isolated MIT KDC test does not establish compatibility with a customer's proxy or identity policy.

  • Confirm a REST API-key check succeeds through the proxy with the normal Speedscale API key.
  • Confirm the inspector can establish and keep its gRPC control stream open through the proxy.
  • Check proxy-side identity and access policy, direct connections covered by NO_PROXY, and a reconnect after credential rotation.

The supported exchange is an initial 407 Negotiate challenge followed by one Kerberos SPNEGO client token on the next CONNECT. Additional negotiation rounds, NTLM and Kerberos authentication at the cloud origin are unsupported. Plain HTTP requests keep their existing transport behavior.

This setting covers the Speedscale cloud connection path. It does not add Kerberos authentication to OTEL exporters or S3-compatible storage connections.

Troubleshooting​

SymptomCheck
Proxy Authentication Required with no Kerberos attemptCheck the client/image version and set SPEEDSCALE_KERBEROS_PROXY=true in the process making the request.
cannot load KRB5_CONFIGCheck the configuration path, mount and process-user read permissions.
cannot load KRB5CCNAME or only FILE credential storage is supportedSelect a readable FILE cache and verify its format and path.
credential cache has no valid TGTRefresh the cache with kinit or your external provider before retrying.
cannot load KRB5_CLIENT_KTNAMECheck the keytab path, Secret key and read permissions; unset KRB5CCNAME to select keytab mode.
cannot acquire Kerberos credentialsCheck ticket expiry, principal/realm, clock synchronization and direct KDC access.
cannot obtain Kerberos service ticket or authenticated CONNECT returns 407Confirm the registered proxy SPN, client authorization and supported challenge sequence with the proxy administrator.
Basic authentication (HTTP 407)Correct the proxy URL credentials, or remove them if Kerberos is required.
REST succeeds but gRPC disconnectsVerify HTTP/2 and long-lived CONNECT tunnels are permitted by the proxy policy.

Share sanitized error text, component versions and the challenge scheme when asking for help. Keep keytabs, credential-cache contents, API keys and Proxy-Authorization tokens out of logs and support messages.

Configuring your application proxy server​

Every language has it's own nuances for how it works with a forward proxy server for outbound traffic. Select your language to see well-known patterns for that language.

Golang supports HTTP_PROXY and HTTPS_PROXY environment variables to configure outbound http or https requests.

spec:
  template:
    spec:
      containers:
      - env:
        - name: HTTP_PROXY
          value: http://127.0.0.1:4140
        - name: HTTPS_PROXY
          value: http://127.0.0.1:4140