This section controls the behavior of the Speedscale load generator.
Number of traffic copies - Determines how many replicas of the traffic the generator will start concurrently. This is similar to the vUser setting in a traditional testing tool but not exactly because a single traffic recording could have multiple users.
Request wait between calls:
- Zero wait time - calls will be sent as fast as possible with no artifical delay
- As recorded - calls will be made with the same delay as observed in the real traffic (note: long snapshots will take a long time to run)
- Fixed time between requests - each request will be sent the specified number of milliseconds after the last response is received
Replay traffic continuously - keeps the generator running the same traffic over and over for the specified timeframe
This section controls how the automatic mocking system (responders) will respond to requests sent from the service under test.
The Speedscale Responder can be configured to provide API request-level chaos responses. This is complimentary to traditional infrastructure chaos engineering because instead of breaking entire components, the application resiliency is tested for a single bad or slow response.
Three kinds of chaos are currently available:
- No Response - the responder will provide no response and allow the app to timeout waiting
- Bad status code (404, 500) - the responder will provide a 404 or 500 response instead of the correct status code
- High response time - the responder will respond correctly, but after a specified duration
Chaos, by definition, is random but we allow you to configure how frequently bad responses will be sent. Set the Active Chaos parameter to determine the probability of the app receiving a bad response. For example, if the Active Chaos is set to 100% all mock responses will experience some kind of chaos. If Active Chaos is set to 0%, no bad responses will be sent.
The responder rotates through each type of bad response that is selected. For instance, if Active Chaos is set to 50% and all three options are checked (No response, Bad status code and High response time) then the app will receive roughly 16.6% No Response, 16.6% Bad status code and 16.6% High response time (16.6+16.6+16.6 = 50%).
The UI provides a nice experience, but test configs are represented with simple JSON and accessible via
Let's look at some sample test config JSON and associated values.
Let's discuss what goes in each section:
A common identifier for the test config.
Chaos configuration for adding gremlins to your replay. These settings affect responder traffic from your "mocked" dependencies.
chaosPercent- The percentage of responses the chaos settings affect
badStatusCodes- Inject bad status codes like 404 and 500 into responses
randomLatency- Add random latency to the response
randomHighLatencyFactor- Scale the latency factor
randomHighlatencyMs- Milliseconds of latency to inject
Configuration to control how the generator process creates load with your recorded traffic
numReplicas- The number of "virtual users" to run, with each virtual user replaying a full copy of the snapshot traffic
replicaDelay- Amount of time, in milliseconds, to wait in between starting each virtual user
requestTiming- One of "NONE", "FLAT", "RECORDED", or "MULTIPLE" to set the delay pattern between virtual user requests
flatRequestDelay- Milliseconds to wait between virtual user requests when the requestTiming is "FLAT"
requestTimeoutSeconds- Maximum number of seconds to wait for a virtual user request to complete
runDuration- When set, the replay will cycle through snapshot traffic for this period of time
Configuration to control how the responder creates "mock" instances of your dependencies.
Configuration for various "asserters" which validate replay traffic against the original traffic. The various asserters are disussed in subsequent sections.
Rules or "goals" to determine whether a replay has passed or failed.
metricName- Name of the metric to validate
value- Threshold for the alert rule
type- One of "TOO_HIGH" or "TOO_LOW" determines how to evaluate the value, whith "TOO_HIGH" meaning the rule will trigger if the observed value is higher than the set value
action- Action taken on failure
location- An optional URI location limits the rule evaluation to specific requests