generation API load test mocks: measure orchestration without buying renders
A load test of your application should not accidentally become a large paid generation batch. Route test traffic to a local simulator that implements the response shapes your client expects. Then measure queueing, database writes, observer traffic, connection pools, and delivery scheduling independently from real model latency or quality.
Make the test destination unmistakable
Use a dedicated test configuration that permits only a loopback simulator origin and refuses a production API key. Remove production credentials from the load-test environment. A hostname typo should stop the test, not fall back to the public generation API.
export function mockLoadTarget(value) {
const url = new URL(value);
if (url.protocol !== 'http:' || url.hostname !== '127.0.0.1') {
throw new Error('Load tests require the local mock origin');
}
if (process.env.GENACE_API_KEY) {
throw new Error('Remove production credentials before this test');
}
return url.origin;
}This guard is one layer in a test harness. Also restrict external networking and verify the actual client uses the validated target. Do not place this test-only flexibility into a public production proxy accepting arbitrary base URLs.
Simulate lifecycle behavior, not only fast success
The simulator should return HTTP 202 with unique synthetic job_id values, then GET snapshots using id. Provide configurable queued and running periods, succeeded outputs, failed outcomes, canceled jobs, and null progress. Include HTTP 429 branches with distinct rate and concurrency error codes.
Control scenario selection through the test setup rather than random unrecorded behavior. A repeatable distribution of slow and failed jobs is more useful for comparing application versions than a simulator whose outcomes change unpredictably every run.
Measure the application bottleneck
Track dispatch admission, database operation latency, queue age, active collectors, connection acquisition delay, status request rate, and artifact scheduling. Count jobs and requests separately. Ten observers for one job create different pressure from ten new jobs with one observer each.
Test bursts and sustained traffic under the local allowances you intend to enforce. A mock cannot certify your actual Genace account quota or upstream capacity. Its role is to reveal whether your application responds sensibly when a configured limit is reached.
Include recovery load
Restart collectors during the run and verify they resume saved IDs without generating replacement submissions. Introduce delayed status responses to exercise connection pooling. Make some delivery tasks fail and confirm they retry existing artifacts rather than the creation route.
Inspect memory usage and temporary-file growth during slow downloads. An application that remains responsive under fast JSON responses may still fail when many bounded streams are active.
Report scope honestly
Record simulator version, scenario mix, concurrency, machine resources, and which production dependencies were replaced. Describe results as orchestration capacity under those conditions, not real rendering throughput or model availability.
Before finishing, verify the simulator recorded all requests, external request counters stayed zero, and no production credentials entered logs. Preserve safe metrics and configuration so the next run can reproduce the experiment.
See connection pooling, queue visibility, and fault injection.
Continue with production operations
Where these facts come from
- Editorial keyword hypothesis; search volume not measured
- codebase: src/ai/jobs/http.ts
- codebase: src/ai/api/concurrency.ts
- codebase: src/ai/api/rate-limit.ts