Why Mock Mode Matters for LLM Development

If you've ever accidentally left your development environment pointing at production API keys, you know the pain. That innocent test script suddenly costs you $50 in API calls. Mock mode is here to prevent that.

The Cost of LLM Development

LLM APIs are priced per token. A single GPT-4 call can cost several cents. Multiply that by hundreds of test runs during development, and you're looking at real money:

Unit tests: Running your test suite shouldn't cost money

Local development: Iterating on prompts and UI shouldn't drain your credits

CI/CD pipelines: Every pull request triggering API calls adds up fast

How Mock Mode Works

IBYOK's mock mode returns realistic-looking but fake API keys based on the provider type:

When your application receives a mock key and tries to use it, the LLM provider will reject it. This is exactly what you want - it forces you to handle API errors gracefully while costing you nothing.

Three-Tier Resolution

IBYOK uses a priority system to determine whether to return mock or real keys:

1. **Infrastructure Guardrails** (highest priority)

• Server admins can force mock mode for specific environments

• Prevents accidental production key usage in dev

2. **Per-Key Overrides**

• Override the default for specific keys

• Useful when testing a new provider while keeping others mocked

3. **Environment Defaults** (lowest priority)

• Set defaults for development, staging, and production

• Most keys follow these settings

Practical Examples

Development Environment

Set your development environment to mock mode. All key retrievals return mock values:

// In development, this returns a mock key

const key = await getKey('openai-key');

// key = "sk-mock-openai-a1b2c3d4e5"

Testing a New Integration

Override a specific key to live mode while keeping everything else mocked:

1. Go to your key's settings in the dashboard

2. Set the development override to "Live"

3. Only that key returns real values; others stay mocked

Production Safety

Lock production to live mode at the infrastructure level. Even if someone accidentally changes their settings, production always gets real keys.

Best Practices

1. **Default to mock in development** - Change to live only when needed

2. **Use per-key overrides sparingly** - They're powerful but can cause confusion

3. **Check modeInfo in responses** - Your app can behave differently based on mode

4. **Log mock mode warnings** - Make it obvious when running against mock keys

Mock mode isn't just a cost-saving feature - it's a safety net that lets you develop with confidence.