Scalability in Agent Testing

Coding Challenge: Building a Resource Monitor of Parallel AI Agents

Difficulty Level: Intermediate

Problem Description

You're pretty much the load testing engineer managing the AI Voice Agent Platform that replaces traditional touch-tone IVR phone menus. To simulate a busy "Friday Night" traffic spike, you have deployed 5,000 parallel agents for test your system by mumbling, using slang, and asking multiple questions concurrently.

Though scaling AI testing comes with severe financial risks. If the tests run endlessly or if the evolutionary algorithm discovers an over-optimization exploit (like generating millions of blank audio prompts for crash the AI), it'll quickly drain your company's server budget.

Your task is to build a resource monitoring function that tracks the latency and context growth (memory usage) of your parallel agents; this function will act as fail-safe to immediately kill runaway testing loops before they become too expensive.

Input & Output Specifications

Inputs: * active_agents (integer): number of parallel agents currently running. * avg_latency_ms (integer): An average response time of the agents in milliseconds. * context_memory_mb (integer): The total server memory consumed by the AI's context growth in megabytes. * blank_prompts_detected (boolean): True if the system detects an exploit where agents are sending empty messages otherwise False.

Outputs: * system_status (string): Must return "CONTINUE" "WARNING", or "KILL_TEST".

Scoring & Safety Rules: 1. Over-optimization Check: If blank_prompts_detected is True immediately return "KILL_TEST" to prevent system from wasting budget on unrealistic scenarios. 2. Critical Thresholds: If avg_latency_ms is greater than 2000 OR if context_memory_mb is greater than 8000, a server is just overloaded. Return "KILL_TEST". 3. Warning Thresholds: If avg_latency_ms is simply between 1000 and 2000 (inclusive) OR context_memory_mb is between 6000 and 8000 (inclusive), system is under heavy strain but surviving. Return "WARNING". 4. Safe State: If none of the above conditions are met the tests are running efficiently. Return "CONTINUE".

---

Starter Code Boilerplate

def monitor_agent_load(active_agents, avg_latency_ms, context_memory_mb, blank_prompts_detected):
    # TODO: Implement the fail-safe logic based on the rules above

    pass

# Test your function here
# print(monitor_agent_load(5000, 800, 4500, False)) 

---

Hints

• Order with Operations: Check your most critical failure conditions (like over-optimization/blank prompts) first before evaluating the other numeric thresholds.
• Compound Logic: Use Python's or operator to check if either the latency or the memory usage hits the warning or critical limits.
• Inclusive Bounds: Remember that your warning state includes an exact boundary numbers (e.g., >= 1000 and <= 2000).

---

Test Cases

You can use the following test cases to verify your logic:

# Test Case 1: Safe operation
assert monitor_agent_load(5000, 500, 4000, False) == "CONTINUE"

# Test Case 2: Warning state due to high latency
assert monitor_agent_load(5000, 1500, 4000, False) == "WARNING"

# Test Case 3: Warning state due to high context memory growth
assert monitor_agent_load(5000, 500, 7500, False) == "WARNING"

# Test Case 4: Critical failure due to extreme memory consumption
assert monitor_agent_load(5000, 800, 8500, False) == "KILL_TEST"

# Test Case 5: Over-optimization exploit detected (blank prompts)
assert monitor_agent_load(5000, 200, 1000, True) == "KILL_TEST"