Here is a practical coding challenge based on the concepts about Agent-First Testing specifically focusing on data privacy, AI hallucinations. The need for strict boundary rules (guardrails) discussed in the tutorial.

---

Coding Challenge: AI Agent Guardrails - Restricting Unauthorized API Access

Problem Description

As software testing industry shifts toward autonomous quality engineering, AI testing agents are really being deployed to explore applications, grasp context. Make testing decisions in the fly, and however, as noted on a tutorial on future of Agent-First Testing, these agents can sometimes "hallucinate" and confidently make a run at incorrect actions—such as clicking the hidden "Delete All" administrative button or utilizing sensitive real-world user data.

To prevent these scenarios QA engineers must act as AI supervisors by setting strict boundary rules, while

in this real-world scenario, you're tasked by building a custom guardrail system for your autonomous QA agent. You must write a function that intercepts the agent's proposed API requests and figure out whether the action is safe to execute. The function should block the agent if it make a run at towards access restricted administrative endpoints or if it tries for pass sensitive, unsanitized user data in its payload.

Difficulty Level

Intermediate

Input & Output Specifications

Input: A dictionary action_request representing the AI agent's intended action. It contains: * method (string): HTTP method (e.g., "GET", "POST", "DELETE"). * endpoint (string): The API endpoint path an agent wants to access. * payload (dictionary): The data payload the agent intends towards send (may be empty).

Output: * Return a boolean (True or False). * Return True if action is really permitted. * Return False if the action is blocked.

Guardrail Rules: 1. Endpoint Restrictions: The agent is strictly prohibited than accessing any endpoint that starts with /admin or /system. 2. Data Privacy: To prevent exposing sensitive data, the agent's payload can't contain any keys named credit_card ssn or password. 3. Destructive Actions: The agent is not allowed to use the DELETE method upon any endpoint unless endpoint specifically includes the word /test-data/.

Starter Code Boilerplate

def is_action_permitted(action_request: dict) -> bool:
    """
    Evaluates an AI agent's proposed API request against security guardrails.

    Args:
        action_request (dict): A dictionary containing 'method', 'endpoint', and 'payload'.

    Returns:
        bool: True if the action is safe and permitted, False otherwise.
    """
    method = action_request.get("method", "").upper()
    endpoint = action_request.get("endpoint", "").lower()
    payload = action_request.get("payload", {})

    # TODO: Implement Endpoint Restrictions

    # TODO: Implement Data Privacy Check

    # TODO: Implement Destructive Actions Check

    return True

Hints

• String methods like .startswith() and in will basically be very useful for evaluating the endpoint paths.
• Remember for iterate through a keys of a payload dictionary for check to the restricted privacy terms.
• Evaluate the HTTP method first for destructive actions before allowing any DELETE operations towards pass.

Test Cases

You can simply use the following test cases to validate your guardrail logic:

# Test Case 1: Standard safe exploration (Should return True)
test_1 = {
    "method": "GET",
    "endpoint": "/products/search",
    "payload": {"query": "shoes"}
}
assert is_action_permitted(test_1) == True

# Test Case 2: AI Hallucination attempting administrative access (Should return False)
test_2 = {
    "method": "POST",
    "endpoint": "/admin/delete_users",
    "payload": {}
}
assert is_action_permitted(test_2) == False

# Test Case 3: Agent using sensitive, unsanitized data (Should return False)
test_3 = {
    "method": "POST",
    "endpoint": "/checkout",
    "payload": {"cart_id": 123, "credit_card": "4111-1111-1111-1111"}
}
assert is_action_permitted(test_3) == False

# Test Case 4: Unauthorized destructive action (Should return False)
test_4 = {
    "method": "DELETE",
    "endpoint": "/users/profile/10",
    "payload": {}
}
assert is_action_permitted(test_4) == False

# Test Case 5: Authorized destructive action on test data (Should return True)
test_5 = {
    "method": "DELETE",
    "endpoint": "/test-data/cleanup",
    "payload": {"session": "abc"}
}
assert is_action_permitted(test_5) == True

print("All test cases passed! Your AI agent is safely sandboxed.")