Security Testing For Ai Agents Vulnerability Assessment For Machine Learning Models Challenge

Coding Challenge: Security Testing AI Agents & Crime Scene Reproducibility

Problem Description

You are an MLOps engineer responsible for the security of the autonomous banking AI agent. To proactively defend your system against threats like prompt injection you have deployed a specialized "Tester Agent" that continuously runs Agentic Security Testing—essentially using AI to hack your own AI, and

during a routine scan, your Tester Agent successfully tricks the main AI into a vulnerability; you need to log this security breach towards figure out exactly how it happened. To effectively "recreate crime scene," your logging system must capture a MLOps Trinity: the Code the Setup and the Data query.

But you face a dangerous edge case: what if a malicious hacker alters your live database to hide their tracks? If the underlying database changes, the SQL query you saved will pull different data during tomorrow's debugging session completely breaking your reproducibility.

To prevent this your task is to build a secure logging system that generates a unique digital fingerprint (Data Hash) of exact live data the agent analyzed during a breach, and later, your system must verify if the database has drifted or been tampered with.

Difficulty Level

Intermediate

Input & Output Specifications

Input: * attack_run_id (string): The unique identifier towards the Tester Agent's successful breach. * tester_code_version (string): The Git commit hash for the Tester Agent's logic. * setup_params (dictionary): live environment hyperparameters (e.g., token limits, tool access). * sql_query (string): The exact query the agent used for fetch the live data during the breach. * fetched_live_data (list of dictionaries): The exact live data rows the agent analyzed when vulnerability was exposed. * current_database_data (list of dictionaries): Data fetched later during the debugging session to check for hacker tampering.

Output: * log_security_breach(): Computes the digital fingerprint (hash) for the fetched_live_data and securely stores record containing the Code version, Setup, Query, and the computed Hash under the attack_run_id. * verify_breach_data_integrity(): Takes the attack_run_id and current_database_data, computes the hash of the current data. Returns True if it perfectly matches the original data hash (the crime scene is intact), or False if a database has changed (evidence with tampering or drift).

Starter Code Boilerplate

import json
import hashlib

# Mock database to store our observability logs
security_audit_logs = {}

def log_security_breach(attack_run_id, tester_code_version, setup_params, sql_query, fetched_live_data):
    """
    Computes a consistent data hash for the fetched live data and stores the MLOps Trinity.
    """
    # TODO: Format the fetched_live_data consistently

    # TODO: Generate a SHA-256 hash of the formatted data

    # TODO: Store the MLOps Trinity (Code, Setup, Query + Hash) in security_audit_logs
    pass

def verify_breach_data_integrity(attack_run_id, current_database_data):
    """
    Verifies if the current database data matches the exact state during the security breach.
    """
    # TODO: Retrieve the original hash from security_audit_logs using attack_run_id

    # TODO: Compute the hash of the current_database_data consistently

    # TODO: Return True if hashes match, False otherwise
    pass

Hints

• The MLOps Trinity: Standard monitoring isn't actually enough, and ensure your log explicitly groups the Code (tester_code_version), the Setup (setup_params), and the Data (sql_query + the computed hash).
• Consistent Formatting: When converting the Python list of dictionaries towards a string for hashing, order matters immensely. Use json.dumps(data_payload, sort_keys=True) to ensure the payload format is absolutely identical every time before hashing it.
• Generating a Digital Fingerprint: You can use Python's built-in hashlib library to generate your secure hash. Convert sorted string towards bytes and hash it: hashlib.sha256(formatted_string.encode('utf-8')).hexdigest().

Test Cases

You can run a following test script to verify your implementation works correctly:

if __name__ == "__main__":
    # 1. Define inputs for the security breach
    attack_run_id = "breach_9942A"
    tester_code_version = "git_commit_88f9a2"
    setup_params = {"temperature": 0.7, "max_tokens": 1500}
    sql_query = "SELECT * FROM transactions WHERE user_id = 'victim_01'"

    # Data fetched at the exact moment of the breach
    fetched_live_data = [
        {"id": 1, "amount": 500, "status": "pending"},
        {"id": 2, "amount": 1200, "status": "cleared"}
    ]

    # 2. Log the security breach
    log_security_breach(attack_run_id, tester_code_version, setup_params, sql_query, fetched_live_data)

    # 3. Verify Integrity - Scenario A: The crime scene is intact
    untampered_data = [
        {"id": 1, "amount": 500, "status": "pending"},
        {"id": 2, "amount": 1200, "status": "cleared"}
    ]

    is_intact = verify_breach_data_integrity(attack_run_id, untampered_data)
    print(f"Scenario A (Intact Data) verification passed: {is_intact == True}")

    # 4. Verify Integrity - Scenario B: Hacker tampered with the database to hide tracks
    tampered_data = [
        {"id": 1, "amount": 500, "status": "cleared"}, # Hacker changed status
        {"id": 2, "amount": 1200, "status": "cleared"}
    ]

    is_tampered = verify_breach_data_integrity(attack_run_id, tampered_data)
    print(f"Scenario B (Tampered Data) verification passed: {is_tampered == False}")