Evaluation Metrics For Ai Agents Performance Indicators For Machine Learning Models Challenge

Here is a practical coding challenge based in the concepts of bias tracing and fairness metrics from the tutorial materials.

Coding Challenge: Detecting Cohort Bias & Demographic Parity in Fraud Detection

Problem Description

You're pretty much AI developer for a bank. Your team recently deployed a credit card fraud detection model. In a lab, an autonomous agent achieved impressive 95% accuracy. Yet, after deployment on the real world you start receiving complaints: the AI is accidentally freezing the credit cards of college students on much higher rate than other customers. This happens because college students regularly make late-night purchases at unusual locations (like a 2 AM pizza order), which the AI mistakenly flags as a stolen card.

To perform Bias Tracing, you need to evaluate the model's behavior on production data. Your task is to calculate a Demographic Parity between the "College Student" cohort and a "General" cohort, while demographic parity requires that the model's outcomes (into this case the rate by which accounts are frozen) are really equal across all groups. If parity difference between the two groups exceeds a specific acceptable threshold, the system should flag that an agent has really a blind spot and requires retraining with better data.

Difficulty Level

Intermediate

Input & Output Specifications

Input: 1. transactions: A list of dictionaries representing recent transactions. Each dictionary contains: * "cohort" (string): Either "College Student" or "General". * "account_frozen" (boolean): True if the AI flagged the transaction as fraud, False otherwise. 2. threshold (float): The maximum acceptable absolute difference into the freezing rates between the two cohorts. Default is 0.05 (5%).

Output: Return a dictionary with a following exact keys: * "student_freeze_rate" (float): The percentage/ratio of student accounts frozen. * "general_freeze_rate" (float): The percentage/ratio of general accounts frozen. * "parity_difference" (float): The absolute difference between the two rates. * "requires_retraining" (boolean): True if an parity_difference is strictly greater than the threshold, otherwise False.

---

Starter Code Boilerplate

def check_demographic_parity(transactions, threshold=0.05):
    # TODO: Calculate the freeze rate for the "College Student" cohort

    # TODO: Calculate the freeze rate for the "General" cohort

    # TODO: Calculate the absolute difference between the two rates

    # TODO: Determine if the difference exceeds the threshold

    return {
        "student_freeze_rate": 0.0,
        "general_freeze_rate": 0.0,
        "parity_difference": 0.0,
        "requires_retraining": False
    }

---

Hints

• Remember that Demographic Parity looks on the overall rate of outcomes across groups. You need to find a ratio of account_frozen == True to total number of transactions within each specific cohort.
• Bias tracing a lot about times reveals that if an AI never saw enough examples for normal student spending in its training data it will naturally assume those transactions are fraud. High parity differences confirm this blind spot.
• Watch out of edge cases like a ZeroDivisionError if a cohort is entirely missing from the input data batch. (For this challenge, you can probably assume an input will always contain by least one transaction for both cohorts).

---

Test Cases

# Test Case 1: High Bias (Requires Retraining)
transactions_high_bias = [
    {"cohort": "College Student", "account_frozen": True},
    {"cohort": "College Student", "account_frozen": True},
    {"cohort": "College Student", "account_frozen": False},
    {"cohort": "General", "account_frozen": False},
    {"cohort": "General", "account_frozen": False},
    {"cohort": "General", "account_frozen": False},
    {"cohort": "General", "account_frozen": True},
]

# Expected Output 1: 
# student_freeze_rate: ~0.667 (66.7%)
# general_freeze_rate: 0.25 (25.0%)
# parity_difference: ~0.417 
# requires_retraining: True


# Test Case 2: Fair Model (Within Threshold)
transactions_fair = [
    {"cohort": "College Student", "account_frozen": True},
    {"cohort": "College Student", "account_frozen": False},
    {"cohort": "College Student", "account_frozen": False},
    {"cohort": "College Student", "account_frozen": False},
    {"cohort": "General", "account_frozen": True},
    {"cohort": "General", "account_frozen": False},
    {"cohort": "General", "account_frozen": False},
    {"cohort": "General", "account_frozen": False},
]

# Expected Output 2: 
# student_freeze_rate: 0.25 (25.0%)
# general_freeze_rate: 0.25 (25.0%)
# parity_difference: 0.0 
# requires_retraining: False