TestRail与Pytest集成：如何按测试步骤同步自动化执行结果？

To achieve per-step result synchronization between your pytest tests and TestRail, you’ll need to use the TestRail API directly with a custom pytest reporter. This gives you full control to map each test method to a TestRail case step and update results correctly.

Step 1: Add Metadata to Your Tests #

Use pytest markers to link each test method to its corresponding TestRail case ID and step number. This ensures we can group results by case later.

import pytest

class TestTestCase1:
    @pytest.mark.testrail(case_id=101, step_number=1)
    def test_tc1_step_1(self):
        assert 1 == 1

    @pytest.mark.testrail(case_id=101, step_number=2)
    def test_tc1_step_2(self):
        assert 2 == 3

    @pytest.mark.testrail(case_id=101, step_number=3)
    @pytest.mark.parametrize("a, b", [("2+4", 7), ("3+5", 8)])
    def test_tc1_step_3(self, a, b):
        assert eval(a) == b

    @pytest.mark.testrail(case_id=101, step_number=4)
    def test_tc1_step_4(self):
        assert 4 == 4

    @pytest.mark.testrail(case_id=101, step_number=5)
    def test_tc1_step_5(self):
        assert 5 == 5

Step 2: Create a Custom Pytest Reporter #

Create a conftest.py file in your test directory. This file will collect test results, group them by TestRail case ID, and send updates to TestRail via its API.

import pytest
import requests
from datetime import datetime

# Configure TestRail API details
TESTRAIL_URL = "https://your-testrail-instance.testrail.io/"
TESTRAIL_USER = "your-email@example.com"
TESTRAIL_API_KEY = "your-api-key"
TESTRAIL_RUN_ID = 123  # Replace with your existing test run ID

# Map pytest statuses to TestRail status IDs (adjust if your instance uses custom statuses)
STATUS_MAP = {
    "passed": 1,
    "failed": 5,
    "skipped": 4
}

# Store results grouped by TestRail case ID
test_results = {}

def pytest_runtest_makereport(item, call):
    if call.when == "call":
        # Extract TestRail metadata from the marker
        testrail_marker = item.get_closest_marker("testrail")
        if not testrail_marker:
            return
        
        case_id = testrail_marker.kwargs.get("case_id")
        step_number = testrail_marker.kwargs.get("step_number")
        
        # Determine test status and comment
        if call.excinfo is None:
            status = "passed"
            comment = "Step executed successfully"
        elif isinstance(call.excinfo.value, AssertionError):
            status = "failed"
            comment = f"Assertion failure: {str(call.excinfo.value)}"
        else:
            status = "failed"
            comment = f"Unexpected error: {str(call.excinfo.value)}"
        
        # Add parametrization details to comment if applicable
        if hasattr(item, "callspec"):
            comment += f"\nParameters used: {item.callspec.params}"
        
        # Group results by case ID
        if case_id not in test_results:
            test_results[case_id] = []
        test_results[case_id].append({
            "step_number": step_number,
            "status_id": STATUS_MAP[status],
            "comment": comment
        })

def pytest_sessionfinish(session, exitstatus):
    # Send aggregated results to TestRail after all tests run
    if not test_results:
        return
    
    api_endpoint_template = f"{TESTRAIL_URL}index.php?/api/v2/add_result_for_case/{TESTRAIL_RUN_ID}/{{case_id}}"
    
    for case_id, steps in test_results.items():
        # Sort steps by their number to match TestRail's order
        sorted_steps = sorted(steps, key=lambda x: x["step_number"])
        
        # Set overall case status to the worst outcome (e.g., if any step fails, case fails)
        overall_status = max(step["status_id"] for step in sorted_steps)
        
        # Prepare payload for TestRail API
        payload = {
            "status_id": overall_status,
            "steps": sorted_steps,
            "comment": f"Automated run completed at {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}",
            "created_on": int(datetime.now().timestamp())
        }
        
        # Send request to TestRail
        response = requests.post(
            api_endpoint_template.format(case_id=case_id),
            auth=(TESTRAIL_USER, TESTRAIL_API_KEY),
            json=payload,
            headers={"Content-Type": "application/json"}
        )
        
        if response.status_code == 200:
            print(f"Successfully updated TestRail case {case_id}")
        else:
            print(f"Failed to update case {case_id}: {response.text}")

Step 3: Prerequisites & Setup #

1. Enable TestRail API: Go to Admin > Site Settings > API and enable the API.
2. Get API Key: Navigate to My Settings > API Key to generate your key.
3. Install Dependencies: Run pip install requests to install the required library.
4. Update Configuration: Replace the placeholder values in conftest.py with your TestRail instance details and test run ID.

Key Notes #

• Step Matching: Ensure the step_number in markers exactly matches the step order in TestRail (e.g., step 1 in TestRail maps to step_number=1).
• Parametrized Steps: The reporter adds parametrization details to the step comment. If a parametrized step has multiple failures, all details will be included.
• Existing Test Runs: This solution updates an existing test run (specified by TESTRAIL_RUN_ID), aligning with your requirement to associate results with existing test plans.
• Custom Statuses: If your TestRail instance uses custom statuses, update the STATUS_MAP to match your status IDs.

内容的提问来源于stack exchange，提问作者msqa