单脚本中使用ClearML多任务时日志指标名称自动变更的问题咨询

This is a known issue in ClearML 0.17.4 where the global task context isn't properly reset between loop iterations. This causes subsequent tasks to append their task name as a prefix to metrics (like networkB:loss instead of just loss), making cross-task comparisons impossible. Since the fix is already in the main branch, here's a workaround for your current version:

Solution Code #

Modify your loop to explicitly reset the ClearML global context after closing each task:

from clearml import Task

names = ['networkA', 'networkB']
for name in names:
    # Initialize a completely isolated task
    task = Task.init(
        project_name="NetworkProject", 
        task_name=name,
        # Force a new task instead of reusing previous context
        reuse_last_task_id=False,
        # Reinitialize framework connections for each task
        auto_connect_frameworks={
            'pytorch_lightning': True,
            'tensorboard': True
        }
    )
    # Run your PyTorch Lightning training workflow
    pl_train(name)
    # Finalize logs and close the current task
    task.close()
    # Clear all global task references to avoid cross-task leakage
    Task.reset()

Why This Works #

1. reuse_last_task_id=False: Tells ClearML to create a brand new task instance instead of reusing the context from the previous loop iteration.
2. Task.reset(): Wipes all global task state and cached loggers, ensuring the next Task.init() starts with a clean slate. This eliminates the metric naming prefix issue entirely.
3. Explicit auto_connect_frameworks: Ensures the connection between ClearML and PyTorch Lightning is reinitialized for each task, rather than relying on a cached connection that might carry over context.

Additional Troubleshooting #

If you still see metric prefixes after trying the above, double-check that your pl_train method creates a fresh TensorBoardLogger instance for each run. Even without explicit ClearML code, reusing a logger across tasks can cause residual context leakage.

内容的提问来源于stack exchange，提问作者Daniel Andres