Azure ML Pipeline在Synapse触发失败及Compute集群认证问题求助

问题背景 #

我在Azure ML中有一个可正常运行的Pipeline，已在Azure ML工作区的Notebook中成功运行并发布，准备在Synapse Analytics中调用。但在Synapse触发该Pipeline时，出现以下错误：

Failed to submit job due to Exception: Response status code does not indicate success: 400 (User starting the run is not an owner or assigned user to the Compute Instance). User starting the run is not an owner or assigned user to the Compute Instance.

我同时尝试了Compute Cluster和Compute Instance，均遇到相同问题，组件代码如下：

import os  
import argparse  
from azureml.core import Dataset
from azure.identity import DefaultAzureCredential
from azure.storage.filedatalake import DataLakeServiceClient
import logging
# import mlflow
def main():
    parser=argparse.ArgumentParser()
    parser.add_argument("--data",type=str,help="data lake gen2 path for input data")
    parser.add_argument("--path",type=str,help="path for output")
    parser.add_argument("--outputdata",type=str,help="data lake gen2 path for output data")
    args=parser.parse_args()
    print("test testtest testtest test")
    # mlflow.start_run()
    #my func
    data_path="csv web path"
    # mlflow.log_metric("data_path", args.data)
    dataset = Dataset.Tabular.from_delimited_files(data_path, separator=',',set_column_types=None)
    print(dataset.to_pandas_dataframe().head())
    df = dataset.to_pandas_dataframe()
    token_credential = DefaultAzureCredential()
    account_url = f"ACCOUNTURl LINK"
    service_client = DataLakeServiceClient(account_url, credential=token_credential)
    file_system_client = service_client.get_file_system_client("dataasset2")
    directory_client = file_system_client.get_directory_client("test")
    file_name = args.path
    file_client = directory_client.get_file_client(file_name)
    csv_data = df.to_csv(index=False).encode('utf-8')
    file_client.upload_data(csv_data, overwrite=True)
    # mlflow.end_run()

if __name__=="__main__":
    main()

额外问题：Compute Cluster运行Pipeline失败 #

尝试创建Compute Cluster后，在Azure ML Notebook中运行Pipeline时出现以下错误：

Execution failed. User process 'python' exited with status code 1. Please check log file 'user_logs/std_log.txt' for error details. Error:     dataflow = _transform_and_validate(
  File "/azureml-envs/sklearn-1.1/lib/python3.8/site-packages/azureml/data/dataset_factory.py", line 1225, in _transform_and_validate
    _validate_has_data(dataflow, 'Failed to validate the data.')
  File "/azureml-envs/sklearn-1.1/lib/python3.8/site-packages/azureml/data/dataset_error_handling.py", line 69, in _validate_has_data
    raise DatasetValidationError(error_message + '\n' + e.compliant_message, exception=e)
azureml.data.dataset_error_handling.DatasetValidationError: DatasetValidationError:
    Message: Failed to validate the data.
ScriptExecutionException was caused by StreamAccessException.
  Unable to authenticate data access to '[REDACTED]'.
    AuthenticationException was caused by AzureIdentityAccessTokenResolutionException.
      Compute has no identity provisioned.
| session_id=771c548f-2dbd-4bce-a8b9-eec29c26257b
    InnerException None
    ErrorResponse 
{
    "error": {
        "code": "UserError",
        "message": "Failed to validate the data.\nScriptExecutionException was caused by StreamAccessException.\n  Unable to authenticate data access to '[REDACTED]'.\n    AuthenticationException was caused by AzureIdentityAccessTokenResolutionException.\n      Compute has no identity provisioned.\n| session_id=771c548f-2dbd-4bce-a8b9-eec29c26257b"
    }
}

核心疑问 #

• 如何让Compute Cluster完成与Azure Data Lake Gen2的认证？
• 如何从Synapse启动Compute Instance或Cluster？

注：我的账号同时是Azure ML和Synapse工作区的所有者。

内容的提问来源于stack exchange，提问作者the programmer