如何通过API等代码方式更新ADF linked service配置并添加init scripts
ADF 非UI方式更新Linked Service添加init scripts配置实现方案
ADF的资源创建和更新共用幂等接口,不需要单独的编辑接口,直接传入修改后的完整配置即可完成覆盖更新,以下是三种常用实现方式:
方法1:使用Azure REST API
- 接口地址格式:
PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataFactory/factories/{factoryName}/linkedservices/{linkedServiceName}?api-version=2018-06-01 - 请求头需要携带Azure AD认证的Bearer token
- 请求Body直接复制原有Linked Service的完整配置,在作业集群属性节点下新增init scripts配置段即可,以Databricks类型的Linked Service为例:
{ "properties": { "type": "AzureDatabricks", "typeProperties": { "domain": "https://xxxx.azuredatabricks.net", "accessToken": { "type": "SecureString", "value": "你的访问令牌" }, "instancePoolId": "你的实例池ID", "clusterVersion": "11.3.x-scala2.12", "workerType": "Standard_D3_v2", "workerCount": 2, "initScripts": [ "dbfs:/databricks/scripts/init1.sh", "dbfs:/databricks/scripts/init2.sh" ] } } }
- 调用成功后会直接覆盖原有Linked Service配置,完成更新。
方法2:使用Azure PowerShell
- 首先安装
Az.DataFactory模块 - 第一步拉取原有Linked Service配置:
$ls = Get-AzDataFactoryV2LinkedService -ResourceGroupName "你的资源组名" -DataFactoryName "你的ADF实例名" -Name "目标Linked Service名称" - 第二步添加init scripts配置:
$ls.Properties.TypeProperties["initScripts"] = @("dbfs:/databricks/scripts/init1.sh", "dbfs:/databricks/scripts/init2.sh") - 第三步执行更新操作:
Set-AzDataFactoryV2LinkedService -ResourceGroupName "你的资源组名" -DataFactoryName "你的ADF实例名" -Name "目标Linked Service名称" -Definition $ls
方法3:使用Python SDK
- 先安装依赖包:
pip install azure-mgmt-datafactory azure-identity - 代码示例:
from azure.identity import DefaultAzureCredential from azure.mgmt.datafactory import DataFactoryManagementClient # 初始化客户端 credential = DefaultAzureCredential() adf_client = DataFactoryManagementClient(credential, "你的订阅ID") # 拉取原有Linked Service配置 ls = adf_client.linked_services.get("你的资源组名", "你的ADF实例名", "目标Linked Service名称") # 添加init scripts配置 ls.properties.type_properties["initScripts"] = ["dbfs:/databricks/scripts/init1.sh", "dbfs:/databricks/scripts/init2.sh"] # 执行更新 adf_client.linked_services.begin_create_or_update("你的资源组名", "你的ADF实例名", "目标Linked Service名称", ls)
注意:更新为全量覆盖操作,务必确保传入的配置包含原有Linked Service的所有属性,不要遗漏非修改项,否则会导致原有配置丢失。如果不确定原有配置,可以先调用对应工具的查询接口拉取完整配置后再做修改。
内容的提问来源于stack exchange,提问作者Vikas
相关产品推荐
相关产品推荐

