如何用PowerShell cmdlets运行/重跑Azure Data Factory切片及设置失败通知

Hey there, let's break down your Azure Data Factory questions with clear, actionable answers:

1. Can I run Azure Data Factory slices using PowerShell cmdlets? #

Absolutely! You can use PowerShell to trigger or re-run ADF slices (whether they're scheduled or failed historical runs). The exact cmdlets depend on whether you're using ADF v1 or ADF v2 (the current recommended version):

• For ADF v2: You'll work with pipeline runs (which map to what you might think of as "slices" in v1). Use Start-AzDataFactoryV2PipelineRun to trigger a new run, or reference a failed run ID to re-execute it.
• For ADF v1: Use Resume-AzureRmDataFactorySlice to restart failed slices directly.

2. How to bulk re-run 500+ failed weekly slices after a server outage? #

Manual re-runs are a total pain—let's automate this with PowerShell. Below is a step-by-step script for ADF v2 (adjust for v1 if needed):

Step 1: Set up your Azure connection #

# Log into your Azure account
Connect-AzAccount

# Select the correct subscription
Select-AzSubscription -SubscriptionId "your-subscription-id-here"

Step 2: Define your ADF details and filter failed runs #

# Define variables for your environment
$resourceGroupName = "your-resource-group-name"
$dataFactoryName = "your-data-factory-name"
$pipelineName = "your-target-pipeline-name"

# Set time range to target the failed slices (adjust to match your outage window)
$startTime = (Get-Date).AddDays(-5) # Covers the start of your 4-day-old outage
$endTime = (Get-Date).AddDays(-3)   # Covers the end of the outage window

# Fetch all failed pipeline runs in the specified time frame
$failedRuns = Get-AzDataFactoryV2PipelineRun `
    -ResourceGroupName $resourceGroupName `
    -DataFactoryName $dataFactoryName `
    -PipelineName $pipelineName `
    -StartTime $startTime `
    -EndTime $endTime | 
    Where-Object { $_.Status -eq "Failed" }

Step 3: Bulk re-run the failed slices #

# Loop through each failed run and re-execute it
foreach ($run in $failedRuns) {
    Write-Host "Re-running pipeline run ID: $($run.RunId)"
    Start-AzDataFactoryV2PipelineRun `
        -ResourceGroupName $resourceGroupName `
        -DataFactoryName $dataFactoryName `
        -PipelineName $pipelineName `
        -ReferencePipelineRunId $run.RunId
}

Note for ADF v1 users: Replace the above with Get-AzureRmDataFactorySlice to fetch failed slices, then use Resume-AzureRmDataFactorySlice to restart them. Be sure to filter slices by their Failed status and the correct time window.

3. How to set up email notifications for failed slice runs? #

You have two reliable options to get alerted when slices/runs fail:

Option 1: Use Azure Monitor Alerts (built-in to ADF) #

This is the simplest, no-code method:

• Go to your Azure Data Factory in the portal.
• Left navigation > Alerts > New alert rule.
• Under Condition, select Pipeline run failed (or "Slice failed" for ADF v1). Configure the trigger to alert on failed runs for your target pipeline.
• Under Actions, create a new action group. Add an Email/SMS/Push/Voice action and enter the email addresses you want to notify.
• Name your alert rule, add a brief description, and save it. You'll get an email instantly every time a pipeline run (or slice) fails.

Option 2: Use Logic Apps for customized notifications #

If you need more control (e.g., include detailed run info, filter specific failure reasons), use a Logic App:

• Create a new Logic App.
• Choose the When a resource event occurs trigger, set the resource type to Microsoft.DataFactory/factories/pipelineruns, and filter for Failed status events.
• Add an Send email (Office 365 Outlook) action (or another email provider) to send a notification with run ID, pipeline name, failure reason, and other relevant details.

内容的提问来源于stack exchange，提问作者Veerendra Patil