寻求Azure Data Factory V2与Salesforce跨Org数据迁移实用技巧

Hey there! I’ve spent a fair amount of time working on cross-org Salesforce migrations using ADF v2—here are some hands-on tips that’ve helped me avoid common pitfalls and keep things running smoothly:

• Leverage Salesforce Bulk API v2 for large datasets
  When dealing with big volumes of data (think 10k+ records), always enable the Use Bulk API v2 option in your ADF Salesforce linked service. It’s way more efficient than the older Bulk API v1, handles asynchronous jobs natively, and lets you set batch sizes up to Salesforce’s 10k record limit. Just make sure your ADF pipeline includes logic to wait for batch completion—don’t rush to the next step before the bulk job finishes!

• Handle Salesforce system fields properly
  You can’t directly write to fields like Id, CreatedDate, or LastModifiedDate in the target Org unless you’ve enabled the "Set Audit Fields upon Record Creation" permission in Salesforce (ask your admin to turn this on for your integration user). For cross-org ID mapping, forget about using the source Org’s Id—instead, use a custom External Id field to link records between Orgs. In ADF’s data mapping, use functions like coalesce() to handle missing external IDs gracefully.

• Build incremental syncs to save time and API limits
  Full migrations are a one-time thing, but ongoing syncs should always be incremental. Use Salesforce’s SystemModstamp field (more reliable than LastModifiedDate for system changes) as your filter. In ADF, parameterize your SOQL query like this:
  SELECT Id, Name, External_Id__c, ... FROM Account WHERE SystemModstamp > @pipeline().parameters.LastSuccessfulSync
  Store the last sync timestamp in an Azure SQL table or ADF variable so your pipeline automatically picks up where it left off next time.

• Respect Salesforce object relationship order
  If you’re migrating related objects (like Accounts and Contacts), always migrate parent objects first. In ADF, set up pipeline dependencies using the "Success" trigger between Copy activities—wait for the Account migration to finish before starting Contacts. Again, use External Ids to link child records to their parents in the target Org, not the source Org’s Id values.

• Build robust error handling and retry logic
  Salesforce has strict API limits and validation rules that’ll trip you up. In ADF:

  • Enable exponential backoff retries for your Copy activities to handle transient API errors.
  • Turn on Log Error Rows to send failed records to an Azure Blob Storage container or SQL table—this lets you debug issues like field length violations or missing required fields without re-running the entire pipeline.
  • Keep concurrent pipeline runs low (start with 3-5) to avoid hitting Salesforce’s daily API limit.

• Test with small batches first
  Never run a full migration without testing! Use ADF’s Lookup activity to pull a small sample (100-200 records) from the source Org, run a test Copy to the target, and verify:

  • All fields map correctly (especially custom fields)
  • Relationships between objects are intact
  • System fields like CreatedDate are populated as expected
  • Validation rules in the target Org don’t block records

Hope these tips make your cross-org migration go a lot smoother! If you hit specific snags—like dealing with complex custom objects or weird validation rules—feel free to share details and I can help troubleshoot further.

内容的提问来源于stack exchange，提问作者Radosław Mikołajczyk