咨询：Azure Data Factory中提取JIRA自定义字段数据的可行方案

Hey there! I totally get it—working with a beta dataset can be frustrating when you need those critical custom fields that aren’t pulling through by default. But don’t worry, there are a few solid workarounds to make this happen:

1. Use the JIRA REST API Directly #

Since the beta JIRA dataset connector doesn’t natively support custom fields yet, leaning on JIRA’s REST API is a reliable alternative. Here’s how to set this up in Data Factory:

• Add a Web Activity or use the HTTP Connector in your pipeline.
• Build a GET request to JIRA’s search endpoint, explicitly specifying the custom fields you need using their unique IDs or keys. For example:

  GET /rest/api/3/search?jql=project=YOUR_PROJ_KEY&fields=summary,customfield_12345,customfield_67890
  

• To find your custom field IDs: Head to JIRA Settings → Issues → Custom Fields, click "Edit" next to your field, and you’ll see the ID in the URL (it looks like customfield_12345).
• Configure authentication (either OAuth 2.0 or an API Token, depending on your JIRA setup) to make sure Data Factory can access the endpoint.
• Use a Copy Activity or Data Flow to parse the JSON response and pull both standard and custom fields into your target dataset.

2. Tweak the Beta Dataset’s JSON Configuration #

If you’d rather stick with the existing JIRA connector, you can try modifying its underlying JSON definition to include custom fields:

• Open your JIRA dataset in Data Factory, click the "Code" button to view the raw JSON config.
• Locate the typeProperties section, then add a fields array that lists standard fields alongside your custom field IDs. For example:

  "typeProperties": {
      "project": "YOUR_PROJ_KEY",
      "fields": [
          "summary",
          "description",
          "customfield_12345"
      ]
  }
  

• Save the config and test your Copy Activity—this might force the connector to pull the specified custom fields, though keep in mind this behavior could shift as the beta matures.

3. Combine Connector Data with API Data in Data Flow #

If you already have a pipeline using the beta connector for standard fields, you can augment it with custom fields using this approach:

• Use the beta connector to extract standard issue data (including unique issue IDs) into a staging dataset.
• Add a Web Activity to fetch custom fields via the JIRA API, filtering for the same issue IDs from the staging data.
• In a Data Flow, join the staging dataset (standard fields) with the API response (custom fields) using the issue ID as the key.
• Map the combined fields to your target sink.

A quick note: Since this is a beta feature, these workarounds might need adjustments if Microsoft updates the connector’s functionality. Always test thoroughly and keep an eye on official Data Factory release notes for JIRA connector updates.

内容的提问来源于stack exchange，提问作者Gagan