NiFi跨FTP站点传输SFTP子文件夹文件失败求助

Hey Aadil, let's get this SFTP-to-FTP workflow working properly—your current setup is hitting a path mismatch issue, and we can fix it with a couple of adjustments.

First, let's diagnose the problem with your original config #

When you set ListSFTP to target /rootfolder (with recursion enabled, I assume?), it generates FlowFiles with filenames that include the full relative path from /rootfolder—like subfolder1/file.txt, subfolder2/report.pdf, etc. But when you point each FetchSFTP to a fixed subfolder (e.g., /rootfolder/subfolder1), NiFi tries to combine that fixed path with the FlowFile's filename, resulting in invalid paths like /rootfolder/subfolder1/subfolder1/file.txt—which doesn't exist on your SFTP server. That's why the workflow fails.

Solution 1: Simplified Workflow (Recommended) #

You don't need three separate FetchSFTP components. We can handle all subfolders with a single pipeline that preserves the directory structure:

1. Configure ListSFTP

   • Remote Path: /rootfolder
   • Recurse Subdirectories: Set to true (this tells NiFi to scan all subfolders under /rootfolder)
   • Verify the generated FlowFiles have attributes like:
     • filename: subfolder1/file.txt (relative path from /rootfolder)
     • absolute.path: /rootfolder/

2. Configure a Single FetchSFTP

   • Remote Path: Leave this blank, or set it to ${absolute.path} (NiFi will automatically combine this with the filename attribute to locate the correct remote file—e.g., /rootfolder/ + subfolder1/file.txt = /rootfolder/subfolder1/file.txt)
   • Fill in your SFTP server credentials and connection details as usual

3. Configure PutFTP to Preserve Structure

   • Remote Directory: Your target FTP root directory (e.g., /ftp_target_root)
   • Create Directory: Set to true (NiFi will automatically create subfolder1, subfolder2, etc., in the FTP target)
   • Remote Filename: ${filename} (this ensures files land in the matching subfolder—e.g., /ftp_target_root/subfolder1/file.txt)

Solution 2: Split Workflow for Per-Subfolder Processing #

If you need separate logic for each subfolder (e.g., different renaming, filtering), use a routing step to split the FlowFiles first:

1. ListSFTP (same as Solution 1: /rootfolder with recursion enabled)
2. Add RouteOnAttribute
   • Create three routing rules to split FlowFiles by their subfolder:
     • Rule 1: Route to subfolder1 when filename starts with 'subfolder1/'
     • Rule 2: Route to subfolder2 when filename starts with 'subfolder2/'
     • Rule 3: Route to subfolder3 when filename starts with 'subfolder3/'
3. Configure FetchSFTP for Each Subfolder
   • For the subfolder1 branch:
     • Remote Path: /rootfolder/subfolder1
     • Remote Filename: ${filename:substringAfter('subfolder1/')} (this strips the subfolder1/ prefix from the filename, so NiFi looks for file.txt instead of subfolder1/file.txt in /rootfolder/subfolder1)
   • Repeat this pattern for the other two subfolders
4. Configure PutFTP for Each Subfolder
   • For the subfolder1 branch:
     • Remote Directory: /ftp_target_root/subfolder1
     • Create Directory: true
     • Remote Filename: ${filename:substringAfter('subfolder1/')}

Quick Troubleshooting Tips #

• Use the LogAttribute component after ListSFTP to inspect FlowFile attributes—this helps confirm that filename and absolute.path are correctly set
• Ensure your NiFi service account has read permissions on all /rootfolder subdirectories, and write permissions on the target FTP directories
• Test with a small set of files first to validate the directory structure is preserved correctly

内容的提问来源于stack exchange，提问作者Aadil M