如何用Java程序对SFTP服务器文件夹内文件执行压缩与Base64编码

Hey there! Let's walk through how to implement your requirement—zipping files from an SFTP folder and encoding the zip to Base64—using Java. I'll break this down into manageable steps with code snippets and key considerations to avoid pitfalls.

Step 1: Add Required Dependencies #

You'll need JSch for SFTP connectivity (since Java doesn't have built-in SFTP support). If you're using Maven, add this to your pom.xml:

<dependency>
    <groupId>com.jcraft</groupId>
    <artifactId>jsch</artifactId>
    <version>0.1.55</version> <!-- Use the latest stable version -->
</dependency>

For Gradle, add:

implementation 'com.jcraft:jsch:0.1.55'

Step 2: Connect to SFTP & Download Files to a Temporary Directory #

First, we'll establish an SFTP connection, list all files in the target folder, and download them to a temporary local directory (this is safer than holding everything in memory for large files).

Here's a reusable method for SFTP file download:

import com.jcraft.jsch.*;
import java.io.*;
import java.nio.file.*;
import java.util.Properties;

public class SftpZipBase64Handler {

    private static final String SFTP_HOST = "your-sftp-host";
    private static final int SFTP_PORT = 22;
    private static final String SFTP_USER = "your-username";
    private static final String SFTP_PASSWORD = "your-password"; // Or use private key auth
    private static final String SFTP_TARGET_FOLDER = "/path/to/remote/folder";

    private Path downloadSftpFiles() throws JSchException, SftpException, IOException {
        // Create a temporary directory to store downloaded files
        Path tempDir = Files.createTempDirectory("sftp-download-");

        JSch jsch = new JSch();
        Session session = jsch.getSession(SFTP_USER, SFTP_HOST, SFTP_PORT);
        session.setPassword(SFTP_PASSWORD);

        // Disable strict host key checking for testing (adjust for production!)
        Properties config = new Properties();
        config.put("StrictHostKeyChecking", "no");
        session.setConfig(config);

        session.connect();
        ChannelSftp channelSftp = (ChannelSftp) session.openChannel("sftp");
        channelSftp.connect();

        // Navigate to target folder and list files
        channelSftp.cd(SFTP_TARGET_FOLDER);
        @SuppressWarnings("unchecked")
        java.util.Vector<ChannelSftp.LsEntry> fileList = channelSftp.ls(".");

        for (ChannelSftp.LsEntry entry : fileList) {
            String fileName = entry.getFilename();
            // Skip directories and hidden files like "." and ".."
            if (!entry.getAttrs().isDir() && !fileName.startsWith(".")) {
                Path localFilePath = tempDir.resolve(fileName);
                try (OutputStream os = Files.newOutputStream(localFilePath)) {
                    channelSftp.get(fileName, os);
                }
                System.out.println("Downloaded: " + fileName);
            }
        }

        // Clean up SFTP resources
        channelSftp.disconnect();
        session.disconnect();

        return tempDir;
    }
}

Note: For production, avoid disabling StrictHostKeyChecking—instead, add the host key to your known_hosts file. You can also use private key authentication instead of passwords for better security.

Step 3: Compress the Downloaded Files into a Zip Archive #

Next, we'll take the temporary directory of files and compress them into a zip file. We can either write the zip to a temporary file (good for large files) or directly to an in-memory byte array (good for small files).

Option A: Compress to a Temporary Zip File #

private Path compressToZip(Path sourceDir) throws IOException {
    Path zipFile = Files.createTempFile("sftp-files-", ".zip");

    try (ZipOutputStream zos = new ZipOutputStream(Files.newOutputStream(zipFile))) {
        Files.walk(sourceDir)
             .filter(path -> !Files.isDirectory(path))
             .forEach(path -> {
                 ZipEntry zipEntry = new ZipEntry(sourceDir.relativize(path).toString());
                 try (InputStream is = Files.newInputStream(path)) {
                     zos.putNextEntry(zipEntry);
                     byte[] buffer = new byte[1024];
                     int len;
                     while ((len = is.read(buffer)) > 0) {
                         zos.write(buffer, 0, len);
                     }
                     zos.closeEntry();
                 } catch (IOException e) {
                     throw new RuntimeException("Failed to add file to zip: " + path, e);
                 }
             });
    }

    return zipFile;
}

Option B: Compress to an In-Memory Byte Array #

If your files are small enough to fit in memory, you can skip the temporary zip file:

private byte[] compressToByteArray(Path sourceDir) throws IOException {
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    try (ZipOutputStream zos = new ZipOutputStream(baos)) {
        Files.walk(sourceDir)
             .filter(path -> !Files.isDirectory(path))
             .forEach(path -> {
                 ZipEntry zipEntry = new ZipEntry(sourceDir.relativize(path).toString());
                 try (InputStream is = Files.newInputStream(path)) {
                     zos.putNextEntry(zipEntry);
                     byte[] buffer = new byte[1024];
                     int len;
                     while ((len = is.read(buffer)) > 0) {
                         zos.write(buffer, 0, len);
                     }
                     zos.closeEntry();
                 } catch (IOException e) {
                     throw new RuntimeException("Failed to add file to zip: " + path, e);
                 }
             });
    }
    return baos.toByteArray();
}

Step 4: Encode the Zip to Base64 #

Java 8+ has a built-in Base64 class that makes this straightforward.

If using a temporary zip file: #

private String encodeZipToBase64(Path zipFile) throws IOException {
    byte[] zipBytes = Files.readAllBytes(zipFile);
    return Base64.getEncoder().encodeToString(zipBytes);
}

If using the in-memory byte array: #

private String encodeByteArrayToBase64(byte[] zipBytes) {
    return Base64.getEncoder().encodeToString(zipBytes);
}

Step 5: Clean Up Temporary Resources #

Don't forget to delete the temporary files and directories to avoid cluttering the filesystem:

private void cleanUpTempResources(Path tempDir, Path zipFile) throws IOException {
    // Delete zip file if exists
    if (zipFile != null && Files.exists(zipFile)) {
        Files.delete(zipFile);
    }
    // Recursively delete temporary directory
    Files.walkFileTree(tempDir, new SimpleFileVisitor<Path>() {
        @Override
        public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
            Files.delete(file);
            return FileVisitResult.CONTINUE;
        }

        @Override
        public FileVisitResult postVisitDirectory(Path dir, IOException exc) throws IOException {
            Files.delete(dir);
            return FileVisitResult.CONTINUE;
        }
    });
}

Putting It All Together #

Here's a main method to tie everything together:

public static void main(String[] args) {
    SftpZipBase64Handler handler = new SftpZipBase64Handler();
    Path tempDir = null;
    Path zipFile = null;
    try {
        // Step 1: Download files from SFTP
        tempDir = handler.downloadSftpFiles();

        // Step 2: Compress to zip file
        zipFile = handler.compressToZip(tempDir);

        // Step 3: Encode to Base64
        String base64Zip = handler.encodeZipToBase64(zipFile);

        // Do something with the Base64 string (e.g., send to API, store in DB)
        System.out.println("Base64 encoded zip length: " + base64Zip.length());

    } catch (Exception e) {
        e.printStackTrace();
    } finally {
        // Clean up
        try {
            if (tempDir != null && zipFile != null) {
                handler.cleanUpTempResources(tempDir, zipFile);
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Key Considerations #

• Large Files: If you're dealing with very large files, stick to temporary files instead of in-memory byte arrays to avoid OutOfMemoryError.
• Error Handling: Add more granular exception handling (e.g., handle SFTP connection failures, file not found errors) based on your use case.
• Security: Never hardcode SFTP credentials—use environment variables, configuration files, or a secrets manager instead.
• Performance: Adjust the buffer size (currently 1024 bytes) based on your file sizes for better throughput.

内容的提问来源于stack exchange，提问作者Sai