AWS Lambda克隆私有GitHub仓库后无法打包Zip上传至S3求助

Alright, let's fix your Lambda function step by step—there are a few key issues with path handling, S3 upload logic, and best practices for security and reliability here.

Core Issues in Your Current Code #

• Lambda only allows write access to the /tmp directory—your /gitpull path doesn't exist and will throw an error.
• s3_client.upload_file expects a file path, not a directory (you're passing os.getcwd() which is wrong).
• You're trying to zip the current working directory instead of the cloned repo specifically.
• Using os.system doesn't give you error feedback if the git clone fails, and hardcoding credentials is a major security risk.

Fixed Step-by-Step Solution #

1. Use Lambda's /tmp Directory for All File Operations #

Lambda's execution environment only permits writing to /tmp (it has ~512MB of space by default), so we'll base all our operations here.

2. Securely Clone the Private Repo #

Replace hardcoded credentials with a GitHub Personal Access Token (PAT) (with repo scope) and use subprocess instead of os.system to capture errors. Store the PAT in Lambda environment variables instead of hardcoding it.

3. Properly Zip the Cloned Repo #

Target the specific cloned repo directory when creating the zip, not the entire /tmp folder.

4. Correctly Upload the Zip to S3 #

Pass the actual zip file path to s3_client.upload_file, not a directory.

5. Clean Up Temporary Files #

Remove the cloned repo after zipping to avoid leftover files in subsequent executions.

Full Fixed Code #

import boto3
import logging
import os
import shutil
import subprocess
from botocore.exceptions import ClientError

# Configure logging
logger = logging.getLogger()
logger.setLevel(logging.INFO)

# Initialize clients
s3_client = boto3.client('s3')

def lambda_handler(event, context):
    # Lambda's only writable directory
    tmp_dir = "/tmp"
    repo_name = "testrepo"
    repo_clone_path = os.path.join(tmp_dir, repo_name)
    zip_output_path = os.path.join(tmp_dir, "Gitpull")
    s3_bucket_name = "gitpulls3"
    s3_zip_key = "Gitpull.zip"

    # Retrieve GitHub PAT from Lambda environment variables (SAFE!)
    github_pat = os.environ.get("GITHUB_PAT")
    if not github_pat:
        logger.error("GITHUB_PAT environment variable not set")
        return {"statusCode": 500, "body": "Missing GitHub PAT"}

    # Clone command using PAT (replace with your repo URL)
    clone_cmd = f"git clone https://{github_pat}@github.com/awsdemos/{repo_name}.git {repo_clone_path}"

    try:
        # Navigate to tmp directory
        os.chdir(tmp_dir)

        # Clone the repo with error checking
        logger.info(f"Cloning repo to {repo_clone_path}")
        subprocess.check_call(clone_cmd, shell=True)

        # Zip only the cloned repo directory (no extra parent folders)
        logger.info(f"Zipping repo at {repo_clone_path}")
        shutil.make_archive(zip_output_path, 'zip', root_dir=tmp_dir, base_dir=repo_name)
        full_zip_path = f"{zip_output_path}.zip"

        # Upload zip to S3
        logger.info(f"Uploading {full_zip_path} to S3://{s3_bucket_name}/{s3_zip_key}")
        s3_client.upload_file(full_zip_path, s3_bucket_name, s3_zip_key)

        logger.info("Upload complete!")
        return {"statusCode": 200, "body": "Repo cloned and uploaded successfully"}

    except subprocess.CalledProcessError as e:
        logger.error(f"Git clone failed: {str(e)}")
        return {"statusCode": 500, "body": f"Clone error: {str(e)}"}
    except ClientError as e:
        logger.error(f"S3 upload failed: {str(e)}")
        return {"statusCode": 500, "body": f"S3 upload error: {str(e)}"}
    finally:
        # Clean up: delete cloned repo and zip file to free space
        if os.path.exists(repo_clone_path):
            shutil.rmtree(repo_clone_path)
        if os.path.exists(full_zip_path):
            os.remove(full_zip_path)
        logger.info("Cleanup complete")

Key Improvements & Notes #

• Security: Credentials are stored in Lambda environment variables (never hardcode them!). Create a GitHub PAT with minimal permissions (only repo scope for private repos).
• Reliability: subprocess.check_call will throw an error if the git clone fails, letting you catch and log issues.
• Path Handling: All operations use /tmp correctly, and we target the exact repo directory for zipping.
• Cleanup: The finally block ensures temporary files are removed, preventing bloat in Lambda's /tmp directory.

Setup Steps for Environment Variables #

1. Go to your Lambda function in the AWS Console.
2. Under Configuration > Environment variables, add a key GITHUB_PAT with your GitHub Personal Access Token as the value.
3. Save the configuration.

内容的提问来源于stack exchange，提问作者Rajarshi Das