如何基于单个Amazon AMI部署三个不同环境并传入自定义配置文件

Great question! Running dev, qa, and prod environments from a single Amazon AMI is a fantastic approach to maintain consistency across your deployments while cutting down on the overhead of managing multiple separate images. Let’s walk through how to make this work, plus how to inject custom configs when setting up or waiting on instances.

The key here is to create a base "golden" AMI that contains all the shared components of your application—like the OS, runtime (Python/Node.js/Java), dependencies, and core application binaries—but no environment-specific configuration. All environment differences (database URLs, API keys, feature flags, etc.) will be injected at instance launch time.

Here’s how to set up your AMI to adapt to dev, qa, or prod:

• Build your golden AMI consistently
  Use a tool like Packer to automate AMI creation. This ensures every version of your base AMI is identical, eliminating "it works on my machine" issues across environments. Your Packer template would install all shared dependencies, but skip any environment-specific setup.

• Use Instance Metadata & User Data to Define Environment Context
  When launching an instance, pass a user data script that tells the instance which environment it’s running in, then pulls the corresponding config. For example:

  #!/bin/bash
  # Set environment variable for the instance
  echo "ENVIRONMENT=prod" >> /etc/environment
  source /etc/environment
  
  # Pull environment-specific config from S3 (ensure instance has IAM permissions for this bucket)
  aws s3 cp s3://your-config-bucket/${ENVIRONMENT}/app-config.yaml /opt/your-app/config.yaml
  
  # Restart the application to apply new config
  systemctl restart your-app.service
  

  For dev, qa, or prod, you’d just change the ENVIRONMENT value in the user data when launching the instance.

• Securely Store Environment Configs with AWS Services
  Instead of hardcoding configs in user data, use SSM Parameter Store or Secrets Manager to store sensitive and non-sensitive configs. For example:

  • Store /dev/db-url, /qa/db-url, /prod/db-url in Parameter Store
  • Assign an IAM instance profile to your instances that grants access only to the parameters matching their environment (e.g., dev instances can read /dev/* parameters)
  • Modify your user data script to fetch these parameters at launch:

    DB_URL=$(aws ssm get-parameter --name "/${ENVIRONMENT}/db-url" --query Parameter.Value --output text)
    sed -i "s/DB_PLACEHOLDER/${DB_URL}/g" /opt/your-app/config.yaml
    

If you need to pass custom configs while the instance is being set up or in a wait state (like before it joins an Auto Scaling group), here are reliable methods:

• CloudInit for Automated Configuration
  Most Amazon Linux and Ubuntu AMIs support CloudInit, which lets you define setup tasks in a YAML file. You can use this to write config files, set environment variables, or run commands. Example cloud-config:

  #cloud-config
  write_files:
    - path: /opt/your-app/custom-config.yaml
      content: |
        log_level: ${LOG_LEVEL}
        feature_flags:
          new_ui: ${NEW_UI_FLAG}
  runcmd:
    - export ENVIRONMENT=qa
    - LOG_LEVEL=$(aws ssm get-parameter --name "/${ENVIRONMENT}/log-level" --query Parameter.Value --output text)
    - NEW_UI_FLAG=$(aws ssm get-parameter --name "/${ENVIRONMENT}/new-ui-flag" --query Parameter.Value --output text)
    - sed -i "s/\${LOG_LEVEL}/${LOG_LEVEL}/g" /opt/your-app/custom-config.yaml
    - sed -i "s/\${NEW_UI_FLAG}/${NEW_UI_FLAG}/g" /opt/your-app/custom-config.yaml
    - systemctl start your-app
  

  Pass this cloud-config as user data when launching the instance.

• Infrastructure-as-Code (IaC) for Orchestration
  Tools like Terraform or CloudFormation let you dynamically inject configs based on environment variables. For example, in Terraform:

  resource "aws_instance" "app_server" {
    ami           = var.golden_ami_id
    instance_type = var.instance_type
  
    # Pass environment-specific variables to the user data script
    user_data = templatefile("${path.module}/user_data.sh", {
      environment = var.environment # Set to "dev", "qa", or "prod"
      config_path = "/${var.environment}/app-config"
    })
  
    # IAM profile with permissions to access SSM/Secrets Manager
    iam_instance_profile = var.app_instance_profile
  }
  

  This way, you can reuse the same Terraform module for all environments, just changing the environment variable.

• Auto Scaling Lifecycle Hooks for Wait States
  If you need to pause an instance after launch (e.g., to run custom config checks or inject secrets before it goes live), use Auto Scaling Lifecycle Hooks. Here’s how:

  1. Create a lifecycle hook that triggers when an instance enters the Launching state
  2. The hook invokes a Lambda function that fetches custom configs, applies them to the instance, and runs any validation checks
  3. Once the Lambda completes, it sends a signal to the Auto Scaling group to move the instance to the InService state

• Isolate Environment Permissions: Ensure each environment’s instance IAM role only has access to its own configs and resources—never let dev instances access prod secrets.
• Version Control Configs: Store all environment configs in Git, then sync them to SSM/Secrets Manager. This gives you audit trails and rollback capabilities.
• Test Across Environments: Every time you update your golden AMI, test it in dev first, then qa, before promoting to prod.
• Embrace Immutable Infrastructure: Never modify a running instance’s config directly. Instead, update your config store and launch new instances with the updated configs.

内容的提问来源于stack exchange，提问作者Srinivas R