Elastic Beanstalk单EC2实例日志占满磁盘致应用崩溃求助

Hey there, let's tackle this log disk space issue head-on—super common with apps that generate heavy logs, so you're not alone! Here are practical, actionable solutions to prevent your EC2 instances from crashing due to filled disks:

1. Set Up Log Rotation with logrotate (Local Disk Management) #

Elastic Beanstalk supports custom configuration via .ebextensions to enforce log rotation, which will automatically archive and delete old logs before they fill the disk.

Create a file named .ebextensions/logs.config in your application root directory with the following content (adjust paths to match your app's log location):

files:
  "/etc/logrotate.d/app-logs":
    mode: "000644"
    owner: root
    group: root
    content: |
      /var/app/current/logs/*.log {
          daily
          missingok
          rotate 7
          compress
          delaycompress
          notifempty
          copytruncate
      }

commands:
  run-logrotate:
    command: "/usr/sbin/logrotate /etc/logrotate.d/app-logs"
    ignoreErrors: true

• daily: Rotates logs every day
• rotate 7: Keeps up to 7 days of archived logs
• compress: Compresses old logs to save space
• copytruncate: Lets your app keep writing to the log file while rotation happens (avoids breaking log streams)

Deploy this updated app to Elastic Beanstalk, and the log rotation will kick in automatically.

2. Push Logs to CloudWatch Logs (Offload Storage) #

Instead of storing all logs locally, send them to AWS CloudWatch Logs. This offloads the disk usage entirely and gives you centralized log management, search, and retention policies.

Option A: Configure via Elastic Beanstalk Console #

1. Go to your Elastic Beanstalk environment
2. Navigate to Configuration > Software
3. Under Logging, enable Stream logs to CloudWatch
4. Specify a log group name, and set the retention period (e.g., 30 days)
5. Save the configuration—Elastic Beanstalk will set up the necessary IAM permissions and log streaming.

Option B: Use .ebextensions for Code-Based Configuration #

Add this to your .ebextensions/cloudwatch-logs.config:

option_settings:
  aws:elasticbeanstalk:cloudwatch:logs:
    StreamLogs: true
    DeleteOnTerminate: false
    RetentionInDays: 30
  aws:elasticbeanstalk:cloudwatch:logs:health:
    RotationCount: 10
    RotationSize: 100M

This enables streaming and sets a 30-day retention policy for logs in CloudWatch.

3. Increase the EC2 Instance Root Volume Size #

If you need to keep more logs locally (even with rotation), increasing the root disk size gives you more breathing room.

How to Adjust: #

1. In the Elastic Beanstalk console, go to Configuration > Instances
2. Under Instance type, select an instance type with a larger default root volume (or)
3. Use an .ebextensions file to resize the root volume during instance launch:

commands:
  01_resize_root:
    command: |
      # Resize the root volume to 50GB (adjust as needed)
      growpart /dev/xvda 1
      resize2fs /dev/xvda1

Note: This works for Linux instances; adjust device names if your instance uses a different block device (e.g., /dev/nvme0n1 for newer instances).

4. Optimize Log Output at the Application Level #

Sometimes the simplest fix is to reduce the amount of logs your app generates:

• Lower the log level: Switch from DEBUG to INFO or WARN unless you need verbose debugging logs
• Remove redundant log statements: Audit your code to eliminate duplicate or unnecessary log entries
• Use structured logging: Tools like JSON logging can help reduce bloat and make logs easier to parse without extra noise

Any combination of these solutions will prevent your instances from hitting disk full errors and crashing. Start with log rotation + CloudWatch streaming for the most robust long-term fix!

内容的提问来源于stack exchange，提问作者Mateusz Kubuszok