EC2实例部署Grafana并通过IAM认证连接RDS MySQL的方法及故障排查

Hey there! Let's tackle your Grafana + RDS MySQL with IAM authentication questions step by step—first covering the deployment basics, then troubleshooting your current startup errors.

Here's a straightforward, actionable workflow to get this set up:

• Prepare IAM & RDS Foundations
  • Attach an IAM role to your EC2 instance with permission to connect to your RDS MySQL instance. The policy should look like this (replace placeholders with your AWS region, account ID, RDS resource ID, and Grafana IAM username):

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": "rds-db:connect",
                "Resource": "arn:aws:rds-db:<your-region>:<your-account-id>:dbuser:<your-rds-resource-id>/<grafana-iam-user>"
            }
        ]
    }
    

  • Enable IAM database authentication on your RDS MySQL instance (check the RDS console under "Database authentication" settings).
  • Create a Grafana-specific database and IAM-authenticated user in RDS:

    CREATE DATABASE grafana_db;
    CREATE USER 'grafana_iam_user'@'%' IDENTIFIED WITH AWSAuthenticationPlugin AS 'RDS';
    GRANT ALL PRIVILEGES ON grafana_db.* TO 'grafana_iam_user'@'%';
    FLUSH PRIVILEGES;
    

• Install & Configure Grafana on EC2
  • Install Grafana via your package manager (e.g., yum install grafana for Amazon Linux, apt install grafana for Ubuntu). Stop the service before editing configs: sudo systemctl stop grafana-server
  • Edit /etc/grafana/grafana.ini and update the [database] section (leave password blank for now—we'll handle tokens next):

    [database]
    type = mysql
    host = <your-rds-endpoint>:3306
    name = grafana_db
    user = grafana_iam_user
    password = ${GF_DATABASE_PASSWORD}
    ssl_mode = require
    

  • Create a script to generate temporary IAM auth tokens (valid for 15 mins) at /usr/local/bin/get_rds_token.sh:

    #!/bin/bash
    REGION="<your-aws-region>"
    RDS_ENDPOINT="<your-rds-endpoint>"
    USER="grafana_iam_user"
    aws rds generate-db-auth-token --hostname $RDS_ENDPOINT --port 3306 --username $USER --region $REGION
    

  • Make the script executable: sudo chmod +x /usr/local/bin/get_rds_token.sh
  • Update the Grafana systemd service (/etc/systemd/system/grafana-server.service) to generate a token on startup and pass it via environment variable:

    ExecStartPre=/bin/bash -c "export GF_DATABASE_PASSWORD=$(/usr/local/bin/get_rds_token.sh)"
    Environment="GF_DATABASE_PASSWORD=$(/usr/local/bin/get_rds_token.sh)"
    

  • Reload systemd and start Grafana: sudo systemctl daemon-reload && sudo systemctl start grafana-server

Since you already have ELK/Grafana deployed and your IAM user can access RDS, let's narrow down the issue:

• First, Validate Your grafana.ini Config
  • Double-check the [database] section: ensure type = mysql, host includes the full RDS endpoint + port 3306, name matches your Grafana DB, and user is your IAM-authenticated RDS user. Do NOT set a static password here—IAM auth uses temporary tokens, not fixed passwords.
• Test the IAM Connection Manually
  • On your EC2 instance, run this command to generate a token and test MySQL access (replace placeholders):

    mysql -h <your-rds-endpoint> -P 3306 -u <grafana-iam-user> --password=$(aws rds generate-db-auth-token --hostname <your-rds-endpoint> --port 3306 --username <grafana-iam-user> --region <your-region>) --ssl-mode=REQUIRED
    

  • If this works, your IAM/RDS setup is solid—problem lies with Grafana's token integration. If it fails, fix the IAM permissions or RDS user config first.
• Check Grafana Logs for Specific Errors
  • Look at /var/log/grafana/grafana.log for exact error messages:
    • Access denied for user 'grafana_iam_user'@...: Likely a token issue (not being passed correctly) or incorrect username/DB name.
    • Unable to connect to database: Verify EC2 security group allows outbound 3306 to RDS, and RDS security group allows inbound 3306 from your EC2 instance's security group/IP.
• Fix the Token Integration
  • Use the systemd setup from the first section to auto-generate and pass the token to Grafana via environment variable. Hardcoding tokens won't work (they expire every 15 mins), and leaving password blank won't trigger IAM auth automatically.
• Common Mistakes to Avoid
  • Forgetting to enable IAM database authentication on your RDS instance (this is a required toggle in the RDS console).
  • Using the wrong resource ARN in your EC2 IAM policy—get the correct DbiResourceId via aws rds describe-db-instances --db-instance-identifier <your-rds-name> --query 'DBInstances[0].DbiResourceId'.

内容的提问来源于stack exchange，提问作者Necoras