Ubuntu EC2上用Java 9.0.4启动AWS Data Pipeline任务运行器遇log4j报错求助

Hey there, let's break down this issue and get your TaskRunner up and running smoothly. That log4j warning is simply telling you: log4j knows it needs to log messages, but it doesn't have instructions for where to send those logs—like the console, a local file, or your specified S3 bucket. Let's fix this for your Ubuntu EC2 setup with Java 9.0.4.

Step 1: Check for an existing log4j config file #

First, see if you already have a log4j.properties or log4j.xml file lurking somewhere. TaskRunner expects this config to be on its classpath or in its working directory. Run this command to search:

find ~/taskrunner -name "log4j.*"

If it finds a file, skip to Step 3 to adjust it. If not, move to Step 2 to create a new one.

Step 2: Create a basic log4j.properties file #

Make a log4j.properties file in your ~/taskrunner directory (or wherever your TaskRunner jar lives) with this content. This sets up console logging and local file logging—we can add S3 support too if you want:

# Root logger setup (sets default log level and appenders)
log4j.rootLogger=INFO, console, file

# Console appender (logs to your EC2 terminal)
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.Target=System.out
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n

# Local file appender (logs to a file on EC2)
log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.File=/var/log/taskrunner.log
log4j.appender.file.MaxFileSize=10MB
log4j.appender.file.MaxBackupIndex=5
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n

If you want to send logs directly to your S3 bucket (matching your --logUri flag), add this S3-specific appender section (TaskRunner should already have the AWS SDK jars needed for this):

# S3 appender (logs directly to your bucket)
log4j.appender.s3=com.amazonaws.services.log4j.AWSAppender
log4j.appender.s3.layout=org.apache.log4j.PatternLayout
log4j.appender.s3.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n
log4j.appender.s3.bucket=<my bucket>
log4j.appender.s3.keyPrefix=logs_datapipeline/
log4j.appender.s3.region=us-east-1

Then update the root logger line to include the S3 appender:

log4j.rootLogger=INFO, console, file, s3

Step 3: Make sure the config is accessible to Java #

You have two easy ways to ensure Java picks up the log4j config:

• Drop log4j.properties in the same folder as TaskRunner-1.0.jar—Java will automatically detect it.
• Or explicitly specify the config path with a JVM argument when starting the jar:

java -Dlog4j.configuration=file:/home/ubuntu/taskrunner/log4j.properties -jar TaskRunner-1.0.jar --config ~/taskrunner/credentials.json --workerGroup=<group name> --region=us-east-1 --logUri=s3://<my bucket>/logs_datapipeline

Just replace /home/ubuntu/taskrunner/log4j.properties with your actual file path.

Step 4: Double-check IAM permissions (for S3 logging) #

If you're using the S3 appender, your EC2 instance's IAM role needs permission to write to the bucket. Add a policy to the role that allows s3:PutObject on arn:aws:s3:::<my bucket>/logs_datapipeline/*.

Step 5: Test the fix #

Restart TaskRunner with your updated command. The log4j warning should vanish, and you'll start seeing logs in your chosen destination—console, local file, or S3 bucket.

内容的提问来源于stack exchange，提问作者Andres Urrego Angel