MLFlow访问S3模型制品时出现InvalidToken错误求助
I'm trying to start an MLFlow server in my local Python virtual environment using this command:
mlflow server --backend-store-uri postgresql://mlflow_user:mlflow@localhost/mlflow --artifacts-destination S3://<S3 bucket name>/mlflow/ --serve-artifacts -h 0.0.0.0 -p 8000
I've exported the following environment variables in the activated virtual environment:
export AWS_ACCESS_KEY_ID=<access key> export AWS_SECRET_ACCESS_KEY=<secret key> export DEFAULT_REGION_NAME=<region name> export DEFAULT_OUTPUT_FORMAT=<output format>
But when MLFlow tries to access model artifacts for any runs, I get this error:
botocore.exceptions.ClientError: An error occurred (InvalidToken) when calling the ListObjectsV2 operation: The provided token is malformed or otherwise invalid.
What can I do to fix this?
Possible Fixes for the InvalidToken Error
This error almost always ties back to AWS credential formatting, configuration mismatches, or permission issues. Here are the key steps to troubleshoot and resolve it:
Verify AWS credential validity and formatting
Double-check that yourAWS_ACCESS_KEY_IDandAWS_SECRET_ACCESS_KEYhave no extra spaces, line breaks, or special characters. It's common to accidentally copy trailing whitespace when pasting credentials from the AWS console. Run these commands in your terminal to confirm the values are clean:echo $AWS_ACCESS_KEY_ID echo $AWS_SECRET_ACCESS_KEYIf you see any unexpected characters, re-export the variables with the correct, trimmed values.
Fix the AWS region environment variable name
MLFlow relies on standard AWS environment variables, and you're using an incorrect name for the region. ReplaceDEFAULT_REGION_NAMEwith the standardAWS_DEFAULT_REGIONvariable. This is a frequent oversight that can cause credential parsing issues. Update your export command to:export AWS_DEFAULT_REGION=<your bucket's region>Also ensure the region value exactly matches your S3 bucket's region (e.g.,
us-west-2instead ofuswest2or a misspelled variant). You can confirm the bucket's region via the AWS S3 console.Validate IAM permissions for the S3 bucket
Make sure the IAM user associated with your AWS credentials has the necessary permissions to interact with the bucket. At minimum, the user needs these permissions:s3:ListBucket(to list artifacts in the bucket)s3:GetObject(to retrieve artifacts)s3:PutObject(to upload artifacts)
Test if the credentials work outside MLFlow by running this AWS CLI command:
aws s3 ls s3://<your bucket name>/mlflow/If this command throws the same error, your IAM policy or credentials are the root cause.
Check S3 URI formatting
Ensure your--artifacts-destinationURI is correctly formatted. While MLFlow accepts bothS3://ands3://, confirm the bucket name is spelled correctly (AWS bucket names are case-insensitive but typos will break access). Also, avoid special characters in the bucket name that might cause parsing issues.Reload environment variables and restart the server
Sometimes environment variables don't propagate correctly to the MLFlow process. Exit your virtual environment, reactivate it, re-export the corrected variables, then restart the MLFlow server. This ensures the server picks up the latest configuration.
内容的提问来源于stack exchange,提问作者Deepak Tatyaji Ahire

