开发Web应用：S3与块存储的文件存储方案选型咨询

Great question—let’s break this down step by step since you’re building a web app with user file uploads, which is a super common (and important) use case!

S3 vs. Block Storage: Which Fits Your Needs? #

S3 is almost certainly the better choice for your scenario, and here’s why:

• Block Storage: Think of this as a virtual hard drive—you have to mount it to a server (like an EC2 instance) and interact with it via a traditional file system. It shines for workloads that need low-latency random access (e.g., databases, virtual machine disks, or apps that rely on direct file system APIs). But for user-uploaded files, it’s overkill: you’d have to handle server scaling, storage capacity planning, and file access routing manually, adding unnecessary complexity.
• S3 (Object Storage): Built specifically for storing unstructured data (exactly the PDFs, images, and spreadsheets you’re dealing with) over HTTP. It’s fully managed—AWS handles redundancy, global availability, and scaling behind the scenes, so you don’t have to worry about server maintenance. And as you noticed, it’s significantly cheaper than block storage for most file storage scenarios, especially if you use tiered storage options (like S3 Standard-Infrequent Access or Glacier) for files that aren’t accessed often.

Is S3 Easy to Use Programmatically? #

Absolutely—this is one of S3’s biggest strengths. Every major programming language has an official AWS SDK that simplifies uploads, downloads, and file management to just a few lines of code.

For example, here’s a quick snippet using Python’s boto3 SDK to upload a file to S3 with a customer-specific prefix:

import boto3

s3 = boto3.client('s3')
# Upload a local file to your bucket, organized under a customer's unique path
s3.upload_file('invoice.pdf', 'your-app-file-bucket', 'customer-456/invoices/invoice.pdf')

You can also generate pre-signed URLs to let users upload or download files directly from S3 without going through your web server. This reduces load on your backend and speeds up transfers, and most SDKs have built-in methods to create these URLs in seconds.

When Would Block Storage Be a Better Pick? #

Block storage makes sense only if your app has specific requirements S3 can’t meet, like:

• You need to run a database that requires direct file system access (e.g., PostgreSQL hosted on EC2)
• Your app relies on legacy file system APIs that don’t work with object storage
• You need ultra-low-latency random read/write operations for frequently modified files

But for user-uploaded content that’s mostly read-after-write or stored for later access, S3 is the right fit.

How to Ensure Customers Only Access Their Own Files #

This is straightforward with S3’s access control tools. The most scalable approach is:

1. Use a prefix-based structure: Store each customer’s files under a unique prefix (e.g., customer-{user-id}/). So customer 456’s files live in s3://your-app-file-bucket/customer-456/.
2. Restrict access with IAM policies: If your backend uses an IAM role, create a policy that only allows access to objects matching the current customer’s prefix. For example:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": ["s3:GetObject", "s3:PutObject"],
         "Resource": "arn:aws:s3:::your-app-file-bucket/customer-${customer-id}/*"
       }
     ]
   }
   

3. Limit pre-signed URLs to customer prefixes: When generating pre-signed URLs for frontend uploads/downloads, only generate them for objects under the customer’s specific path. This ensures users can’t access files belonging to others.

Creating a separate bucket per customer is an option, but AWS limits the number of buckets per account (default 100), so the prefix approach is far more scalable for large user bases.

内容的提问来源于stack exchange，提问作者Sergey Terehin