基于create-react-app+react-router v3的S3托管应用无法通过Fetch as Google抓取

Hey there, let's work through this 404 issue you're facing with your React app hosted on S3 and CloudFront—especially since Google's "Fetch as Google" tool can only crawl your homepage but not inner pages. I've solved this exact problem before, so here's a step-by-step breakdown to get it sorted:

1. Configure S3 Static Website Hosting to Handle Client-Side Routes #

Since your app uses browserHistory, all requests (even for routes like /about or /dashboard) need to serve your index.html file—this is where React Router takes over and renders the correct page. S3's default behavior is to return 404 when it can't find a file matching the request path, so we need to override that:

• Go to your S3 bucket's Properties tab
• Enable Static website hosting
• Set the Index document to index.html
• Crucially, set the Error document to index.html too. This tells S3 to return your app's entry point instead of a 404 page whenever a requested path doesn't exist as a static file.

2. Update CloudFront Custom Error Responses #

CloudFront might still cache or return 404s even after S3 is configured, so we need to map 404 errors to return index.html with a 200 OK status:

• Open your CloudFront distribution's Error Pages settings
• Click Create Custom Error Response
• Fill in the details:
  • HTTP Error Code: 404 Not Found
  • Customize Error Response: Check this box
  • Response Page Path: /index.html
  • HTTP Response Code: 200 OK
  • Cache Minimum TTL: You can set this to 0 to avoid caching outdated responses (adjust based on your app's update frequency)
• Save the changes. This ensures CloudFront serves your app's entry point instead of forwarding a 404 to crawlers or users.

3. Verify React Router v3 Configuration #

Double-check your router setup to make sure browserHistory is properly implemented:

import { Router, Route, browserHistory } from 'react-router';

ReactDOM.render(
  <Router history={browserHistory}>
    {/* Your routes go here */}
    <Route path="/" component={Home} />
    <Route path="/about" component={About} />
  </Router>,
  document.getElementById('root')
);

Make sure you're not using hashHistory by mistake—browserHistory requires the server/CDN setup we just configured to work correctly.

4. Clear CloudFront Cache and Test #

Before testing, invalidate CloudFront's cached content to ensure the new settings take effect immediately:

• Go to your CloudFront distribution's Invalidations tab
• Create a new invalidation with the path /* to clear all cached files
• Once the invalidation completes, re-run Google's "Fetch as Google" tool for your inner pages. They should now render correctly instead of returning 404.

Quick Notes #

• Ensure your CloudFront origin points to your S3 bucket's static website endpoint (not the S3 object storage URL)—the static endpoint respects the index/error document settings, while the object storage URL doesn't.
• If you have API requests to your PHP backend, double-check CORS settings on your API server to avoid cross-origin issues, but this isn't related to the 404 crawler problem you're facing now.

内容的提问来源于stack exchange，提问作者mikeg542