Let's break down the most likely causes and actionable fixes for why Google isn't picking up your prerendered pages, even though Facebook and Twitter are working correctly:

1. Verify Prerender's User-Agent Routing Rules #

First, double-check that your ASP.NET MVC server's routing (likely IIS URL Rewrite) includes Googlebot-related user agents in the rules that forward requests to Prerender.io. It’s easy to accidentally only list social bots initially.

Make sure your rewrite rules target these UA strings:

• Googlebot
• Googlebot-Image
• Googlebot-Mobile
• Googlebot-News
• Googlebot-Video

For example, in your IIS web.config, the rule's condition should include these patterns alongside Facebook/Twitter bots to ensure Googlebot gets routed to Prerender.

2. Check Prerender.io's Access Logs #

Log into your Prerender.io dashboard and review the request logs:

• If you don’t see any entries from Googlebot, this means requests from Google aren’t reaching Prerender at all—pointing to a server-side routing misconfiguration.
• If you do see Googlebot entries, check response status codes:
  • 200: Prerender returned a rendered page (good, but keep investigating other angles)
  • 404/500: Prerender couldn’t fetch or render your page (debug the underlying error, like broken resource links or timeouts)

3. Test Googlebot's View of Your Pages #

Use Google Search Console’s URL Inspection Tool to submit a page for crawling. After the crawl finishes, check the "Rendered Content" tab. If it shows only your AngularJS app’s root element (no actual page content), Prerender isn’t serving the pre-rendered page to Googlebot.

You can also simulate a Googlebot request manually with curl:

curl -A "Googlebot/2.1 (+http://www.google.com/bot.html)" https://yourdomain.com/your-target-page

If the response is your raw Angular index.html (not the fully rendered content), your server isn’t forwarding the request to Prerender correctly.

4. Ensure AngularJS Notifies Prerender When Content Is Ready #

AngularJS’s async data loading can cause Prerender to capture the page before content is fully loaded. Fix this by explicitly telling Prerender when your page is ready:

Add this snippet to your Angular controllers/components once all async data is loaded:

angular.module('yourApp').controller('YourController', function($scope, $http, $window) {
  $http.get('/api/your-data').then(function(response) {
    $scope.content = response.data;
    // Notify Prerender the page is fully rendered
    if (typeof $window.prerenderReady !== 'undefined') {
      $window.prerenderReady();
    }
  });
});

This prevents Prerender from returning a partially empty page to Googlebot.

5. Validate Robots.txt & Crawler Permissions #

• Make sure your robots.txt isn’t blocking Googlebot or Prerender.io’s IP addresses. Prerender needs to crawl your site to generate pre-rendered pages, so its IP range must be allowed.
• Avoid Disallow: / or specific rules that block Googlebot from accessing your core pages.

6. Fix HTML5 Mode Routing (If Enabled) #

If you’re using AngularJS’s HTML5 mode, ensure your ASP.NET MVC routes include a catch-all fallback to index.html for non-API requests. This ensures Prerender can access the correct entry point for your Angular app.

In your RouteConfig.cs, add:

routes.MapRoute(
    name: "AngularFallback",
    url: "{*url}",
    defaults: new { controller = "Home", action = "Index" }
);

This ensures any non-API URL routes to your Angular app’s index page, which Prerender can then render.

7. Check for HTTPS/SSL Issues #

If your site uses HTTPS:

• Confirm Prerender.io is configured to use HTTPS (your dashboard should show HTTPS endpoints).
• Ensure your server’s HTTP-to-HTTPS redirects don’t break Prerender’s requests. Test that Prerender can fetch your HTTPS pages without certificate errors.

8. Review Google Search Console for Crawl Errors #

Check the Crawl Errors section in Google Search Console for 404s, 500s, or other issues. If Googlebot encounters server errors when crawling your pages, it won’t index them—even if Prerender is set up correctly.

内容的提问来源于stack exchange，提问作者Mohammed Gadi