如何在Drupal 10的Guzzle 7请求中绕过Google Cookie同意重定向

问题根源 #

Google现在会对无合规标识的请求强制跳转至Cookie同意页，curl默认的请求配置可能自带了浏览器标识或自动处理了Cookie逻辑，而Guzzle默认的请求头过于“干净”，再加上Symfony 6升级后HttpClient的重定向处理逻辑有调整，导致自动跟踪重定向无法跳过同意页。

具体解决方法 #

1. 模拟浏览器请求头，带上必要标识 #

给Guzzle请求添加模拟真实浏览器的请求头，同时开启Cookie会话，让Google识别为合法请求：

$client = \Drupal::httpClient();
$response = $client->request('GET', 'https://news.google.com/rss/articles/CBMiVWh0dHBzOi8vd3d3LmxvbmRvbi1maXJlLmdvdi51ay9pbmNpZGVudHMvMjAyMy9qYW51YXJ5L21haXNvbmV0dGUtZmlyZS1zdHJlYXRoYW0taGlsbC_SAQA?oc=5', [
  'allow_redirects' => [
    'max' => 5,
    'strict' => false,
    'referer' => true,
    'protocols' => ['http', 'https'],
    'track_redirects' => true,
  ],
  'headers' => [
    'User-Agent' => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36',
    'Accept' => 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8',
    'Accept-Language' => 'en-US,en;q=0.5',
    'Accept-Encoding' => 'gzip, deflate, br',
    'DNT' => '1',
    'Connection' => 'keep-alive',
    'Upgrade-Insecure-Requests' => '1',
  ],
  'cookies' => true,
  'verify' => true,
]);

// 获取目标内容
$content = $response->getBody()->getContents();

2. 手动处理重定向，跳过同意页 #

如果自动跟踪重定向失效，手动捕获跳转链接，判断是否为同意页后构造带参数的请求：

$client = \Drupal::httpClient();
$initialResponse = $client->request('GET', '你的Google RSS文章链接', [
  'allow_redirects' => false,
  'headers' => [
    'User-Agent' => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36',
  ],
]);

$redirectUrl = $initialResponse->getHeaderLine('Location');
if (str_contains($redirectUrl, 'consent.google.com')) {
  // 发起同意页请求，自动跳转至目标地址
  $consentResponse = $client->request('GET', $redirectUrl, [
    'headers' => [
      'User-Agent' => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36',
      'Referer' => '你的Google RSS文章链接',
    ],
    'allow_redirects' => true,
    'cookies' => true,
  ]);
  $content = $consentResponse->getBody()->getContents();
} else {
  $content = $initialResponse->getBody()->getContents();
}

3. 优化Symfony HttpClient重定向配置 #

Drupal 10依赖的Symfony 6 HttpClient在重定向处理上有变化，确保allow_redirects配置正确：

• strict设为false，允许非严格模式的重定向
• referer设为true，自动传递Referer头
• 开启cookies以保留会话信息

4. 用CookieJar持久化同意Cookie #

如果需要多次请求，创建CookieJar保存同意状态，避免每次都触发同意页：

use GuzzleHttp\Cookie\CookieJar;

$cookieJar = new CookieJar();
$client = \Drupal::httpClient();

// 先处理一次同意页，保存Cookie
$client->request('GET', '你的Google RSS文章链接', [
  'cookies' => $cookieJar,
  'headers' => [
    'User-Agent' => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36',
  ],
  'allow_redirects' => true,
]);

// 后续请求复用CookieJar，直接获取目标内容
$response = $client->request('GET', '其他Google RSS文章链接', [
  'cookies' => $cookieJar,
  'allow_redirects' => true,
]);
$content = $response->getBody()->getContents();

内容的提问来源于stack exchange，提问作者Daniel