使用Ruby RSS gem解析RSS feed时遭遇400 Bad Request错误求助

架构老林

2026-8-2

解决Ruby解析phys.org RSS Feed时的400错误

问题诊断

400 Bad Request错误大概率是因为请求头不符合服务器要求。Ruby的URI.open默认使用的User-Agent标识较为简单，被phys.org的服务器拦截了——很多网站会通过检查User-Agent来区分合法浏览器请求和自动化请求。

解决方案

1. 给请求添加自定义User-Agent

修改URI.open调用，通过headers参数指定一个更贴近浏览器的User-Agent：

require 'rss'
require 'uri'

url = 'https://phys.org/rss-feed/biology-news/biology-other/'
URI.open(url, 'User-Agent' => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36') do |rss|
  feed = RSS::Parser.parse(rss)
  puts "Title: #{feed.channel.title}"
  feed.items.each do |item|
    puts "Item: #{item.title}"
  end
end

2. 用Net::HTTP精细控制请求

如果URI.open仍有问题，可直接用Net::HTTP构建请求，更灵活配置请求头：

require 'rss'
require 'net/http'
require 'uri'

uri = URI('https://phys.org/rss-feed/biology-news/biology-other/')
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true

request = Net::HTTP::Get.new(uri)
request['User-Agent'] = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36'

response = http.request(request)
if response.is_a?(Net::HTTPSuccess)
  feed = RSS::Parser.parse(response.body)
  puts "Title: #{feed.channel.title}"
  feed.items.each do |item|
    puts "Item: #{item.title}"
  end
else
  puts "Request failed: #{response.code} #{response.message}"
end