如何用BeautifulSoup获取ID标签后的多个段落

我需要用BeautifulSoup获取维基百科页面中特定标题（Tourism）后的所有段落内容，目前只能正常获取第一段，尝试获取后续段落时要么报错要么返回空字符串。

目标HTML结构（取自维基百科拉斯维加斯页面的Tourism标题区域） #

<h3>
    <span id="Tourism" class="mw-headline">Tourism</span>
</h3>
<div class="thumb tright">....</div>
<div class="thumb tright">....</div>
<div class="thumb tright">....</div>
<div class="thumb tright">....</div>
<p>Text I can get</p>
<p>Text that I cant get</p>
<h4>....</h4>

初始代码 #

这段代码仅能获取第一段内容：

from flask import Flask, jsonify, request
from flask_restful import Resource, Api
import requests
from bs4 import BeautifulSoup

app = Flask(__name__)
api = Api(app)

class WikiScrape(Resource):
    def get(self, location):
        tour_page = requests.get(
            url = f"https://en.wikipedia.org/wiki/{location}"
        )
        soup = BeautifulSoup(tour_page.content, 'html.parser')
        title = soup.find(id = "Tourism")
        para = title.find_next('p').get_text(strip = True)

api.add_resource(WikiScrape, '/<location>')

if __name__ == '__main__':
    app.run(debug = True)

尝试过的方法及问题 #

• 使用title.find_all获取ID标签后的所有段落，返回空列表；遍历列表时仅得到空字符串
• 修改代码用title.findAllNext('p')获取，会返回大量无关段落内容：

soup = BeautifulSoup(tour_page.content, 'html.parser')
title = soup.find('span',{'id': 'Tourism'})
paras = [x.contents[0] for x in title.findAllNext('p')]
return jsonify({'message': f'{title.string}: {paras}'})

最终可用代码 #

经帮助后得到的有效代码，可获取Tourism标题后直到下一个h4标签前的所有段落内容：

from flask import Flask, jsonify, request
from flask_restful import Resource, Api
import requests
from bs4 import BeautifulSoup

app = Flask(__name__)
api = Api(app)

class WikiScrape(Resource):
    def get(self, location):
        tour_page = requests.get(
            url = f"https://en.wikipedia.org/wiki/{location}"
        )
        
        soup = BeautifulSoup(tour_page.content, 'html.parser')
        title = soup.find('h3', text='Tourism')
        paras = ''
        
        for sibling in title.find_next_siblings():
            if sibling.name == 'p':
                paras = paras + sibling.text.strip()
            elif sibling.name == 'h4':
                break

        return jsonify({'message': f'{paras}'})

api.add_resource(WikiScrape, '/<location>')

if __name__ == '__main__':
    app.run(debug = True)

内容的提问来源于stack exchange，提问作者Bran The Builder