Python网络爬虫无法正常请求输入URL，如何解决？

问题描述 #

跟着基础教程制作Python网络爬虫，已安装requests和bs4依赖。运行时终端不打印「Enter the URL of the blog: 」提示，输入URL也无法返回爬取的h2文本，VSCode和终端均无报错信息。

运行日志：

PS C:\Users\delma\Python\Web Scraper>c:; cd 'c:\Users\delma\Python\Web Scraper'; & 'C:\Users\delma\AppData\Local\Programs\Python\Python312\python.exe' 'c:\Users\delma\.vscode\extensions\ms-python.python-2023.20.0\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\launcher' '56485' '--' 'C:\Users\delma\Python\Web Scraper\webscraper.py'
PS C:\Users\delma\Python\Web Scraper> 

使用的代码：

import requests 
from bs4 import BeautifulSoup
def scrape_blog(url):
    try:
        response = requests.get(url)
        response.raise_for_status()
    except requests.exceptions.RequestException as e:
        print(f"Failed to retrieve the page: {e}")
        return
    soup = BeautifulSoup(response.text, 'html.parser')
    articles = soup.find_all('h2')
    if articles:
        for article in articles:
            print(article.get_text())
        else:
            print("No article titles found on this page")
    if __name__ == "__main__":
        url = input("Enter the URL of the blog: ")
        scrape_blog(url)

代码与教程一致，请问哪里出错了？

问题根源 #

代码存在两处缩进错误：

• if __name__ == "__main__": 代码块被缩进在scrape_blog函数内部，导致程序没有顶层执行入口，运行时不会触发输入提示和爬虫调用。
• if articles:块内的else语句缩进错误，它被绑定到了for循环（Python中for支持else分支，循环正常结束时执行），而非if分支，逻辑不符合需求。

修正后的代码 #

import requests 
from bs4 import BeautifulSoup

def scrape_blog(url):
    try:
        response = requests.get(url)
        response.raise_for_status()
    except requests.exceptions.RequestException as e:
        print(f"Failed to retrieve the page: {e}")
        return
    
    soup = BeautifulSoup(response.text, 'html.parser')
    articles = soup.find_all('h2')
    
    if articles:
        for article in articles:
            print(article.get_text())
    else:
        print("No article titles found on this page")

if __name__ == "__main__":
    url = input("Enter the URL of the blog: ")
    scrape_blog(url)

修正说明 #

1. 将if __name__ == "__main__":代码块移到scrape_blog函数外部，作为程序的顶层执行入口，运行脚本时会自动触发输入提示并调用爬虫函数。
2. 调整else语句的缩进，使其属于if articles:的分支，当页面未找到h2标签时，正确打印提示信息。

内容的提问来源于stack exchange，提问作者Mike Del