如何通过Django/Flask将Python爬取的JEE试题展示到HTML页面？

步骤1：重构爬取代码为结构化数据函数 #

把原终端打印的代码改成返回字典列表的函数，方便传递到网页模板：

import requests
from bs4 import BeautifulSoup

def fetch_jee_questions():
    webpage_codes = ['lgnz9975', 'lgnzeywd', 'lgnzj56q', 'lgnzt580', 'lgnw7ha3', 'lgnwmpwc','lgnx2ht3', 'lgnx23k6', 'lgny1m5w', 'lgny7rsq']
    questions_data = []
    question_number = 1
    base_url = "https://questions.examside.com/past-years/year-wise/jee/jee-main/jee-main-2023-online-15th-april-morning-shift/"
    
    for code in webpage_codes:
        r = requests.get(base_url + code)
        soup = BeautifulSoup(r.content, 'html.parser')
        s = soup.find('div', class_="flex-grow flex flex-col gap-2 lg:gap-4")
        result_set = s.find_all('div')
        
        questions = []
        options = []
        
        # 提取题目文本
        for h in result_set:
            if h.get('class') == ['question', 'xl:text-lg', 'px-4', 'py-2.5']:
                questions.append(h.text.replace("$", "$$"))
        
        # 提取选项文本
        for a in result_set:
            if a.get('class') == ['grow', 'question', 'xl:text-lg']:
                options.append(a.text.replace("$", "$$"))
        
        w = 0
        for q_text in questions:
            if options[w] == '':
                w +=4
                continue
            # 组装单题数据
            question = {
                'number': question_number,
                'text': q_text,
                'options': {
                    'A': options[w],
                    'B': options[w+1],
                    'C': options[w+2],
                    'D': options[w+3]
                }
            }
            questions_data.append(question)
            question_number +=1
            w +=4
    
    return {
        'exam_title': "JEE MAINS 15th Morning Session",
        'difficulty': "Medium",
        'questions': questions_data
    }

步骤2：搭建Flask项目 #

1. 安装Flask：pip install flask
2. 项目结构：

jee-questions/
├── app.py
├── scraper.py  # 存放上面的爬取函数
└── templates/
    └── questions.html

3. 编写app.py：

from flask import Flask, render_template
from scraper import fetch_jee_questions

app = Flask(__name__)

@app.route('/')
def show_questions():
    exam_data = fetch_jee_questions()
    return render_template('questions.html', data=exam_data)

if __name__ == '__main__':
    app.run(debug=True)

4. 编写templates/questions.html模板：

<!DOCTYPE html>
<html>
<head>
    <title>JEE MAINS 真题</title>
    <style>
        .question-container {
            margin: 20px auto;
            padding: 15px;
            width: 85%;
            border: 1px solid #eee;
            border-radius: 8px;
            box-shadow: 0 2px 4px rgba(0,0,0,0.1);
        }
        .question-number {
            font-weight: bold;
            font-size: 1.2em;
            color: #2c3e50;
        }
        .question-text {
            margin: 10px 0;
            font-size: 1.1em;
            line-height: 1.5;
        }
        .option {
            margin: 6px 0;
            padding-left: 8px;
        }
        .exam-header {
            text-align: center;
            margin: 30px 0;
            color: #34495e;
        }
    </style>
</head>
<body>
    <div class="exam-header">
        <h1>{{ data.exam_title }}</h1>
        <p>难度：{{ data.difficulty }}</p>
    </div>
    
    <div class="questions-list">
        {% for question in data.questions %}
        <div class="question-container">
            <div class="question-number">{{ question.number }}.</div>
            <div class="question-text">{{ question.text }}</div>
            <div class="options">
                <div class="option">A. {{ question.options.A }}</div>
                <div class="option">B. {{ question.options.B }}</div>
                <div class="option">C. {{ question.options.C }}</div>
                <div class="option">D. {{ question.options.D }}</div>
            </div>
        </div>
        {% endfor %}
    </div>
</body>
</html>

步骤1：复用爬取函数 #

将上面的fetch_jee_questions函数保存为questions/scraper.py。

步骤2：搭建Django项目 #

1. 安装Django：pip install django
2. 创建项目和应用：

django-admin startproject jee_exam
cd jee_exam
python manage.py startapp questions

3. 在jee_exam/settings.py中注册应用：

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'questions',  # 添加这行
]

4. 编写questions/views.py：

from django.shortcuts import render
from .scraper import fetch_jee_questions

def question_list(request):
    exam_data = fetch_jee_questions()
    return render(request, 'questions/question_list.html', {'data': exam_data})

5. 配置URL路由：
   在jee_exam/urls.py中添加：

from django.contrib import admin
from django.urls import path
from questions.views import question_list

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', question_list, name='question_list'),
]

6. 创建模板文件templates/questions/question_list.html：

<!DOCTYPE html>
<html>
<head>
    <title>JEE MAINS 真题</title>
    <style>
        .question-container {
            margin: 20px auto;
            padding: 15px;
            width: 85%;
            border: 1px solid #eee;
            border-radius: 8px;
            box-shadow: 0 2px 4px rgba(0,0,0,0.1);
        }
        .question-number {
            font-weight: bold;
            font-size: 1.2em;
            color: #2c3e50;
        }
        .question-text {
            margin: 10px 0;
            font-size: 1.1em;
            line-height: 1.5;
        }
        .option {
            margin: 6px 0;
            padding-left: 8px;
        }
        .exam-header {
            text-align: center;
            margin: 30px 0;
            color: #34495e;
        }
    </style>
</head>
<body>
    <div class="exam-header">
        <h1>{{ data.exam_title }}</h1>
        <p>难度：{{ data.difficulty }}</p>
    </div>
    
    <div class="questions-list">
        {% for question in data.questions %}
        <div class="question-container">
            <div class="question-number">{{ question.number }}.</div>
            <div class="question-text">{{ question.text }}</div>
            <div class="options">
                <div class="option">A. {{ question.options.A }}</div>
                <div class="option">B. {{ question.options.B }}</div>
                <div class="option">C. {{ question.options.C }}</div>
                <div class="option">D. {{ question.options.D }}</div>
            </div>
        </div>
        {% endfor %}
    </div>
</body>
</html>

7. 运行项目：

python manage.py runserver

优化建议 #

• 爬取时添加请求头和延迟，避免被封禁：
  在requests.get中加入请求头：

  headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36'}
  r = requests.get(base_url + code, headers=headers)
  

  导入time模块，在循环内添加time.sleep(1)控制爬取频率。
• 建议将爬取数据存入数据库（如SQLite），避免每次请求重复爬取，提升页面加载速度。

内容的提问来源于stack exchange，提问作者Prajeeth Muthukumar