多线程拉取JIRA数据时Cookie认证重复登录问题求助

业务场景与实现 #

我需要从JIRA拉取大量数据，为提升效率采用多线程方式拉取，核心代码如下：

import concurrent.futures as fs

session = Session(user, password)
total_num_issues = send_request(url, session,
                                    params={**params, **{
                                      "maxResults": "1"
                                    }})["total"]
url = "https://jira_link_for_my_company.com/rest/api/2/search?jql=project%3Dsome_project"
page_size = 100
future_list = []
with fs.ThreadPoolExecutor(max_workers=30) as ex:
    for start_at in range(0, total_num_issues + 1, page_size):
        extra_params = {"startAt": start_at, "maxResults": page_size}
        future_list.append(ex.submit(send_request, url, session, {**params, **extra_params}))
    for future in fs.as_completed(future_list):
        issues += future.result()["issues"]

JIRA拉取数据需要登录，为避免每次请求都重复登录，我基于Cookie认证实现了复用逻辑：

• 登录一次获取Cookie，有效期5分钟；
• Cookie未过期时无需重新登录。

对应的请求函数和Session类代码如下：

def send_request(url, session, params={}):
  headers = {
  "cookie" : session.get_cookie(),
  "Content-Type" : "application/json"
  }
  for attr_item in params.items():
  # attribute item is a tuple of name and value pair of the attribute
      url = f"{url}&{attr_item[0]}={attr_item[1]}"
  response = requests.get(url=url, headers=headers)
  return json.loads(response.text)

import time
import base64
import requests
import json

class Session:
  def __init__(self, user, password):
    self.user = user
    self.password = password
    # session_url是JIRA的登录接口，用于获取会话Cookie
    self.session_url = "https://jira_link_for_my_company.com/rest/auth/1/session"
    # Cookie有效期300秒（5分钟）
    self.session_time = 300 
    self.cookie_time = None
    self.cookie = None
    
    self.set_session()
  
  def set_session(self):
    auth_string = base64.b64encode(str.encode(f"{self.user}:{self.password}"))
    headers = {
    "Authorization" : "Basic %s" % str(auth_string)[2:-1],
    "Content-Type" : "application/json"
    }
    response = requests.get(url=self.session_url, headers=headers)
    self.cookie = response.headers['Set-Cookie'].split(';')[0]
    self.cookie_time = time.time()
  
  def get_cookie(self):
    if time.time() - self.cookie_time > self.session_time:
      self.set_session()
    return self.cookie

问题现象与推测 #

原本预期长耗时任务（比如7分钟）仅需2次登录（首次获取+5分钟后更新），但实际登录次数远超预期——30个线程会各自触发重新登录，导致登录次数达到30次。

最初用全局变量存储Cookie和生成时间未解决问题，改用Session类后，线程仍无法感知其他线程已更新的Cookie和cookie_time，每个线程检测到Cookie过期时都会自行调用set_session重新登录。

内容的提问来源于stack exchange，提问作者Saguro