将mitmdump捕获数据推送至Azure Blob Storage时遇连接错误求助

问题场景 #

尝试用mitmdump捕获HTTP请求/响应并推送至Azure Blob Storage，运行脚本时出现以下连接错误：

'in script .\log_azureblob.py: ('Cannot connect to proxy.', NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x00000256D5BF62D0>: Failed to establish a new connection: [WinError 10061] 由于目标计算机积极拒绝，无法连接。'))'

原实现代码：

from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient, BlobType
from mitmproxy import http, flowfilter,ctx

blob_service_client = BlobServiceClient.from_connection_string(connection_string)

blob_client = blob_service_client.get_blob_client(container=container_name, blob=blob_name)

if not blob_client.exists():
# if the blob doesn't exist, create it with some initial data
initial_data = b'Initial data for the blob'
blob_client.upload_blob(initial_data)

def start():
# set up the logger
ctx.log.info("BlobLogger started.")

def request(flow):
# get the request data and append it to the blob
request_data = f"{flow.request.method} {flow.request.url} HTTP/1.1\n"
for name, value in flow.request.headers.items():
request_data += f"{name}: {value}\n"
request_data += "\n" # add a blank line between the headers and body
if flow.request.content:
request_data += flow.request.content.decode("utf-8") + "\n"

try:
    blob_client.append_block(request_data.encode("utf-8"))
except Exception as e:
    ctx.log.error(f"Error appending request data to blob: {e}")
def response(flow):
# get the response data and append it to the blob
response_data = f"HTTP/1.1 {flow.response.status_code} {flow.response.reason}\n"
for name, value in flow.response.headers.items():
response_data += f"{name}: {value}\n"
response_data += "\n" # add a blank line between the headers and body
if flow.response.content:
response_data += flow.response.content.decode("utf-8") + "\n"

try:
    blob_client.append_block(response_data.encode("utf-8"))
except Exception as e:
    ctx.log.error(f"Error appending response data to blob: {e}")
def done():
# clean up the logger
ctx.log.info("BlobLogger stopped.")

排查与解决步骤 #

1. 解决代理冲突问题 #

mitmdump会启动本地代理服务，Azure SDK默认会读取系统代理配置，导致尝试通过mitmdump代理连接Azure服务，而mitmdump不转发该请求，引发拒绝连接。

修复方式：初始化BlobServiceClient时强制禁用代理：

import os
# 全局禁用代理
os.environ['NO_PROXY'] = '*'

# 或创建客户端时显式设置代理为None
blob_service_client = BlobServiceClient.from_connection_string(
    connection_string,
    proxy_options=None
)

2. 调整脚本初始化时机 #

原代码在模块加载阶段就执行Azure客户端初始化和Blob检查操作，此时mitmdump环境尚未完全就绪，容易引发网络请求异常。

修复方式：将Azure相关初始化逻辑移至start()函数中：

from azure.storage.blob import BlobServiceClient, BlobClient
from mitmproxy import http, ctx
import os

# 全局变量声明
blob_client = None
connection_string = "你的Azure存储账户连接字符串"
container_name = "目标容器名称"
blob_name = "存储日志的Blob文件名"

def start():
    global blob_client
    # 禁用代理
    os.environ['NO_PROXY'] = '*'
    # 初始化Blob服务客户端
    blob_service_client = BlobServiceClient.from_connection_string(
        connection_string,
        proxy_options=None
    )
    blob_client = blob_service_client.get_blob_client(container=container_name, blob=blob_name)
    
    # 创建可追加类型的Blob（必须指定blob_type）
    if not blob_client.exists():
        initial_data = b'Initial data for the blob'
        blob_client.upload_blob(initial_data, blob_type="AppendBlob")
    ctx.log.info("BlobLogger started.")

# 以下request、response、done函数逻辑保持不变
def request(flow):
    if not blob_client:
        return
    # ... 原有数据处理逻辑 ...

def response(flow):
    if not blob_client:
        return
    # ... 原有数据处理逻辑 ...

def done():
    ctx.log.info("BlobLogger stopped.")

3. 确保Blob类型匹配 #

调用append_block要求目标Blob必须是AppendBlob类型，原代码创建Blob时未指定类型，默认是BlockBlob，会导致追加操作失败。修复时已在upload_blob中指定blob_type="AppendBlob"。

4. 基础连通性检查 #

• 验证Azure连接字符串的正确性，确保包含有效的账户密钥和存储账户名。
• 测试本地网络能否直接访问Azure Blob存储服务（可执行命令 curl https://<你的存储账户名>.blob.core.windows.net 检查连通性）。
• 确认存储账户的防火墙规则允许本地IP地址访问。

内容的提问来源于stack exchange，提问作者satish