如何通过编程下载CSV文件？遇远程连接关闭异常求助

我有一个指向CSV文件的URL，文件大小300KB，包含2700行、15列。尝试多种Python和C#下载方式均触发异常：

Remote end closed connection without response

Python 方法一 #

import pandas as pd
import numpy as np
import os

# 用read_csv下载CSV
df = pd.read_csv('https://nsearchives.nseindia.com/products/content/sec_bhavdata_full_17072024.csv', low_memory=False)

Python 方法二 #

import urllib.request

url = 'https://nsearchives.nseindia.com/products/content/sec_bhavdata_full_17072024.csv'
filename = 'large_file.csv'

def download_large_file(url, filename):
    with urllib.request.urlopen(url) as response, open(filename, 'wb') as out_file:
        while True:
            chunk = response.read(8192)  # 分8KB块下载
            if not chunk:
                break
            out_file.write(chunk)

download_large_file(url, filename)
print("文件下载成功！")

C# 方法 #

using System.Net;

WebClient webClient = new WebClient();
webClient.DownloadFile("URL");

这个错误是目标网站（NSE India）的反爬机制拦截了请求导致的——默认HTTP客户端的请求头没有模拟浏览器标识，被服务器判定为非合法请求后主动断开连接。以下是修复后的代码：

Python 修复方案 #

修复后的pandas代码

import pandas as pd

# 添加模拟浏览器的请求头
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36'
}

# 带请求头发起请求
req = pd.io.common.urlopen('https://nsearchives.nseindia.com/products/content/sec_bhavdata_full_17072024.csv', headers=headers)
df = pd.read_csv(req, low_memory=False)

修复后的urllib代码

import urllib.request

url = 'https://nsearchives.nseindia.com/products/content/sec_bhavdata_full_17072024.csv'
filename = 'large_file.csv'

# 添加请求头
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36'
}

def download_large_file(url, filename):
    req = urllib.request.Request(url, headers=headers)
    with urllib.request.urlopen(req) as response, open(filename, 'wb') as out_file:
        while True:
            chunk = response.read(8192)  # 分8KB块下载
            if not chunk:
                break
            out_file.write(chunk)

download_large_file(url, filename)
print("文件下载成功！")

C# 修复方案 #

修复后的WebClient代码

using System.Net;

using (WebClient webClient = new WebClient())
{
    // 添加User-Agent请求头
    webClient.Headers.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36");
    webClient.DownloadFile("https://nsearchives.nseindia.com/products/content/sec_bhavdata_full_17072024.csv", "sec_bhavdata_full_17072024.csv");
}

更推荐的HttpClient方案（.NET Core/.NET 5+）

using System.Net.Http;
using System.IO;
using System.Threading.Tasks;

class Program
{
    static async Task Main(string[] args)
    {
        string url = "https://nsearchives.nseindia.com/products/content/sec_bhavdata_full_17072024.csv";
        string filePath = "sec_bhavdata_full_17072024.csv";

        using (HttpClient client = new HttpClient())
        {
            // 添加User-Agent请求头
            client.DefaultRequestHeaders.UserAgent.ParseAdd("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36");
            using (HttpResponseMessage response = await client.GetAsync(url))
            {
                response.EnsureSuccessStatusCode();
                using (var stream = await response.Content.ReadAsStreamAsync())
                using (var fileStream = new FileStream(filePath, FileMode.Create, FileAccess.Write))
                {
                    await stream.CopyToAsync(fileStream);
                }
            }
        }
    }
}

额外提示 #

• 若仍失败，可复制浏览器请求的完整头信息（如Accept、Referer）添加到代码中；
• 注意NSE India的文件URL日期格式（示例中17072024对应17-07-2024）；
• 避免频繁请求，防止IP被封禁。

内容的提问来源于stack exchange，提问作者NutsAndBolts