C#使用HttpWebRequest读取URL源码返回403 Forbidden错误如何解决

该错误是目标网站配置了反爬校验，你当前的请求参数不符合常规浏览器的请求规则，按以下步骤调整即可正常获取页面源码： #

调整要点 #

• 替换自定义的User-Agent为真实浏览器的UA标识，非常规UA会被反爬策略直接拦截，常规UA示例：Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36
• 补充Accept、Accept-Language两个标准请求头，模拟浏览器的默认请求规则
• 开启自动解压缩支持，处理网站返回的gzip/deflate压缩内容，避免被反爬策略识别
• 用using语法自动释放请求、流对象，避免内存泄漏，同时指定读取编码防止乱码

调整后的完整代码 #

CookieContainer cookierJar = new CookieContainer();
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create("https://www.hepsiburada.com/dyson-v11-absolute-extra-kablosuz-supurge-dyson-turkiye-garantili-p-HBV00000W4W6U");

webRequest.AllowAutoRedirect = true;
webRequest.CachePolicy = new System.Net.Cache.RequestCachePolicy(System.Net.Cache.RequestCacheLevel.NoCacheNoStore);
webRequest.CookieContainer = cookierJar;
// 替换为真实浏览器UA
webRequest.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36";
// 补充必要的标准请求头
webRequest.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9";
webRequest.Headers.Add("Accept-Language", "zh-CN,zh;q=0.9,en;q=0.8");
webRequest.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
// 若仍报错可补充以下Referer头，模拟从站点首页跳转访问
// webRequest.Referer = "https://www.hepsiburada.com/";

int statusCode = 0;
string sourceCode = string.Empty;

try
{
    using (HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse())
    {
        statusCode = (int)webResponse.StatusCode;
        using (StreamReader readContent = new StreamReader(webResponse.GetResponseStream(), System.Text.Encoding.UTF8))
        {
            sourceCode = readContent.ReadToEnd();
        }
    }
}
catch (WebException xc)
{
    if (xc.Response is HttpWebResponse rs)
    {
        using (StreamReader readContent = new StreamReader(rs.GetResponseStream(), System.Text.Encoding.UTF8))
        {
            sourceCode = readContent.ReadToEnd();
        }
        statusCode = (int)rs.StatusCode;
    }
    else
    {
        statusCode = (int)xc.Status;
        sourceCode = xc.Message;
    }
}
catch (Exception xc)
{
    sourceCode = xc.Message;
}

内容的提问来源于stack exchange，提问作者Burhan Umutlu