如何使用C#结合HtmlAgilityPack获取网页中<video>标签的源？

Hey there! I’ve worked with HtmlAgilityPack quite a bit for scraping media elements, so let me walk you through exactly how to pull the source from a <video> tag in C#.

First, make sure you’ve got HtmlAgilityPack installed. You can grab it via NuGet Package Manager, or run this command in your terminal:

Install-Package HtmlAgilityPack

Or if you’re using .NET CLI:

dotnet add package HtmlAgilityPack

Step 1: Load the HTML Document #

First, you’ll need to fetch the HTML content from the target page (or load it from a local file). Here’s how to do it with HttpClient:

using HtmlAgilityPack;
using System.Net.Http;

// Initialize HttpClient (best practice: reuse this instance instead of creating new ones)
var httpClient = new HttpClient();
var htmlContent = await httpClient.GetStringAsync("https://your-target-page.com");

// Load the HTML into HtmlAgilityPack's document object
var doc = new HtmlDocument();
doc.LoadHtml(htmlContent);

Step 2: Extract Video Sources #

There are two common scenarios for video sources:

1. The <video> tag has a direct src attribute.
2. The <video> tag contains nested <source> tags (used for multiple format options).

Here’s code that handles both cases:

// Grab the first <video> tag on the page
var videoNode = doc.DocumentNode.SelectSingleNode("//video");

if (videoNode != null)
{
    // Check for direct src attribute first
    var directSrc = videoNode.GetAttributeValue("src", string.Empty);
    if (!string.IsNullOrEmpty(directSrc))
    {
        // Convert relative URLs to absolute (if needed)
        var baseUri = new Uri("https://your-target-page.com");
        var absoluteDirectSrc = new Uri(baseUri, directSrc).ToString();
        Console.WriteLine($"Direct video source: {absoluteDirectSrc}");
    }
    else
    {
        // Look for nested <source> tags inside the video element
        var sourceNodes = videoNode.SelectNodes(".//source");
        if (sourceNodes != null)
        {
            Console.WriteLine("Found multiple video sources:");
            foreach (var source in sourceNodes)
            {
                var sourceSrc = source.GetAttributeValue("src", string.Empty);
                var sourceType = source.GetAttributeValue("type", "unknown format");
                
                // Convert relative paths to absolute URLs
                var baseUri = new Uri("https://your-target-page.com");
                var absoluteSourceSrc = new Uri(baseUri, sourceSrc).ToString();
                
                Console.WriteLine($"- {sourceType}: {absoluteSourceSrc}");
            }
        }
        else
        {
            Console.WriteLine("No video sources found in the <video> tag.");
        }
    }
}
else
{
    Console.WriteLine("No <video> tag found on the page.");
}

Important Note About Dynamic Content #

If the video source is loaded dynamically with JavaScript (e.g., it’s populated after page load), HtmlAgilityPack won’t be able to see it—since it only parses static HTML. In that case, you’ll need to use a tool that can render JavaScript, like Selenium or PuppeteerSharp. These tools simulate a real browser, so they’ll capture the fully rendered page including JS-loaded content.

内容的提问来源于stack exchange，提问作者Aaron Gallick