求助：使用rvest仅返回标题，代码仅获取列标题问题无法解决

Hey there! Let's figure out why your rvest code is only pulling in table headers instead of the actual data—this is a super common snag, so let's break down the most likely fixes step by step.

1. Check if the data is dynamically loaded #

A lot of modern websites use JavaScript to load content after the initial page loads. Since rvest grabs the static HTML (the code sent by the server before any JS runs), if your table data is added via JS, you won't see it in the static source.

To confirm this:

• Open your target URL in a browser, hit F12 to open DevTools
• Go to the "Page Source" tab (not the "Elements" tab—Elements shows the rendered DOM)
• Search for a piece of data you expect to see in the table. If it doesn't show up, you're dealing with dynamic content.

Fix this by using a tool that can render JavaScript, like the chromote package (lightweight headless Chrome):

library(rvest)
library(chromote)

# Start a headless Chrome session
session <- ChromoteSession$new()
session$Page$navigate("YOUR_TARGET_URL")
session$Page$loadEventFired() # Wait for page to fully load

# Grab the fully rendered HTML
rendered_html <- session$Runtime$evaluate("document.documentElement.outerHTML")$result$value
page <- read_html(rendered_html)

# Now try extracting the table again
table_data <- page %>% html_node("table") %>% html_table(fill = TRUE)
print(table_data)

# Clean up the session
session$close()

2. Verify your CSS/XPath selectors are targeting the right elements #

It's easy to accidentally target only the header row instead of the entire table. For example, if you're using html_nodes("th") you'll only get headers. Or if your table has nested structures (like <thead> for headers and <tbody> for data), you might need to be more specific.

Try this approach to manually extract rows and data:

library(rvest)

page <- read_html("YOUR_TARGET_URL")

# Grab table headers first
headers <- page %>% html_nodes("table thead th") %>% html_text(trim = TRUE)

# Grab all data rows from the tbody
data_rows <- page %>% html_nodes("table tbody tr")

# Extract each cell's text from the rows
table_rows <- lapply(data_rows, function(row) {
  row %>% html_nodes("td") %>% html_text(trim = TRUE)
})

# Convert to a data frame and add headers
table_df <- do.call(rbind, table_rows)
colnames(table_df) <- headers

print(table_df)

3. Check if the "table" isn't actually a table #

Some websites fake table layouts using <div> or <span> elements instead of proper <table>, <tr>, <td> tags. In this case, html_table() won't work—you'll need to target the container elements directly.

For example, if rows are in <div class="table-row"> and cells in <div class="table-cell">:

library(rvest)

page <- read_html("YOUR_TARGET_URL")

# Grab headers (assuming they're in a div with class "table-header")
headers <- page %>% html_nodes(".table-header .table-cell") %>% html_text(trim = TRUE)

# Grab all data rows
data_rows <- page %>% html_nodes(".table-row")

# Extract cell content
table_rows <- lapply(data_rows, function(row) {
  row %>% html_nodes(".table-cell") %>% html_text(trim = TRUE)
})

# Build data frame
table_df <- do.call(rbind, table_rows)
colnames(table_df) <- headers

print(table_df)

Quick Debug Tip #

Always use your browser's DevTools to inspect the elements: right-click on a data cell, select "Inspect", and copy the CSS selector or XPath directly. This ensures you're targeting exactly the elements you need.

内容的提问来源于stack exchange，提问作者hpesoj626