如何用R及rvest抓取无特定URL、由combo boxes生成的动态表格？

Absolutely! This is totally doable using R's rvest package (along with httr which it relies on). The key here is that the combo boxes are part of an HTML form—when you select options and click "Consultar", the form submits a POST request to the server with your selected parameters, which returns the page containing the target table. We can simulate this form submission programmatically.

Here's a step-by-step guide tailored to the ODEPA site you linked:

Step 1: Load required libraries #

First, make sure you have rvest installed, then load it (and httr for some underlying request handling):

install.packages("rvest")
library(rvest)
library(httr)

Step 2: Inspect the form to find parameter values #

Before writing code, you need to know the exact values the combo boxes send when selected. You can get these by inspecting the page's HTML:

• Right-click the region combo box → Inspect
• Look for the <select> tag with name="region"—each <option> inside has a value attribute (like "1" for Arica y Parinacota)
• Repeat this for the product combo box (name="producto") and any other fields you need to set.

Alternatively, you can scrape these values directly with rvest:

url <- "http://www.odepa.gob.cl/precios/precios-al-consumidor-en-linea"
page <- read_html(url)

# Get region options (value + label)
region_options <- page %>% 
  html_nodes("select[name='region'] option") %>% 
  tibble(
    value = html_attr("value"),
    label = html_text(trim = TRUE)
  )

# Get product options
product_options <- page %>% 
  html_nodes("select[name='producto'] option") %>% 
  tibble(
    value = html_attr("value"),
    label = html_text(trim = TRUE)
  )

Step 3: Simulate form submission #

Once you have the values you want to use, fill out the form and submit it:

# Start a session (important for maintaining cookies if needed)
session <- html_session(url)

# Extract the form (it's the first form on the page)
form <- html_form(session)[[1]]

# Fill the form with your desired values
# Example: Select region "1" (Arica y Parinacota) and product "1001" (Aceite de Oliva)
filled_form <- set_values(
  form,
  region = "1",
  producto = "1001",
  fecha_inicio = "01-01-2024",  # Optional: set custom date range
  fecha_fin = "31-01-2024"
)

# Submit the form (the submit button is named "consultar")
response <- submit_form(session, filled_form, submit = "consultar")

Step 4: Extract and process the table #

Now that you have the response page with the table, extract it into a data frame:

# Find the table (it's the main result table on the page)
result_table <- response %>% 
  html_node("table.table") %>%  # Using the class "table" to target it
  html_table(header = TRUE, trim = TRUE)

# View the result
head(result_table)

Notes to keep in mind #

• Required fields: Make sure you set all mandatory fields (the form might have default values for dates, but it's safer to specify them explicitly).
• Encoding: Since the site uses Spanish characters, ensure your R session handles UTF-8 correctly (most modern R setups do this by default).
• Dynamic content: If the site ever switches to loading tables via JavaScript (AJAX) instead of form submission, you'd need a tool like RSelenium to simulate a browser. But for now, the standard form submission works perfectly with rvest.

内容的提问来源于stack exchange，提问作者user191919