技术求助：外部网页表格数据导出至本地CSV及空列填充方案

Got it, let's break this down into simple, actionable steps. We'll use Python for this task—it's ideal for web scraping and data cleanup like filling empty column values.

Step 1: Install the Tools You'll Need #

First, make sure you have these Python libraries installed. Open your terminal and run:

pip install pandas requests beautifulsoup4

(If the table loads dynamically with JavaScript, you'll also need selenium—we'll cover that later.)

Step 2: Grab the Table from the Webpage #

Pandas has a handy read_html() function that can pull tables directly from a URL. This works great for static tables (the ones that load with the page, no JS required). Here's how to use it:

import pandas as pd

# Your target URL
url = "http://www.rafisklaget.no/portal/pls/portal/PORTAL.LANDINGSPLANNING.show?p_arg_names=p_regmerke&p_arg_values=T%200182T"

# Load all tables on the page into a list of DataFrames
tables = pd.read_html(url)

# Most of the time, the table you want is the first one (index 0)
# To confirm, print the first few rows: print(tables[0].head())
df = tables[0]

Step 3: Fill Empty Column Values Downward #

This is the key part. Pandas' ffill() method (short for "forward fill") will take the last non-empty value in a column and repeat it until it hits a new value or the end of the table. First, we'll convert any blank/whitespace-only entries to NaN (Pandas' missing value marker) so ffill() works correctly:

# Replace blank/whitespace-only cells with NaN
df = df.replace(r'^\s*$', pd.NA, regex=True)

# Fill empty values downward for all columns
df_filled = df.ffill()

# If you only want to fill specific columns, use subset:
# df_filled = df.ffill(subset=["Column Name 1", "Column Name 2"])

Step 4: Export to Local CSV #

Finally, save the cleaned DataFrame to a CSV file on your computer:

# Export without the extra index column
df_filled.to_csv("landing_planning.csv", index=False, encoding='utf-8')

print("Done! Your CSV is ready with filled empty values.")

Handling Dynamic Tables (If read_html() Fails) #

If the table doesn't show up in tables (because it's loaded by JavaScript), use Selenium to simulate a browser loading the page:

1. Install Selenium and a browser driver (like ChromeDriver):

pip install selenium

2. Use this code instead to load the page:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import pandas as pd

# Set up headless Chrome (no browser window pops up)
chrome_options = Options()
chrome_options.add_argument("--headless=new")
driver = webdriver.Chrome(options=chrome_options)

# Load the page and wait for JS to render the table
driver.get(url)

# Get the full page source after JS loads
html = driver.page_source

# Extract tables from the rendered HTML
tables = pd.read_html(html)
driver.quit()

# Rest of the code (fill values, export to CSV) is the same as above!

Quick Tips #

• Double-check which table you're using: If tables[0] isn't your target, loop through tables and print each one's head to find the right index.
• Verify the filled data: Run print(df_filled.head(20)) to make sure empty values are being filled correctly.
• Fix encoding issues: If your CSV has weird characters, try encoding='latin-1' instead of utf-8 in the to_csv() call.

内容的提问来源于stack exchange，提问作者Rune Larsen