如何在登录状态下下载完整网站用于离线阅读？

Got it, this is a super common pain point—most off-the-shelf site downloaders fail at authenticated sessions because they don't carry over your logged-in cookies/session data. Here are three reliable approaches I've used successfully for this exact scenario:

Wget is a classic command-line tool, and it works great if you can grab your logged-in cookies from your browser. Here's how:

• Step 1: Export your logged-in cookies
  After logging into the target site in your browser (Chrome/Firefox), open the developer tools (F12), go to the Application (Chrome) or Storage (Firefox) tab, expand Cookies, select the site's domain. You can use a browser extension like "Get Cookies.txt LOCALLY" to export the cookies into a plain text file named cookies.txt (make sure it's in the Netscape format that Wget understands).

• Step 2: Run the Wget command
  Open your terminal and run this command, replacing the URL with your target site:

  wget --mirror --load-cookies cookies.txt --convert-links --adjust-extension --page-requisites --no-parent https://your-target-site.com
  

  Let's break down the key flags:

  • --mirror: Enables recursive downloading and mirrors the site structure
  • --load-cookies: Uses your exported authenticated cookies
  • --convert-links: Rewrites links to work offline
  • --page-requisites: Downloads all assets (CSS, JS, images) needed to render pages
  • --no-parent: Prevents Wget from crawling outside the target directory

If the site uses a more complex login flow (like requiring a CSRF token), a curl script lets you automate the login first, then crawl the site. Here's a basic example:

• Step 1: Grab the CSRF token (if needed)
  First, fetch the login page to get the CSRF token (many sites require this for POST requests):

  CSRF_TOKEN=$(curl -s https://your-target-site.com/login | grep -o 'name="csrf_token" value="[^"]*"' | cut -d '"' -f 4)
  

• Step 2: Authenticate and save cookies
  Send a POST request to the login endpoint with your credentials and CSRF token:

  curl -c cookies.txt -d "username=your-username&password=your-password&csrf_token=$CSRF_TOKEN" https://your-target-site.com/login
  

• Step 3: Recursively crawl the site
  Use curl's recursive mode with the saved cookies:

  curl -L -b cookies.txt -r -k -O -J -K- << EOF
  url = "https://your-target-site.com"
  recursive = true
  level = inf
  include = "*.html,*.css,*.js,*.png,*.jpg"
  EOF
  

  Note: Adjust the include list to match the file types you need to download.

If command-line tools aren't your thing, browser extensions work directly in your logged-in session, so they automatically use your authenticated cookies. Two great options:

• SingleFileZ: This extension lets you save entire websites (including subpages) in a single ZIP file. After logging in, right-click the extension icon, select "Save all pages in this tab", then set the recursive depth (e.g., "infinite" to grab everything). It preserves all styling and assets, and works offline.
• WebScraper: While primarily for scraping data, it can also save full pages with its "Export as HTML" feature. Set up a sitemap to crawl the site's structure, then run the scraper in your logged-in browser— it will pull all pages using your active session.

Important Notes #

• Some sites use anti-scraping measures (like rate limiting or session timeouts). If you hit issues, add delays to your command-line tools (e.g., --wait 2 in Wget to wait 2 seconds between requests) to avoid getting blocked.
• Always check the site's Terms of Service before scraping—make sure you're allowed to download their content for your use case.

内容的提问来源于stack exchange，提问作者alaskaman