能否通过Python或PHP在服务器端对指定URL执行JavaScript代码？

Absolutely! You can absolutely pull this off on the server side with either Python or PHP—here’s a breakdown of how to implement your requirement for both languages, plus key considerations for server environments.

Python Solutions #

Since you’re already familiar with Selenium, adapting it for server use is straightforward (you just need to run it in headless mode, since servers don’t have a GUI). We’ll also cover a lighter alternative for static pages.

1. Headless Selenium (For JS-Rendered Pages) #

This mirrors your local setup but configures the browser to run without a visible window:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

def scrape_prices(url):
    # Configure Chrome for headless server execution
    chrome_options = Options()
    chrome_options.add_argument("--headless=new")  # Modern headless mode
    chrome_options.add_argument("--no-sandbox")  # Required for most server environments
    chrome_options.add_argument("--disable-dev-shm-usage")  # Avoids resource limits

    driver = webdriver.Chrome(options=chrome_options)
    try:
        driver.get(url)
        # Execute your existing JavaScript logic
        price_data = driver.execute_script("""
            var priceElements = document.getElementById('priceblock_ourprice');
            var prices = [];
            for (var i = 0; i < 3; i++) {
                prices.push(priceElements.children[i].innerHTML);
            }
            return prices;
        """)
        return price_data
    finally:
        driver.quit()  # Always clean up the browser instance

# Usage example
target_url = "https://your-target-site.com/product"
prices = scrape_prices(target_url)
print(prices)

2. Requests + BeautifulSoup (For Static Pages) #

If the page doesn’t rely on JavaScript to load the price element, this lighter approach avoids the overhead of a full browser:

import requests
from bs4 import BeautifulSoup

def scrape_static_prices(url):
    # Mimic a real browser to avoid being blocked
    headers = {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36"
    }
    response = requests.get(url, headers=headers)
    response.raise_for_status()  # Handle HTTP errors

    soup = BeautifulSoup(response.text, "html.parser")
    price_block = soup.find(id="priceblock_ourprice")
    
    if price_block:
        # Extract the first 3 child elements' text
        prices = [child.get_text(strip=True) for child in price_block.children[:3]]
        return prices
    return None

PHP Solutions #

PHP offers both static scraping tools and options for JS-rendered pages. Here are the most reliable approaches:

1. Goutte (For Static Pages) #

Goutte is a lightweight scraping library built on Guzzle and Symfony’s DomCrawler:
First, install dependencies via Composer:

composer require fabpot/goutte

Then implement the scraper:

<?php
require 'vendor/autoload.php';

use Goutte\Client;

$client = new Client();
// Set a realistic user agent to bypass basic anti-scraping measures
$client->setHeader('User-Agent', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36');

$crawler = $client->request('GET', 'https://your-target-site.com/product');

$prices = [];
$priceBlock = $crawler->filter('#priceblock_ourprice')->first();

if ($priceBlock->count() > 0) {
    $children = $priceBlock->children();
    // Grab the first 3 child elements' text
    for ($i = 0; $i < 3; $i++) {
        if (isset($children[$i])) {
            $prices[] = trim($children[$i]->text());
        }
    }
}

print_r($prices);
?>

2. PuPHPeteer (For JS-Rendered Pages) #

PuPHPeteer is a PHP wrapper for Node.js’s Puppeteer, which lets you control a headless Chrome browser:
First, install dependencies:

composer require nesk/puphpeteer
# Make sure Node.js and Puppeteer are installed on your server
npm install puppeteer

Then the code:

<?php
require 'vendor/autoload.php';

use Nesk\Puphpeteer\Puppeteer;

$puppeteer = new Puppeteer();
$browser = $puppeteer->launch([
    'headless' => true,
    'args' => ['--no-sandbox', '--disable-dev-shm-usage']  // Server-friendly flags
]);

try {
    $page = $browser->newPage();
    $page->setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36');
    $page->goto('https://your-target-site.com/product');

    // Run your JavaScript logic directly in the browser
    $prices = $page->evaluate("""
        var priceElements = document.getElementById('priceblock_ourprice');
        var prices = [];
        for (var i = 0; i < 3; i++) {
            prices.push(priceElements.children[i].innerHTML);
        }
        return prices;
    """);

    print_r($prices);
} finally {
    $browser->close();  // Clean up the browser instance
}
?>

Key Server-Side Considerations #

• Headless Mode: Always enable this for browser-based tools—servers don’t have a graphical interface.
• Dependencies: For Python, install Chrome/Firefox and their corresponding drivers (e.g., chromedriver). For PHP, ensure Node.js is installed if using PuPHPeteer.
• Anti-Scraping Measures: Many sites block server-side requests. Use realistic user agents, add small delays between requests, and consider using proxies if needed.
• Resource Usage: Browser-based tools (Selenium, PuPHPeteer) are resource-heavy. If you have high traffic, cache results or use static scraping where possible.

内容的提问来源于stack exchange，提问作者Blake