求助：通过SSH在共享服务器运行定时爬虫Python脚本

I’ll walk you through every step clearly—no confusing jargon, since you’re new to server operations. Let’s get started:

1. Install PuTTY (if you haven’t already) #

• Search for "PuTTY download" and grab the installer from its official site.
• Run the installer and follow the prompts to set it up on your computer.

2. Connect to Your Server via SSH #

• Open PuTTY. In the "Host Name (or IP address)" field, paste the server’s hostname/IP your admin provided.
• Leave the "Port" set to 22 (default for SSH connections).
• Click "Open" to launch the connection.
• When prompted, enter your username and password (note: you won’t see characters as you type the password—this is normal; just finish typing and press Enter).

3. Verify Python is Installed on the Server #

Once logged in (you’ll see a command-line prompt), check if Python is available:

• For Python 3 (the version you should use): python3 --version
• If you see a version number like Python 3.8.10, you’re good to go. If not, ask your server admin to install Python 3 for you.

4. Upload Your Script to the Server #

You need to copy your local Python script to the server using PuTTY’s built-in pscp tool:

• Open a command prompt (CMD) on your local computer.
• Navigate to the folder where your script is saved. For example: cd C:\Users\YourName\Scripts
• Run this command (replace placeholders with your details):

  pscp your_script.py your_username@server_hostname:/home/your_username/
  

• Enter your server password when prompted. This copies the script to your home directory on the server.

5. Modify Your Script for Non-Interactive Execution (Critical!) #

Your current script uses input() which requires interactive typing—but when running in the background, you can’t do that. Let’s adjust it to accept the wait time as a command-line argument:

• Open your script in PyCharm.
• Replace the line s = input("No. of hours to wait before beginning crawl: ") with:

  import sys
  if len(sys.argv) != 2:
      print("Usage: python3 your_script.py <wait_hours>")
      sys.exit(1)
  try:
      wait_hours = float(sys.argv[1])
  except ValueError:
      print("Error: Please enter a valid number for wait hours.")
      sys.exit(1)
  

• Update your code to use wait_hours (convert to seconds if using time.sleep(): time.sleep(wait_hours * 3600)).
• Save the modified script and re-upload it to the server using the pscp command above.

6. Run the Script in the Background #

To keep the script running even after you close PuTTY:

• On the server’s command line, go to your home directory: cd ~
• Run this command (replace 24 with your desired wait time in hours):

  nohup python3 your_script.py 24 > crawler_logs.out 2>&1 &
  

  • nohup ensures the script keeps running when you log out.
  • > crawler_logs.out saves the script’s output to a log file for later checking.
  • 2>&1 sends error messages to the same log file.
  • & runs the script in the background.
• You’ll see a process ID (PID) printed—keep this handy if you need to stop the script later.

7. Check if the Script is Running #

• To confirm the script is active: ps aux | grep your_script.py
• To view the log file (to debug or verify functionality): cat crawler_logs.out or tail -f crawler_logs.out (the latter shows real-time updates).

8. Stop the Script (If Needed) #

If you need to halt the script, find its PID with ps aux | grep your_script.py, then run: kill <PID> (replace <PID> with the actual number).

Alternative: Use Cron for Reliable Scheduling #

If you’re worried about the script crashing and not restarting, set up a cron job to run it daily instead of relying on the script’s internal loop:

• First, modify your script to skip the initial wait and just run the crawl once per execution.
• Open the cron editor: crontab -e
• Add a line to run the script daily at your preferred time (example: 2 AM every day):

  0 2 * * * python3 /home/your_username/your_script.py >> /home/your_username/cron_crawler_logs.out 2>&1
  

• Save and exit (in nano, press Ctrl+O, Enter, then Ctrl+X).

内容的提问来源于stack exchange，提问作者Deepon GhoseRoy