Python实现同目录下多文件遍历循环的技术求助

Hey there! I get that you’ve been hunting for solutions to batch process your 1000 eye-tracking files and haven’t found what you need yet. Let’s break this down step by step to get your PyGaze analysis running smoothly across all your files.

Core Approach #

We’ll structure this workflow to:

• Safely traverse all your target files without relying on os.chdir (which can cause path confusion)
• Wrap your single-file analysis logic into a reusable function
• Loop through every file, run the analysis, and handle potential errors gracefully

Step 1: Set Up Your Imports #

First, make sure you’ve got all necessary libraries imported:

import pandas as pd
from pygazeanalyser import gazeplotter  # Adjust to match the PyGaze modules you use
from pathlib import Path
import glob

Step 2: Define Your Single-File Analysis Function #

Let’s encapsulate the logic for reading a file into a DataFrame and running your PyGaze analysis. Tweak this to match your specific analysis needs:

def process_eye_tracking_file(file_path):
    # Read file into DataFrame (adjust sep/header to match your file format)
    df = pd.read_csv(file_path, sep="\t", header=0)
    
    # Your PyGaze analysis logic here
    # Example: Generate a heatmap (adjust parameters to your screen/data specs)
    gazeplotter.draw_heatmap(
        df['x'], df['y'],
        imgshape=(1920, 1080),  # Match your experiment's screen resolution
        savefilename=f"{file_path.stem}_heatmap.png"
    )
    
    # Add other analysis steps: fixation detection, saccade analysis, etc.
    # ...
    
    return df  # Or return key metrics if you want to aggregate results later

Step 3: Batch Process All Files #

Here are two clean ways to loop through your files—pick whichever fits your coding style:

Option 1: Using Pathlib (Modern & Recommended)

# Set your root directory (no more os.chdir!)
root_dir = Path("/Users/Documents/Analyse/Eye movements/Python - Eye Analyse")

# Get all eye-tracking files (adjust the pattern to match your file extensions: *.csv, *.txt, etc.)
eye_tracking_files = list(root_dir.glob("*.csv"))

# Loop through each file with error handling
for file in eye_tracking_files:
    print(f"Processing {file.name}...")
    try:
        process_eye_tracking_file(file)
    except Exception as e:
        print(f"Failed to process {file.name}: {str(e)}")
        # Optional: Log errors to a file for later review
        with open("error_log.txt", "a") as f:
            f.write(f"{file.name}: {str(e)}\n")

Option 2: Using Glob

import os

root_dir = "/Users/Documents/Analyse/Eye movements/Python - Eye Analyse"
eye_tracking_files = glob.glob(os.path.join(root_dir, "*.csv"))

for file_path in eye_tracking_files:
    print(f"Processing {os.path.basename(file_path)}...")
    try:
        process_eye_tracking_file(Path(file_path))
    except Exception as e:
        print(f"Failed to process {os.path.basename(file_path)}: {str(e)}")
        with open("error_log.txt", "a") as f:
            f.write(f"{os.path.basename(file_path)}: {str(e)}\n")

Step 4: Optional Optimizations #

• Speed Up with Multiprocessing: For 1000 files, parallel processing can cut down runtime significantly. Use the multiprocessing library:

  from multiprocessing import Pool
  
  if __name__ == "__main__":
      with Pool(processes=4) as pool:  # Adjust processes based on your CPU core count
          pool.map(process_eye_tracking_file, eye_tracking_files)
  

• Aggregate Results: If you need to combine metrics from all files, modify the function to return key data points and collect them in a master DataFrame.

Key Notes #

• Double-check that your DataFrame column names (like x/y for gaze coordinates) match what’s expected by your PyGaze functions.
• If your files are in non-CSV formats (e.g., .edf), you’ll need additional libraries like pyedfread to read them into DataFrames.

内容的提问来源于stack exchange，提问作者Sarah.d