Python遍历文件时因文件被并发删除引发FileNotFoundError的问题解决咨询

The root cause here is a race condition: between the time you list files with os.listdir() and the time you try to retrieve their modification time with os.path.getmtime(), another process deletes the file. Since os.listdir() only returns filenames (not their current existence status), this leads to an error when Python tries to access a file that no longer exists.

Here's a robust fix that handles these edge cases while preserving your original logic:

def poke(self, context):
    full_path = self.filepath
    file_pattern = re.compile(self.filepattern)
    
    sorted_files = []
    # Use os.scandir for efficient metadata access and reduced race condition window
    with os.scandir(full_path) as entries:
        valid_files = []
        for entry in entries:
            # Skip directories immediately
            if not entry.is_file(follow_symlinks=False):
                continue
            try:
                # Safely get modification time, catching deletions mid-process
                mtime = entry.stat().st_mtime
                valid_files.append((mtime, entry.path))
            except FileNotFoundError:
                self.log.warning(f"File {entry.name} was deleted before processing, skipping.")
                continue
        
        # Sort valid files by modification time
        valid_files.sort(key=lambda x: x[0])
        sorted_files = [file_path for (mtime, file_path) in valid_files]
    
    for file_path in sorted_files:
        filename = os.path.basename(file_path)
        # Double-check file exists before pattern matching
        if not os.path.isfile(file_path):
            self.log.warning(f"File {filename} no longer exists, skipping.")
            continue
        
        if not re.match(file_pattern, filename):
            self.log.info(f"Ignoring non-matching file: {filename}")
            self.log.info(f"Used pattern: {file_pattern.pattern}")
        else:
            self.log.info(f"File Sensed: {filename}")
            return True
    return False

Key Improvements: #

1. Removed os.chdir(): Using full paths eliminates working directory side effects and makes the code more reliable.
2. os.scandir() Usage: This is more efficient than os.listdir() because it retrieves file metadata (like mtime) in the same system call as listing files, reducing the window for race conditions.
3. Safe Metadata Retrieval: Wrapped entry.stat() in a try-except block to catch FileNotFoundError if a file is deleted between scanning and accessing its metadata.
4. Post-Sort Existence Check: Verifies the file still exists before processing, covering cases where files are deleted after sorting but before pattern matching.
5. Directory Filtering: Explicitly skips directories to avoid accidental processing.

Minimal Changes Alternative: #

If you want to modify your original code as little as possible, use a helper function to handle exceptions in the sort key:

def poke(self, context):
    full_path = self.filepath
    file_pattern = re.compile(self.filepattern)
    
    def safe_get_mtime(filename):
        try:
            return os.path.getmtime(os.path.join(full_path, filename))
        except FileNotFoundError:
            # Assign old timestamp to push deleted files to the end of the sorted list
            return 0.0
    
    for filename in sorted(os.listdir(full_path), key=safe_get_mtime):
        file_path = os.path.join(full_path, filename)
        if not os.path.isfile(file_path):
            self.log.warning(f"File {filename} no longer exists, skipping.")
            continue
        if not re.match(file_pattern, filename):
            self.log.info(filename)
            self.log.info(file_pattern)
        else:
            self.log.info(f"File Sensed {filename}")
            return True
    return False

This version retains your original structure while adding safety checks to handle deleted files gracefully.

内容的提问来源于stack exchange，提问作者pushpendu dhara