Python脚本执行报Memory Error求助：为每行添加指定字符串出错

Hey there! Let’s figure out why you’re hitting that Memory Error and get your script running smoothly. The #1 cause of this issue when working with file line modifications is loading the entire file into memory at once—Python just can’t handle that if your file is bigger than your available RAM. Here’s how to fix it:

Common Culprits & Solutions #

1. You’re reading the entire file into memory #

If your script uses code like this, it’s pulling every line into RAM all at once:

# Bad: Loads all lines into a list
lines = open("your_file.txt").readlines()
# OR
with open("your_file.txt") as f:
    content = f.read()  # Reads the entire file into a single string

This works for small files, but it’ll crash with a Memory Error for large ones.

Fix: Process lines one at a time
Instead, iterate over the file directly and write each modified line immediately to an output file. This keeps only one line in memory at a time:

with open("input.txt", "r") as infile, open("output.txt", "w") as outfile:
    specific_string = "YOUR_CUSTOM_TEXT"
    for line in infile:
        # Adjust line handling to keep/remove newlines as needed
        modified_line = line.rstrip("\n") + f" {specific_string}\n"
        outfile.write(modified_line)

2. You’re storing all modified lines in a list #

If you’re collecting every modified line in a list before writing (like below), that list will eat up memory as it grows:

# Bad: Stores all modified lines in memory
modified_lines = []
with open("input.txt") as f:
    for line in f:
        modified_lines.append(line + " YOUR_STRING")

with open("output.txt", "w") as f:
    f.writelines(modified_lines)

Fix: Ditch the list and write immediately
Use the line-by-line approach from the first solution—no need to hoard all lines in memory.

3. Bonus: Use generators for complex processing #

If you need to add extra logic to line modifications, a generator function lets you process lines lazily (only generating one line at a time):

def generate_modified_lines(input_path, add_string):
    with open(input_path, "r") as f:
        for line in f:
            yield line.rstrip("\n") + f" {add_string}\n"

# Use the generator to write to output
with open("output.txt", "w") as outfile:
    for modified_line in generate_modified_lines("input.txt", "YOUR_TEXT"):
        outfile.write(modified_line)

Quick Additional Checks #

• Double-check that your "specific string" isn’t accidentally huge (like a multi-megabyte blob)—that could add unnecessary memory load per line.
• If your file is massive (think tens of gigabytes), line-by-line processing is still the way to go; it’ll work regardless of file size as long as you have disk space for the output.

Give these fixes a shot—line-by-line processing is the simplest and most effective fix for most Memory Error cases in file handling. Let me know if you need help tweaking your exact script!

内容的提问来源于stack exchange，提问作者Jialong Xu