如何不解压30GB大型gzip文件提取指定行（第300021行）至新文件

Absolutely, this is totally possible—and you don’t need to waste space or time decompressing the entire 30GB file. The key is using tools that stream decompressed content line-by-line, so only the necessary parts are processed. Here are the most efficient methods:

Method 1: Using sed (with early exit for efficiency) #

The sed tool can print your target line and immediately stop processing the rest of the file, which is ideal for large archives. Run this command:

zcat your_large_file.gz | sed -n '300021{p;q}' > extracted_line.txt

• zcat: Decompresses the gzip file and sends content to stdout (use gzcat on macOS if zcat isn’t available, or gzip -dc your_large_file.gz which works everywhere).
• sed -n '300021{p;q}': The -n flag suppresses default output. 300021{p;q} tells sed to print (p) line 300021, then quit (q) immediately so it doesn’t read the rest of the 30GB file.
• > extracted_line.txt: Redirects the output to your new file.

Method 2: Using awk (another efficient option) #

Awk is equally effective here, with straightforward syntax for line number checks. This command also exits early once the target line is found:

zcat your_large_file.gz | awk 'NR==300021 {print; exit}' > extracted_line.txt

• NR==300021: Checks if the current line number (NR) matches your target.
• {print; exit}: Prints the line and exits the script right away, avoiding unnecessary processing.

Key Notes #

• Efficiency: Both methods avoid reading the entire file beyond your target line, saving significant time and system resources compared to full decompression.
• Line Number Verification: If you want to confirm the file has at least 300021 lines, run zcat your_large_file.gz | wc -l—but be aware this will process the entire file, which may take a while for a 30GB archive.
• Alternative Tools: gzip -dc works as a drop-in replacement for zcat in all commands above.

内容的提问来源于stack exchange，提问作者Jeeppp