如何使用grep等Linux基础工具提取conda依赖中的Python包名与版本

Problem Context #

I'm learning to use the grep tool, and I have a conda environment file that records Python package info, looking like this:

channels:
  - conda-forge
  - defaults
dependencies:
  - numpy=1.21.1=py39h6635163_0
  - pyinstaller=4.2=py39h4dafc3f_1
  - ...

I only care about the content after dependencies:, and want to use bash tools like grep/sed/awk to iterate over these lines, store the package name in one variable and the version number (ignoring everything after the last =) in another, then call a function with these variables. For example, after processing the first line, I want output like:

$ echo $1 $2
numpy 1.21.1

Solution 1: Use awk (Most Clean & Efficient) #

awk is perfect here because it can easily target the right lines and split fields exactly how we need:

awk '/^dependencies:/ {flag=1; next} flag && /^  - / {
    split($2, arr, "=")
    pkg=arr[1]
    ver=arr[2]
    # Replace the echo line with your function call, e.g., my_function "$pkg" "$ver"
    echo "$pkg $ver"
}' your_env_file.yaml

Breakdown:

• /^dependencies:/ {flag=1; next}: When we hit the line starting with dependencies:, set a flag to start capturing lines, then skip to the next line.
• flag && /^ - /: Only process lines that start with - once the flag is set (these are our dependency lines).
• split($2, arr, "="): Split the second field (the package=version=buildstring part) into an array using = as the delimiter.
• arr[1] gives us the package name, arr[2] gives the version (automatically ignoring everything after the second =).

If you want to loop through each package and handle them one by one, wrap it in a bash while loop:

while read -r pkg ver; do
    # Add your processing logic here
    echo "Handling package: $pkg (version $ver)"
    # my_function "$pkg" "$ver"
done < <(awk '/^dependencies:/ {flag=1; next} flag && /^  - / {split($2, arr, "="); print arr[1], arr[2]}' your_env_file.yaml)

Solution 2: grep + sed Combo #

If you prefer sticking to grep and sed, you can chain them to filter and extract the data:

grep -A999 '^dependencies:' your_env_file.yaml | grep '^  - ' | sed -E 's/^  - ([^=]+)=([^=]+)=.*/\1 \2/'

Again, wrap this in a loop to process each package:

while read -r pkg ver; do
    echo "$pkg $ver"
    # my_function "$pkg" "$ver"
done < <(grep -A999 '^dependencies:' your_env_file.yaml | grep '^  - ' | sed -E 's/^  - ([^=]+)=([^=]+)=.*/\1 \2/')

Breakdown:

• grep -A999 '^dependencies:': Matches the dependencies: line and outputs the next 999 lines (plenty to cover all dependencies).
• grep '^ - ': Filters down to only the actual dependency lines.
• sed -E 's/^ - ([^=]+)=([^=]+)=.*/\1 \2/': Uses regex to capture the package name (before the first =) and version (between first and second =), then replaces the line with just those two values.

Notes #

• Make sure your environment file follows the standard conda YAML format, with dependency lines starting with -.
• Even if package names had spaces (uncommon for conda packages), these methods still work since we're splitting on = instead of spaces.
• Always wrap variables in double quotes when calling functions to avoid issues with special characters or spaces.

内容的提问来源于stack exchange，提问作者Romain Bosq