如何用Clang Python绑定解析含OpenACC的C/C++ AST并生成JSON

Prerequisites #

First, install the required Clang Python bindings:

pip install clang

Note: You'll need a local LLVM installation (version 12+ recommended). Adjust the library path in the script to match your LLVM setup.

Python Script #

import clang.cindex
import json
from clang.cindex import CursorKind

# Configure Clang library path (update this to your LLVM installation path)
clang.cindex.Config.set_library_path('/usr/lib/llvm-14/lib')
index = clang.cindex.Index.create()

def get_cursor_text(cursor):
    """Extract raw source text from a given AST cursor"""
    return cursor.extent.source.get_bytes().decode('utf-8').strip()

def collect_openacc_pairs(node, directive_pairs):
    # Check for OpenACC attributes attached to the current node
    for attr in node.get_children():
        if attr.kind == CursorKind.ATTRIBUTE and 'acc_' in attr.spelling:
            # Traverse up to find the code block the directive applies to
            target_node = node
            valid_target_kinds = [
                CursorKind.COMPOUND_STMT, CursorKind.FOR_STMT,
                CursorKind.WHILE_STMT, CursorKind.DO_STMT, CursorKind.IF_STMT
            ]
            while target_node.kind not in valid_target_kinds and target_node.parent:
                target_node = target_node.parent
            
            # Add the directive-code block pair to our list
            directive_pairs.append({
                "directive_type": attr.spelling.replace('acc_', 'acc '),
                "directive_text": get_cursor_text(attr),
                "code_block": get_cursor_text(target_node)
            })
    
    # Recurse into child nodes to find more directives
    for child in node.get_children():
        collect_openacc_pairs(child, directive_pairs)

if __name__ == "__main__":
    input_file = "sample_openacc_code.c"  # Replace with your input file path
    # Parse with OpenACC support enabled
    translation_unit = index.parse(
        input_file,
        args=['-x', 'c', '-std=c11', '-fopenacc']
    )
    
    directive_pairs = []
    collect_openacc_pairs(translation_unit.cursor, directive_pairs)
    
    # Write results to JSON
    with open("openacc_directive_map.json", "w") as outfile:
        json.dump(directive_pairs, outfile, indent=4)
    
    print(f"Generated JSON with {len(directive_pairs)} directive-code block pairs")

Sample Input & Output #

Sample OpenACC Code (sample_openacc_code.c) #

#include <stdio.h>

int main() {
    int arr[100];
    int i;

    #pragma acc parallel loop
    for (i = 0; i < 100; i++) {
        arr[i] = i * 2;
    }

    #pragma acc data copy(arr)
    {
        #pragma acc kernels
        for (i = 0; i < 100; i++) {
            arr[i] += 1;
        }
    }

    return 0;
}

Sample Output JSON (openacc_directive_map.json) #

[
    {
        "directive_type": "acc parallel_loop",
        "directive_text": "#pragma acc parallel loop",
        "code_block": "for (i = 0; i < 100; i++) {\n        arr[i] = i * 2;\n    }"
    },
    {
        "directive_type": "acc data",
        "directive_text": "#pragma acc data copy(arr)",
        "code_block": "{\n        #pragma acc kernels\n        for (i = 0; i < 100; i++) {\n            arr[i] += 1;\n        }\n    }"
    },
    {
        "directive_type": "acc kernels",
        "directive_text": "#pragma acc kernels",
        "code_block": "for (i = 0; i < 100; i++) {\n            arr[i] += 1;\n        }"
    }
]

Key Explanations #

• Clang AST Parsing: Clang natively supports OpenACC and represents directives as AST attributes, making it reliable for accurate parsing.
• AST Traversal: The script recursively walks the AST to locate OpenACC attributes, then maps each to its associated code block by traversing up to valid statement types.
• Text Extraction: Uses cursor extents to pull the exact source text of both directives and their target code blocks.

Relevant Learning Resources #

• Clang Python Bindings: Learn to navigate AST nodes, cursor types, and attribute handling to extend the script for complex cases.
• OpenACC Specification: Understand directive syntax and scope rules to refine how you map directives to code blocks.
• Clang AST Reference: Familiarize yourself with Clang's AST node hierarchy to handle edge cases like nested directives or conditional code blocks.
• Python JSON Module: Master serialization options to customize the output format as needed.

内容的提问来源于stack exchange，提问作者Samyak Jhaveri