如何通过BQ CLI将以Ctrl A(
)为分隔符的文件加载至BigQuery

Got it, let's tackle this. Ctrl A (the
character, ASCII hex 0x01) is technically a single character—so even though it's a control character, the BQ CLI can handle it. The trick is inputting it correctly in your shell. Here are the methods for common environments:

Bash/Zsh (Linux/macOS) #

There are two reliable ways to pass the Ctrl A delimiter:

1. Direct Terminal Input #

You can insert the literal Ctrl A character directly into your command:

• Type bq load --source_format=CSV --field_delimiter='
• Press Ctrl + V followed by Ctrl + A (this tells your terminal to insert the control character instead of triggering a shortcut)
• Finish the command with the rest of your parameters:

  bq load --source_format=CSV --field_delimiter='
  ' your_dataset.your_table gs://your_bucket/your_file.csv
  

2. Hex Escape Sequence (Great for Scripts) #

If you're writing a shell script or want a more reproducible command, use bash's ANSI-C escape syntax with the hex code for Ctrl A (\x01):

bq load --source_format=CSV --field_delimiter=$'\x01' your_dataset.your_table gs://your_bucket/your_file.csv

This avoids needing to manually input control characters, which is perfect for automation.

PowerShell (Windows) #

PowerShell makes it easy to generate the Ctrl A character using its type system:

1. Inline Character Generation #

Pass the character directly using [char]0x01:

bq load --source_format=CSV --field_delimiter ([char]0x01) your_dataset.your_table gs://your_bucket/your_file.csv

2. Variable for Reuse #

If you need to use the delimiter multiple times, store it in a variable first:

$ctrlADelimiter = [char]0x01
bq load --source_format=CSV --field_delimiter $ctrlADelimiter your_dataset.your_table gs://your_bucket/your_file.csv

Command Prompt (Windows) #

CMD is less friendly with control characters, so we recommend using PowerShell instead if possible. If you must use CMD, you can generate the character via a workaround:

for /f %%a in ('powershell -command "[char]0x01"') do set "DELIM=%%a"
bq load --source_format=CSV --field_delimiter "%DELIM%" your_dataset.your_table gs://your_bucket/your_file.csv

Key Notes #

• Double-check that your file actually uses Ctrl A as the delimiter (you can verify with tools like cat -A your_file.csv on Linux/macOS, which will show ^A for Ctrl A).
• If your data contains quoted values or newlines inside fields, add parameters like --allow_quoted_newlines or adjust the --quote setting as needed to avoid parsing errors.
• For automated workflows, stick to the hex escape or variable-based methods—they're more reliable than manual control character input.

内容的提问来源于stack exchange，提问作者ChitsC