如何使用Sqoop将数据导入为JSON格式？目前无法采用Spark方案

Absolutely! You don’t need Spark to get Sqoop to import data into JSON format — there are a couple of straightforward, native ways to do this depending on your Sqoop version.

Method 1: Use Sqoop’s Built-in --as-jsonfile Parameter (Recommended) #

Starting with Sqoop 1.4.6, there’s a native flag --as-jsonfile that directly exports your imported data into JSON format. Each row from your source table will be saved as a separate JSON object in the output files.

Here’s a sample command for importing from a MySQL database to HDFS as JSON:

sqoop import \
  --connect jdbc:mysql://your-db-host:3306/your-database \
  --username your-username \
  --password your-password \
  --table your-source-table \
  --target-dir /hdfs/path/to/json-output \
  --as-jsonfile

If you want to import to your local filesystem instead of HDFS, add the --local flag to the command:

sqoop import \
  --connect jdbc:mysql://your-db-host:3306/your-database \
  --username your-username \
  --password your-password \
  --table your-source-table \
  --target-dir /local/path/to/json-output \
  --as-jsonfile \
  --local

This method is clean, requires no extra dependencies, and produces well-formatted JSON records.

Method 2: Custom MapReduce Output Format (For Older Sqoop Versions) #

If you’re stuck on a Sqoop version older than 1.4.6, you can specify a JSON-compatible MapReduce output format directly. Hadoop includes a JsonOutputFormat class that you can reference via the --output-format parameter.

Here’s how to use it:

sqoop import \
  --connect jdbc:mysql://your-db-host:3306/your-database \
  --username your-username \
  --password your-password \
  --table your-source-table \
  --target-dir /hdfs/path/to/json-output \
  --output-format org.apache.hadoop.mapreduce.lib.output.JsonOutputFormat

Note that this relies on your Hadoop environment having the necessary classes available (which it should, by default). The output structure will be slightly different than the --as-jsonfile method, but it’s still valid JSON per record.

Key Notes #

• Both methods work entirely without Spark — they’re pure Sqoop/MapReduce operations.
• The JSON output will have each row as an individual JSON object (one per line), which is ideal for most downstream processing tools that consume JSON data.

内容的提问来源于stack exchange，提问作者Bala