提交Druid Supervisor数据源创建任务时遇No FileSystem for scheme: HTTP错误

Looks like you've hit a classic protocol mix-up in your Druid-HDFS setup! Let's break down what's going wrong and fix it step by step.

Root Cause of the Error #

The error No FileSystem for scheme: http occurs because you've configured Druid to use an HTTP URI for HDFS storage—but HDFS uses its own dedicated hdfs:// scheme, not http://. The http://hadoopmachince:9000/... value you set is actually the web UI endpoint of HDFS, not the proper filesystem access URI.

Step-by-Step Fixes #

• Correct the HDFS Storage Path Configuration
  Open your common.runtime.properties file, locate the problematic line:

  druid.storage.storageDirectory=http://hadoopmachince:9000/druid/segments
  

  Replace it with the valid HDFS scheme:

  druid.storage.storageDirectory=hdfs://hadoopmachince:9000/druid/segments
  

  Before restarting services, make sure the target directory exists in HDFS and Druid has write permissions. Run these HDFS commands to set it up:

  hadoop fs -mkdir -p /druid/segments
  hadoop fs -chown -R druid:druid /druid/segments
  

  (Swap druid:druid with the user/group your Druid services run under if different.)

• Verify HDFS Connectivity & Dependencies

  • Check if Druid nodes can reach the HDFS NameNode: run telnet hadoopmachince 9000 to confirm the port is open and accessible.
  • Ensure Druid can access HDFS core configuration files. Copy core-site.xml and hdfs-site.xml from your Hadoop cluster into Druid's conf/druid/_common directory—this lets Druid auto-detect HDFS settings without manual URI adjustments.
  • For imply-2.3.9, HDFS dependencies are included by default, but if issues persist, double-check that hadoop-hdfs-2.7.3.jar and related Hadoop jars are present in Druid's lib folder.

• Restart Druid Services
  After updating the configuration, restart the Druid services that handle storage operations: Coordinator, Overlord, and Historical nodes. This ensures the new HDFS settings are loaded properly.

Additional Checks #

If you still run into issues:

• Confirm your HDFS cluster is healthy (check NameNode and DataNode statuses)
• Verify the HDFS NameNode hostname hadoopmachince is resolvable from all Druid nodes (add it to /etc/hosts if DNS isn't set up)

内容的提问来源于stack exchange，提问作者Sudhanshu Lenka