如何正确覆盖PLUGIN_PATH以在启动Docker后添加外部Kafka Connect连接器？

Hey there! Let's break down how to properly override the CONNECT_PLUGIN_PATH so your external Kafka Connect connectors load correctly after starting Docker.

First, let's recap your setup #

You've mounted your local connector directory /home/madmin/connectorf to the container's /usr/share/java path, and the default CONNECT_PLUGIN_PATH includes two locations: /usr/share/java,/usr/share/confluent-hub-components. Here's how to adjust this based on your needs:

Option 1: Keep default Confluent connectors + load your custom ones #

Right now, mounting your local directory to /usr/share/java replaces the container's original /usr/share/java (which holds Confluent's default connectors). If you want to keep those default connectors and add your own, tweak your setup like this:

1. Adjust the volume mount to a new, unused path in the container (so you don't overwrite default files):

   volumes:
     - /home/madmin/connectorf:/usr/share/custom-connectors
   

2. Override the CONNECT_PLUGIN_PATH to include the new custom path alongside the default ones:

   # In your Docker Compose file
   environment:
     - CONNECT_PLUGIN_PATH=/usr/share/java,/usr/share/confluent-hub-components,/usr/share/custom-connectors
   

   Or if you're using docker run:

   docker run -e CONNECT_PLUGIN_PATH="/usr/share/java,/usr/share/confluent-hub-components,/usr/share/custom-connectors" ...
   

Option 2: Only use your custom connectors #

If you don't need Confluent's default connectors and want to exclusively load your own, you can simplify the path:

1. Keep your existing volume mount (since you're okay replacing /usr/share/java):

   volumes:
     - /home/madmin/connectorf:/usr/share/java
   

2. Set CONNECT_PLUGIN_PATH to just your mounted directory:

   # In Docker Compose
   environment:
     - CONNECT_PLUGIN_PATH=/usr/share/java
   

   Or with docker run:

   docker run -e CONNECT_PLUGIN_PATH="/usr/share/java" ...
   

Verify it's working #

After starting the container, you can confirm everything is set up correctly:

• Check the environment variable inside the container:

  docker exec -it <your-connect-container-name> bash
  echo $CONNECT_PLUGIN_PATH
  

• Check the Connect logs to see if your connectors are being loaded:

  docker logs <your-connect-container-name> | grep "Loading plugin from"
  

Quick note on directory structure #

Make sure your local /home/madmin/connectorf follows Kafka Connect's best practices: place each connector's JARs in its own subdirectory to avoid class conflicts. For example:

/home/madmin/connectorf/
  ├── debezium-mysql/
  │   ├── debezium-connector-mysql-2.3.0.Final.jar
  │   └── mysql-connector-java-8.0.30.jar
  └── confluent-s3-sink/
      └── confluentinc-kafka-connect-s3-10.0.0.jar

内容的提问来源于stack exchange，提问作者roman_