如何基于Confluent Kafka Docker cp-all-in-one添加MySQL连接器

Got it, let's walk through exactly how to add the MySQL connector to your Confluent Kafka cp-all-in-one Docker setup step by step. I've done this before, so here's what works:

1. Prepare the MySQL Driver File #

• First, get the mysql-connector-java-5.1.46 package from the official MySQL distribution. Extract the archive, and pull out the mysql-connector-java-5.1.46-bin.jar file.
• In your local cp-all-in-one example directory, create a new folder (e.g., kafka-connect-mysql) and copy the .jar file into it.
• Make sure the directory has proper permissions so the Docker container can read the file:

  chmod -R 755 ./kafka-connect-mysql
  

2. Update the Docker Compose Configuration #

Open your docker-compose.yml file and modify the connect service section as follows:

Add Volume Mount #

Under the volumes key of the connect service, add a line to mount your local driver directory into the container:

services:
  connect:
    image: confluentinc/cp-kafka-connect:4.0.x
    # ... existing configuration ...
    volumes:
      # ... existing volumes ...
      - ./kafka-connect-mysql:/usr/share/java/mysql-connector

Configure Plugin Path #

Add or update the CONNECT_PLUGIN_PATH environment variable in the connect service's environment section. This tells Kafka Connect where to look for plugin JARs—we need to include both the default Confluent plugin path and our new MySQL driver directory:

environment:
  # ... existing environment variables ...
  CONNECT_PLUGIN_PATH: "/usr/share/java,/usr/share/java/mysql-connector"

Note: If the CONNECT_PLUGIN_PATH already exists in your config, just append the new path (separated by a comma) instead of replacing it entirely.

3. Restart the Connect Service #

Apply the changes by restarting the Docker Compose stack:

docker-compose down
docker-compose up -d

4. Verify the Connector is Loaded #

To confirm everything worked, use Kafka Connect's REST API to check loaded plugins:

curl -X GET http://localhost:8083/connector-plugins

Look for entries like io.confluent.connect.jdbc.JdbcSinkConnector and io.confluent.connect.jdbc.JdbcSourceConnector in the JSON response—this means the MySQL driver is successfully loaded and you're ready to create MySQL source/sink connectors.

内容的提问来源于stack exchange，提问作者Bruno Ripa