AWS Glue Python Shell导入pyodbc库报错：找不到libodbc.so.2文件

Hey there, I’ve dealt with this exact pyodbc import error in AWS Glue Python Shell multiple times—let’s get this sorted. The root issue is straightforward: the default Glue Python runtime environment doesn’t include the system-level ODBC libraries that pyodbc depends on, specifically libodbc.so.2. Here are two solid solutions that work reliably:

Solution 1: Use a Bootstrap Script to Install ODBC Libraries #

Bootstrap scripts run before your Glue job starts, so they’re perfect for setting up system-level dependencies. Here’s how to do it:

• Create a shell script named install_odbc_deps.sh with this content:

  #!/bin/bash
  # Install required ODBC system libraries
  sudo yum install -y unixODBC unixODBC-devel
  

• Upload this script to an S3 bucket you have access to (e.g., s3://your-glue-resources/bootstrap/install_odbc_deps.sh).
• Open your Glue Python Shell job in the AWS Console, navigate to the Job details tab.
• Under Advanced properties, find the Bootstrap script path field and paste the S3 URL of your script.
• Save the job configuration.

Solution 2: Bundle pyodbc + ODBC Libraries (Alternative for Restricted Environments) #

If you can’t use a bootstrap script (due to permissions or policy restrictions), you can package the required libraries with your job:

1. Download the ODBC libraries on an Amazon Linux 2 instance (since Glue uses Amazon Linux 2 under the hood):

   yum install -y unixODBC unixODBC-devel
   # Copy the required files
   cp /usr/lib64/libodbc.so.2 /path/to/your/bundle/
   cp /usr/lib64/libodbcinst.so.2 /path/to/your/bundle/
   

2. Bundle the libraries with your Python code:
   • Create a lib folder in your job directory, add the copied .so files to it.
   • In your Python script, add this code at the very top to tell the system where to find the libraries:

     import os
     import ctypes
     
     # Add the lib folder to the library path
     lib_path = os.path.join(os.path.dirname(__file__), 'lib')
     os.environ['LD_LIBRARY_PATH'] = f"{lib_path}:{os.environ.get('LD_LIBRARY_PATH', '')}"
     
     # Load the libraries explicitly (optional but helps avoid import issues)
     ctypes.CDLL(os.path.join(lib_path, 'libodbc.so.2'))
     ctypes.CDLL(os.path.join(lib_path, 'libodbcinst.so.2'))
     
     # Now import pyodbc
     import pyodbc
     

3. Include pyodbc in your job dependencies:
   • Upload the pyodbc wheel file (compiled for Amazon Linux 2) to S3, or use the --additional-python-modules pyodbc parameter in your job’s Python arguments if your Glue version supports it.

Quick Note #

Don’t forget that if you’re connecting to a specific database (like SQL Server), you’ll also need to install the corresponding ODBC driver (e.g., msodbcsql17). You can add that installation command to your bootstrap script too:

sudo yum install -y https://packages.microsoft.com/config/rhel/7/prod.repo
sudo yum install -y msodbcsql17

内容的提问来源于stack exchange，提问作者Umer