如何用Python向Event Hub发送IoT数据并正确存储至Blob与SQL？

Hey there, let's break down what's happening here and how to fix it!

The weird-looking content you're seeing in Blob Storage is totally expected behavior from Event Hub's Capture feature. By default, Event Hub captures events into Avro-formatted files that wrap your actual payload alongside Event Hub metadata (like SequenceNumber, Offset, and other service-specific fields). That's why you're seeing the Avro schema and wrapper structure instead of your raw IoT data.

To extract your original message content, you'll need to parse these Avro files in Python. Here's a step-by-step guide:

1. Install Required Libraries #

First, grab the packages needed to handle Avro parsing and Azure Blob storage:

pip install avro azure-storage-blob

2. Parse Avro Files from Blob Storage #

Here's a practical code snippet to read an Avro file from your Blob container, parse it, and extract your raw IoT data:

from avro.datafile import DataFileReader
from avro.io import DatumReader
from azure.storage.blob import BlobServiceClient
import json

# Replace these with your actual values
BLOB_CONNECTION_STRING = "<your-blob-storage-connection-string>"
CAPTURE_CONTAINER_NAME = "<your-capture-container-name>"
TARGET_AVRO_BLOB = "<name-of-the-avro-file-in-blob>"

# Connect to Blob Storage and download the Avro file
blob_service_client = BlobServiceClient.from_connection_string(BLOB_CONNECTION_STRING)
blob_client = blob_service_client.get_blob_client(container=CAPTURE_CONTAINER_NAME, blob=TARGET_AVRO_BLOB)
avro_file_content = blob_client.download_blob().readall()

# Save temp file (Avro reader needs a file-like object)
with open("temp_eventhub.avro", "wb") as temp_file:
    temp_file.write(avro_file_content)

# Parse the Avro file and extract raw data
reader = DataFileReader(open("temp_eventhub.avro", "rb"), DatumReader())
for event_record in reader:
    # The raw payload is stored in the "Body" field as bytes
    raw_payload = event_record["Body"].decode("utf-8")
    # If you sent JSON data, parse it into a Python dict
    iot_data = json.loads(raw_payload)
    print("Extracted IoT Data:", iot_data)
reader.close()

3. Bonus: Alternative Workflows #

If you'd prefer to avoid parsing Avro entirely, you have a couple options:

• Use an Azure Function triggered by Event Hub: The function can directly grab the raw message body and write it to Blob Storage/SQL Database without Avro wrapping.
• Use Azure Stream Analytics: Set up a job that reads from Event Hub, extracts the payload, and writes the raw data directly to your target storage/database.

Also, a quick tip: When sending data from your IoT devices, serialize it as JSON before sending—this makes parsing the extracted payload much smoother.

内容的提问来源于stack exchange，提问作者Furkan