已实现BeagleBone与AWS IoT的MQTT消息传输，如何实现双向文件传输？

Great news—AWS IoT fully supports bidirectional transfer of both text and binary files between your BeagleBone and the AWS cloud, and you can build this right on top of your existing MQTT setup. Let’s break down how to make this work:

Text Transfer (Simple & Straightforward) #

Text is the easiest since MQTT natively handles string payloads. You just need paired topics for two-way flow:

• BeagleBone → Cloud: You’re already doing this—your board publishes text messages to an upstream topic (e.g., beaglebone/telemetry/text), and AWS IoT Core receives them.
• Cloud → BeagleBone: Have your BeagleBone subscribe to a dedicated downstream topic (e.g., beaglebone/commands/text). From the AWS side, you can publish text messages to this topic via the IoT Console, CLI, or a Lambda function, and your BeagleBone will pick them up instantly.

You can even use AWS IoT Rules to route incoming text messages to other services (like S3 for storage or DynamoDB for logging) or trigger actions based on message content.

Binary File Transfer (Handling Larger Data) #

Binary files need a bit more work because AWS IoT Core caps MQTT messages at 128KB. The solution is to split files into smaller chunks, send them sequentially, and reassemble them on the other end. Here’s a step-by-step approach:

1. Chunking Strategy #

• Split your binary file into ~100KB chunks (leaving room for metadata like chunk numbers).
• For each chunk, include metadata: chunk index, total number of chunks, and a unique file ID (to avoid mixing up multiple transfers).

2. Bidirectional Flow Implementation #

Cloud → BeagleBone

• On AWS: Split your target file into chunks, then publish each chunk to a downstream topic (e.g., beaglebone/download/binary).
• On BeagleBone: Subscribe to the downstream topic, collect chunks in order, and reassemble once all chunks are received.

BeagleBone → Cloud

• On BeagleBone: Split your local binary file into chunks, publish each to an upstream topic (e.g., beaglebone/upload/binary).
• On AWS: Use a Lambda function or IoT Rule to collect chunks, reassemble the file, and store it in S3 or another storage service.

Example Code Snippet (BeagleBone → Cloud) #

Here’s a quick Python example using paho-mqtt to send a binary file in chunks:

import paho.mqtt.client as mqtt
import os
import json

# Your existing AWS IoT connection setup
client = mqtt.Client(client_id="beaglebone-file-upload")
client.tls_set(ca_certs="/path/to/root-CA.crt", certfile="/path/to/certificate.pem.crt", keyfile="/path/to/private.pem.key")
client.connect("your-aws-iot-endpoint-1234567890.us-east-1.amazonaws.com", 8883, 60)

def send_binary_file(file_path, topic):
    chunk_size = 1024 * 100  # 100KB chunks
    file_size = os.path.getsize(file_path)
    total_chunks = (file_size + chunk_size - 1) // chunk_size
    file_id = "my-unique-file-123"  # Unique ID for this transfer

    with open(file_path, 'rb') as f:
        for chunk_num in range(total_chunks):
            chunk = f.read(chunk_size)
            # Package chunk with metadata
            payload = {
                "file_id": file_id,
                "chunk_num": chunk_num,
                "total_chunks": total_chunks,
                "data": chunk.hex()  # Encode binary to hex string for safe MQTT transfer
            }
            # Publish with QoS 1 for reliability
            client.publish(topic, json.dumps(payload), qos=1)
            print(f"Sent chunk {chunk_num + 1}/{total_chunks}")

# Send your file
send_binary_file("/home/debian/my-image.bin", "beaglebone/upload/binary")

client.loop_forever()

Example Code Snippet (BeagleBone Receiving Cloud Binary) #

import paho.mqtt.client as mqtt
import json
import os

received_chunks = {}
current_file_id = None
total_chunks = None

def on_message(client, userdata, msg):
    global current_file_id, total_chunks
    payload = json.loads(msg.payload.decode())
    
    file_id = payload["file_id"]
    chunk_num = payload["chunk_num"]
    total_chunks = payload["total_chunks"]
    chunk_data = bytes.fromhex(payload["data"])

    # Initialize storage for new file
    if file_id != current_file_id:
        current_file_id = file_id
        received_chunks.clear()
    
    received_chunks[chunk_num] = chunk_data
    
    # Check if all chunks are received
    if len(received_chunks) == total_chunks:
        # Reassemble and save the file
        file_data = b''.join([received_chunks[i] for i in range(total_chunks)])
        with open(f"/home/debian/received-{file_id}.bin", 'wb') as f:
            f.write(file_data)
        print(f"File {file_id} received and saved successfully!")

# Connection setup (same as before)
client = mqtt.Client(client_id="beaglebone-file-download")
client.tls_set(ca_certs="/path/to/root-CA.crt", certfile="/path/to/certificate.pem.crt", keyfile="/path/to/private.pem.key")
client.connect("your-aws-iot-endpoint-1234567890.us-east-1.amazonaws.com", 8883, 60)

client.subscribe("beaglebone/download/binary", qos=1)
client.on_message = on_message

client.loop_forever()

Pro Tips for Reliability & Scalability #

• Use QoS 1 or 2: For critical transfers, avoid QoS 0 (fire-and-forget) to ensure chunks aren’t lost.
• Add ACK Mechanisms: Have the receiver send an acknowledgment for each chunk (e.g., to beaglebone/upload/ack). The sender should wait for the ACK before sending the next chunk to avoid unnecessary retransmissions.
• Larger Files? Use AWS IoT Jobs: If you’re transferring very large files (like firmware updates), AWS IoT Jobs is built for this. It handles chunking, retries, and progress tracking out of the box, so you don’t have to build all that logic yourself.

内容的提问来源于stack exchange，提问作者rohit