如何查看推送至Kinesis Data Stream的实际数据？

Problem #

I'm trying to fetch records from a Kinesis Data Stream using Node.js, but the Data field in the response comes back as a Buffer instead of the actual data I pushed. Here's my code:

var AWS = require('aws-sdk');
var kinesis = new AWS.Kinesis({region: 'us-east-1'});
var params = {
 ShardIterator: 'AAAAAAAAAAGxAQOmSj54ysG1T+k+V3r7T4b4lC6TopuCy2tcyjpAYGMLO+A5ICkeEMhRd5CH2jNqu7ZQpKsr19leAndgedXhM2hygQjEIYvwJPTyVc5rwerewsdfserreJXnI8+uRa123TjF/07lUTrJf2PMGoNL3+4HXaS1TgPrcTX2OdP4K0Dz8tIz613WWYtvh3paw+g07Jz/2Xlt/e+4L',
 Limit: 1
};
kinesis.getRecords(params, function(err, data) {
 if (err) console.log(err, err.stack); // Handle error
 else console.log(data); // Request succeeded
});

The response I receive looks like this:

{
 Records: [
 {
 SequenceNumber: '49580243932170249199867439514950526887184270832636526594',
 ApproximateArrivalTimestamp: 2018-01-01T10:19:12.782Z,
 Data: <Buffer 48 65 6c 6c 6f 20 56 69 73 68 6e 75 20 48 61 70 70 79 20 4e 65 77 20 59 65 61 72 20 21 21 21>,
 PartitionKey: 'testpartition'
 }
 ],
 NextShardIterator: 'AAAAAAAAAAGno9O2jSEPA6Amt4Fad1nP/7VvLjlT0dQhKF1MMqA+gQpS5yPRqHECbrvbxekP5wt35m0On0gnID6aMadRI1arG+EDYzpFvpBEtf1/CGssIVQIoWx3IHjU2cbay4qc242+NyMoTX05LpQqNIlCEru/935W2yZR4unsdzWiguVqCWY6KHgt8vrDueu+52gJ3lUeEpiRhUHSLvMCSqfCAmAv',
 MillisBehindLatest: 576000
}

How can I view the actual human-readable data from the Data Buffer?

Solution #

The Data field is a Node.js Buffer object, which represents raw binary data. To convert it to a readable string (assuming you pushed UTF-8 encoded data, which is the standard for most use cases), you can use the toString() method on the Buffer.

Here's how to modify your code to display the actual data:

kinesis.getRecords(params, function(err, data) {
 if (err) {
  console.log(err, err.stack); // Handle error
 } else {
  // Process each record to convert Buffer to a UTF-8 string
  const processedRecords = data.Records.map(record => ({
    ...record,
    Data: record.Data.toString('utf8') // Convert Buffer to human-readable string
  }));
  console.log('Processed Records:', processedRecords);
 }
});

In your specific case, the Buffer <Buffer 48 65 6c 6c 6f 20 56 69 73 68 6e 75 20 48 61 70 70 79 20 4e 65 77 20 59 65 61 72 20 21 21 21> will convert to the string Hello Vishnu Happy New Year!!! when using toString('utf8').

If your data was encoded in a different format (like ASCII or base64), you can pass the corresponding encoding to toString() (e.g., record.Data.toString('ascii') or record.Data.toString('base64')).

内容的提问来源于stack exchange，提问作者Vishnu Ranganathan