如何为RDD中的文本内容添加首尾引号？

Hey there! Let's get this sorted for you—adding leading and trailing quotes to each line in your RDD is straightforward with a simple transformation.

Step-by-Step Breakdown #

Since sc.textFile() reads each line of your text file as a separate element in the RDD, you just need to use the map() transformation to tweak every element individually. This function lets you apply a custom change to each line, which is exactly what we need here.

Complete Working Code #

val fileRdd = sc.textFile("file://data/sample.txt")
// Wrap each line with Chinese quotation marks
val newRdd = fileRdd.map(line => s"""“$line”""")

// If you need English quotation marks instead, use this line:
// val newRdd = fileRdd.map(line => s""""$line"""")

How This Works #

• The map() function iterates over every line in the original RDD.
• Using Scala's string interpolation (s"""..."""), we wrap each line with your desired quotation marks. Triple quotes let us include the quotes directly without needing escape characters (like \" in regular strings), making the code cleaner.

Quick Verification #

For your sample input line:

I once did an interview for the Banbury Herald. I must look it out one of these days, for the biography. Strange chap they sent me. A boy, really. As tall as a man, but with the puppy fat of youth. It was nightfall now and I must go home.

The transformed RDD element will match your expected output:

“I once did an interview for the Banbury Herald. I must look it out one of these days, for the biography. Strange chap they sent me. A boy, really. As tall as a man, but with the puppy fat of youth. It was nightfall now and I must go home.”

Bonus: If You Need to Wrap the Entire File in Quotes #

If your goal was to treat the whole file as a single string (instead of per-line quotes), use wholeTextFiles() instead. This reads the full file content as one element:

val wholeFileRdd = sc.wholeTextFiles("file://data/sample.txt")
val newWholeFileRdd = wholeFileRdd.map { case (filePath, content) => s"""“$content”""" }

内容的提问来源于stack exchange，提问作者Steve McAffer