Spark技术问询：如何对RDD按字符串进行字母顺序排序

Got it, since sortByKey() sorts based on the first element of your tuple (the integer here), to sort by the string value instead, you’ll want to use Spark’s sortBy() method. This lets you explicitly specify which part of the tuple to use as the sort key.

Here’s the exact code you can run to get your desired output:

scala> val randRDD = sc.parallelize(List( (2,"cat"), (6, "mouse"),(7, "cup"), (3, "book"), (4, "tv"), (1, "screen"), (5, "heater")), 3)
randRDD: org.apache.spark.rdd.RDD[(Int, String)] = ParallelCollectionRDD[0] at parallelize at <console>:27

scala> val sortedByString = randRDD.sortBy(_._2)
sortedByString: org.apache.spark.rdd.RDD[(Int, String)] = MapPartitionsRDD[3] at sortBy at <console>:29

scala> sortedByString.collect()
res0: Array[(Int, String)] = Array((3,book), (2,cat), (7,cup), (5,heater), (6,mouse), (1,screen), (4,tv))

Quick Breakdown: #

• sortBy(_._2) tells Spark to sort each tuple using the second element (_._2)—your string value.
• By default, sortBy() uses ascending order, which gives you the alphabetical sequence you’re looking for. If you ever need reverse alphabetical order, just add the ascending parameter: sortBy(_._2, ascending=false).
• The collect() method pulls the sorted RDD data back to the driver so you can view the final result.

This approach is way more flexible than sortByKey() because it lets you sort on any derived key, not just the first element of the tuple.

内容的提问来源于stack exchange，提问作者vikas