SparkR对RDD相关API的支持情况问询及官方文档查询
Answers to Your SparkR RDD API Questions
Q1: Is there an official SparkR documentation link that clarifies support for RDD APIs?
There isn't a dedicated standalone page in the official Spark docs spelling out the lack of RDD API support for SparkR, but the official position is unambiguous:
- The entire public SparkR documentation centers solely on DataFrame/Dataset and Spark SQL APIs as the supported interface for R developers. You won’t find any mention of RDD APIs being part of the supported feature set.
- As you noted in the Spark issue discussion, core developers have explicitly stated:
需要澄清的是,我们不支持在R中使用RDD。任何通过SparkR:::访问的内容均不受支持,包括unionRDD也不被支持。
(Translation: "To clarify, we do not support using RDDs in R. Anything accessed via SparkR::: is unsupported, including unionRDD.")
Q2: Is support for RDD APIs (like SparkR:::map) in SparkR common? Does SparkR only allow DataFrame-related APIs now?
- Absolutely not. Functions like
SparkR:::mapare internal, unexported methods—they were never meant to be used by end users. RDD API access in SparkR has never been a supported or common use case. - Yes, SparkR’s official, supported functionality is restricted entirely to DataFrame/Dataset, Spark SQL, and DataFrame-based MLlib components. Any RDD-related operations you can access via
SparkR:::are implementation details that could change or be removed without warning in future versions. For any production code, stick strictly to the public, documented APIs to avoid stability issues.
内容的提问来源于stack exchange,提问作者Tony
相关产品推荐
相关产品推荐

