SparkSQL正则表达式问题:无法从文本中移除反斜杠
问题:SparkSQL处理
\nnn格式文本时的正则转义问题 我的文本字段包含需要屏蔽的\nnn格式内容(nnn为三位数字),执行以下SparkSQL语句:
spark.sql("select regexp_replace('ABC\123XYZ\456','[\d][\d][\d]','') as new_value").show()
预期结果为 "ABC\\XYZ\\",但实际输出:
+---------+ |new_value| +---------+ | ABCSXYZĮ| +---------+
无法识别C和Z后的字符。后续尝试移除反斜杠,执行以下语句时程序崩溃:
spark.sql("select regexp_replace('ABC\123XYZ\456','[\\\\][\d][\d][\d]','') as new_value").show()
无论如何转义反斜杠都失败,但正则表达式在本地测试正常。
补充:尝试建议后的报错
按照Daniel的建议修改语句后,出现如下错误:
>>> spark.sql(r"select regexp_replace(r'ABC\123XYZ\456',r'[\\\\][\d][\d][\d]','') as new_value").show() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/spark/python/pyspark/sql/session.py", line 649, in sql return DataFrame(self._jsparkSession.sql(sqlQuery), self._wrapped) File "/usr/lib/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py", line 1305, in __call__ File "/usr/lib/spark/python/pyspark/sql/utils.py", line 134, in deco raise_from(converted) File "<string>", line 3, in raise_from pyspark.sql.utils.ParseException: Literals of type 'R' are currently not supported.(line 1, pos 22) == SQL == select regexp_replace(r'ABC\123XYZ\456',r'[\\\\][\d][\d][\d]','') as new_value ----------------------^^^
求解决思路!
内容的提问来源于stack exchange,提问作者SQL RV
相关产品推荐
相关产品推荐

