AWS Glue读取多个S3 JSON文件触发PARSE_SYNTAX_ERROR语法错误的问题求助

我最近在使用AWS Glue执行任务时遇到了一个奇怪的问题：当我尝试一次性读取S3存储桶里的两个JSON文件时，触发了PARSE_SYNTAX_ERROR语法错误，但单独读取其中任意一个文件却能完全正常工作。

我的代码实现如下：

file_paths = [
    "s3://test_bt/data/batch_date=20241217/20241217/data_0_0_0.json.gz",
    "s3://test_bt/data/batch_date=20241217/20241217/data_0_1_0.json.gz"
]

df_single = spark.read.json(*file_paths)

df_single.show()

运行后抛出的完整错误信息如下：

Spark Error Class: PARSE_SYNTAX_ERROR; Traceback (most recent call last):
  File "/tmp/TEST.py", line 10, in <module>
     df_single = spark.read.json(*s3_paths)
  File "/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 254, in json
    self._set_opts(
  File "/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 50, in _set_opts
    self.schema(schema)  # type: ignore[attr-defined]
  File "/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 118, in schema
    self._jreader = self._jreader.schema(schema)
  File "/opt/amazon/spark/python/lib/py4j-0.10.9.5-src.zip/py4j/java_gateway.py", line 1321, in __call__
    return_value = get_return_value(
  File "/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 196, in deco
    raise converted from None
pyspark.sql.utils.ParseException: 
Syntax error at or near ':'(line 1, pos 2)

== SQL ==
s3://test_bt/data/batch_date=20241217/20241217/data_0_1_0.json.gz
--^^^

备注：内容来源于stack exchange，提问作者abhimanyu