使用TableauScraper爬取Tableau仪表盘时遭遇JSONDecodeError求助

错误详情 #

---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
Cell In[1], line 5
      3 ts = TS()
      4 ts.delayMs = 2000
----> 5 ts.loads(url)

File ~/opt/anaconda3/lib/python3.9/site-packages/tableauscraper/TableauScraper.py:79, in TableauScraper.loads(self, url, params)
     76     r = api.getTableauVizForSession(self, self.session, url)
     77     soup = BeautifulSoup(r, "html.parser")
---> 79 self.tableauData = json.loads(
     80     soup.find("textarea", {"id": "tsConfigContainer"}).text
     81 )
     83 uri = urlparse(url)
     84 self.host = "{uri.scheme}://{uri.netloc}".format(uri=uri)

File ~/opt/anaconda3/lib/python3.9/json/__init__.py:346, in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    341     s = s.decode(detect_encoding(s), 'surrogatepass')
    343 if (cls is None and object_hook is None and
    344         parse_int is None and parse_float is None and
    345         parse_constant is None and object_pairs_hook is None and not kw):
---> 346     return _default_decoder.decode(s)
    347 if cls is None:
    348     cls = JSONDecoder

File ~/opt/anaconda3/lib/python3.9/json/decoder.py:337, in JSONDecoder.decode(self, s, _w)
    332 def decode(self, s, _w=WHITESPACE.match):
    333     """Return the Python representation of ``s`` (a ``str`` instance
    334     containing a JSON document).
    335 
    336     """
---> 337     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    338     end = _w(s, end).end()
    339     if end != len(s):

File ~/opt/anaconda3/lib/python3.9/json/decoder.py:355, in JSONDecoder.raw_decode(self, s, idx)
    353     obj, end = self.scan_once(s, idx)
    354 except StopIteration as err:
---> 355     raise JSONDecodeError("Expecting value", s, err.value) from None
    356 return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

示例代码 #

from tableauscraper import TableauScraper as TS
url = "https://public.tableau.com/views/PlayerStats-Top5Leagues20192020/OnePlayerSummary"
ts = TS()
ts.delayMs = 2000
ts.loads(url)

排查思路与解决方案 #

• 检查页面结构变化：错误核心是找不到tsConfigContainer文本域的有效内容，说明Tableau可能更新了页面结构。手动访问目标URL，打开浏览器开发者工具，搜索tsConfigContainer，确认元素是否存在，或是否有新的容器存储配置JSON。
• 更新工具版本：执行pip install --upgrade tableauscraper更新到最新版本，仓库可能已修复页面结构适配问题。
• 添加浏览器请求头：部分仪表盘会验证请求来源，给session添加模拟浏览器的请求头：

  from tableauscraper import TableauScraper as TS
  url = "https://public.tableau.com/views/PlayerStats-Top5Leagues20192020/OnePlayerSummary"
  ts = TS()
  ts.session.headers.update({
      "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36"
  })
  ts.delayMs = 2000
  ts.loads(url)
  

• 验证请求有效性：先单独请求页面，确认是否获取到正常内容而非错误页面：

  import requests
  url = "https://public.tableau.com/views/PlayerStats-Top5Leagues20192020/OnePlayerSummary"
  res = requests.get(url)
  print(res.status_code)
  print(res.text[:500]) # 打印页面前500字符验证
  

• 手动提取配置JSON：若工具未及时适配，可修改代码逻辑，用BeautifulSoup定位新的配置元素，再解析JSON。

内容的提问来源于stack exchange，提问作者Роберт Лорян