如何在R中使用Socrata API从数字开头的resource.id深度滚动查询

我用R爬取犹他州Socrata门户的所有公开数据集，因为总量超过10000条，必须使用API的deep scrolling机制和scroll_id参数。但发现带scroll_id查询时，结果是按resource.id的字母顺序返回的——当scroll_id设为NULL时，默认从首字母开头的记录开始拉取，导致数千条数字开头的记录无法被获取到。

我现在的临时解决办法是：先不带scroll_id发起查询，会返回10000条随机顺序的结果，从中提取出编号最小的数字开头记录，再用这个ID作为scroll_id重新进行深度滚动查询。但我觉得这种方法可能无法覆盖全部数据，以下是我目前用的代码：

pacman::p_load(tidyverse,
               httr,
               jsonlite)

# Function to query without scroll_id
get_socrata_data <- function(domain, 
                             limit = 15000) {
    "http://api.us.socrata.com/api/catalog/v1?domains=${domain}&limit=${limit}" %>%
        str_interp() %>%
        httr::GET() %>%
        content("text") %>%
        fromJSON(flatten = TRUE)
}

# Scroll function that includes scroll_id
scroll_socrata <- function(domain,
                           limit = 15000,
                           scroll_id = NULL) {
    "http://api.us.socrata.com/api/catalog/v1?domains=${domain}&scroll_id=${scroll_id}&limit=${limit}" %>%
        str_interp() %>%
        httr::GET() %>%
        content("text") %>%
        fromJSON(flatten = TRUE)
}

ut_domain <- 'opendata.utah.gov'

# Initial query without scroll_id
ut_results <- get_socrata_data(ut_domain)
nrow(ut_results$results)

# If results are capped, restart with first digit id using scroll function
if (nrow(ut_results$results) == 10000) {
    
    # Pull out first record starting with digit
    digit_start <- ut_results$results |>
        filter(str_starts(resource.id, '\\d'))
    first_digit_id <- first(sort(digit_start$resource.id))
    
    # Data frame for results
    output <- data.frame()
    
    # Restart query with scroll_socrata function that includes scroll_id
    page <- scroll_socrata(ut_domain,
                           scroll_id = first_digit_id)
    
    output <- bind_rows(output, page$results)
    
    # more pages if necessary
    while (nrow(page$results) == 10000) {
        page <- scroll_socrata(ut_domain,
                               scroll_id = last(page$results$resource.id))
        output <- bind_rows(output, page$results)
    }
}

# check total
nrow(output)

内容的提问来源于stack exchange，提问作者Chris