如何编程抓取ArcGIS Online地图数据及指定地图弹窗要素数据？

First: Getting the Popup Data from That Specific MapSeries App #

That red feature popup doesn't have to be clicked manually—under the hood, all that data is pulling from an ArcGIS Feature Service. Here's how to grab it programmatically:

1. Locate the underlying Feature Service URL

   • Open the map in your browser, hit F12 to launch DevTools, then switch to the Network tab.
   • Filter requests by typing FeatureServer into the search bar. You’ll spot requests like query?where=... pointing to a URL formatted like https://services.arcgis.com/[unique-id]/arcgis/rest/services/[service-name]/FeatureServer/0.
   • Copy that base Feature Service URL (stop at /FeatureServer/[layer-number]).

2. Fetch all data via the ArcGIS REST API
   ArcGIS has a query endpoint that lets you pull every feature directly, no click simulation needed. Use these key parameters:

   • where=1=1: Returns every feature (no filters applied)
   • outFields=*: Retrieves all fields tied to the features
   • f=json: Gets data in easy-to-parse JSON format

   Here’s a quick Python example using the requests library:

   import requests
   
   # Replace with your actual Feature Service URL
   feature_service_url = "https://services.arcgis.com/xxx/arcgis/rest/services/xxx/FeatureServer/0/query"
   
   params = {
       "where": "1=1",
       "outFields": "*",
       "f": "json"
   }
   
   response = requests.get(feature_service_url, params=params)
   data = response.json()
   
   # Loop through features and extract popup-ready attributes
   for feature in data["features"]:
       popup_data = feature["attributes"]
       print("Popup content:", popup_data)
   

   This will give you all the exact data that appears in those red popups—no manual clicks required.

Second: General Tips for Scraping ArcGIS Online Maps #

Most ArcGIS Online maps rely on Feature Services, Web Maps, or Tile Services. Here’s how to tackle them:

• For Web Maps:

  1. Grab the Web Map’s Item ID from its URL (e.g., in https://arcgis.com/home/item.html?id=abc123, the ID is abc123).
  2. Call the ArcGIS REST API’s item info endpoint: https://www.arcgis.com/sharing/rest/content/items/[item-id]/data?f=json
  3. Parse the response to find the operationalLayers array—each entry includes a url pointing to the underlying Feature Service. Use the same query steps above to pull data from each layer.

• Handling Authenticated Data:
  If the map requires a login, generate an access token first:

  1. Send a POST request to https://www.arcgis.com/sharing/rest/generateToken with parameters like username, password, referer, and f=json.
  2. Include the token as a token parameter in all subsequent requests.

• Critical Best Practices:

  • Always check the data’s license and ArcGIS Online’s terms of service—never scrape data you don’t have permission to use.
  • Respect rate limits: add small delays between requests to avoid overwhelming the API.
  • Watch for pagination: if you don’t get all features at once, look for nextQueryParams in the response to fetch the next batch.

内容的提问来源于stack exchange，提问作者curious_nustian