创建自有WHOIS服务器：如何获取及更新域名所有者数据？

Hey there! I've helped a few folks set up custom WHOIS servers before, so let's walk through your questions step by step to clarify where domain owner data lives, how to legally get it, and the protocols you should use.

一、域名所有者信息到底存在哪？ #

First off, domain registration data isn’t stored in a single global database—it’s split across a hierarchical system:

• TLD Registries (the authoritative players): For example, Verisign manages .com/.net, Public Interest Registry runs .org. These are the official sources for their respective top-level domains. Every domain under their TLD has its core registration records stored here.
• Domain Registrars: When someone registers a domain, they work with a registrar (like GoDaddy or Namecheap). The registrar keeps a copy of the user’s data and syncs it to the corresponding TLD registry.
• WHOIS/RDAP Cache Servers: Public WHOIS services (like ICANN’s) often cache registry data to speed up queries, but this isn’t real-time and shouldn’t be your primary source for a self-hosted server.

二、合法获取数据的途径 #

You’ve got a few legitimate options to build and update your WHOIS database—avoid scraping public WHOIS services (it’s against most terms of service and will get your IP blocked):

1. Registry Data Feeds & RDAP #

Most reputable TLD registries offer two key methods for structured data access:

• RDAP (Registration Data Access Protocol): This is the modern replacement for the old port 43 WHOIS protocol. It uses RESTful APIs and returns structured JSON/XML data, making it easy to parse and integrate. Many registries allow authorized bulk queries and incremental updates (you’ll need to apply for access, which may be free or paid depending on the TLD).
• Data Feeds: Some registries provide full or incremental data feeds (often via FTP or API) that push all new/updated domain records on a regular schedule. This is the most reliable way to keep your self-hosted database up to date, but you’ll need to comply with the registry’s usage terms (including privacy rules like GDPR).

2. Registrar Partnerships #

If you have a formal relationship with a domain registrar, you might be able to negotiate access to their domain data. However, this only covers domains registered through that specific registrar, so it’s limited in scope—great for niche use cases but not a full solution.

3. Public WHOIS/RDAP (for verification only) #

Public WHOIS/RDAP services are fine for checking individual domain records to verify your data, but don’t rely on bulk scraping them. Most registries and public services explicitly prohibit automated bulk queries to prevent abuse.

三、Keeping Your Data Updated #

• Prioritize Incremental Syncs: Instead of pulling the entire database every time, only sync changes (new registrations, updates, expirations). This is faster, uses less bandwidth, and is kinder to the registry’s servers.
• Stay Compliant: Make sure you follow ICANN’s General Registration Data Specification (GRDS) and any regional privacy laws (like GDPR). You’ll need to redact sensitive personal information (like full phone numbers or home addresses) in your WHOIS responses unless the domain owner has opted out of privacy protection.
• Implement Caching: If your server is open to public queries, add a cache layer to reduce the number of requests you send to registries. Just set a reasonable TTL (time-to-live) to balance speed and data freshness (24-48 hours is typical for most domains).

四、Quick Pro Tips #

• Start small: Pick a smaller TLD (like a country-code TLD) to experiment with—their registries often have more flexible access rules and lower barriers to getting data feeds.
• Build with RDAP, not old WHOIS: The old port 43 WHOIS is clunky, unstructured, and being phased out. RDAP is the future, and it’ll make your server easier to maintain.
• Test with a sandbox first: Many registries offer sandbox environments for RDAP/data feeds so you can test your integration without hitting production limits.

内容的提问来源于stack exchange，提问作者P.Carlino