Let's break down how to speed up your Solr indexing with 200 million documents on hand—right now you're looking at 3-4 seconds per 100 docs, which adds up way too fast for that volume. Here's what you can do to cut down that time significantly:

• Increase your batch size: 100 docs per batch is pretty small—try bumping it to 500-2000 docs per batch (test to find the sweet spot; too big can cause memory issues). Adjust this in your client code or via Solr's API parameters like batchSize.
• Use commitWithin instead of frequent manual commits: Committing is one of the most expensive operations in Solr. Instead of committing after every batch, set commitWithin=60000 (1 minute) to let Solr handle automatic, batched commits. This drastically reduces the number of disk writes and segment merges during indexing.
• Disable autoCommit temporarily: If you don't need real-time search during indexing, turn off autoCommit entirely in solrconfig.xml—you can re-enable it once indexing is done.

• Trim unnecessary field settings: For fields that don't need to be searched, set indexed=false; for fields you don't need to retrieve in search results, set stored=false. Every extra field you index/store adds overhead, especially with 25-30 fields per doc.
• Simplify analyzers: Complex analyzers (with synonym filters, stemming, etc.) eat up CPU during indexing. For non-text fields (like IDs, dates), use KeywordTokenizer or skip analysis entirely. For text fields, only keep the filters you absolutely need.
• Turn off live docs: If you don't need to fetch individual docs while indexing, set enableLiveDocs=false in solrconfig.xml—this removes overhead from tracking document versions.

• Give Solr enough heap memory: Allocate 50% of your server's RAM to Solr (capped at 31GB, since JVMs lose compressed pointers above 32GB). For example, if your server has 64GB RAM, set SOLR_HEAP=31G in your Solr startup script. More heap means more caching of index segments, reducing disk I/O.
• Upgrade to SSD storage: HDDs are a major bottleneck for indexing—SSD random write speeds are 10-100x faster. If SSD isn't an option, use RAID 0/10 for HDDs to boost throughput.
• Tune JVM garbage collection: Use G1GC with pause time limits to avoid long GC stops. Add these flags to your Solr startup: -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:-UseBiasedLocking (biased locking can hurt performance in high-concurrency scenarios).

• Split your index into shards: 200 million docs is a lot for a single shard. Split them into 10-20 shards (each handling 10-20 million docs) using SolrCloud. This lets you index across multiple nodes in parallel, cutting total time by roughly the number of shards (assuming you have enough hardware).
• Adjust merge policies: Use TieredMergePolicy (Solr's default) and tweak parameters like maxMergeAtOnce and segmentsPerTier to reduce the number of segment merges during indexing. You can also disable auto-merges temporarily and run a manual optimize after indexing (note: optimize is resource-heavy, so do it off-peak).

• Use bulk import tools: Skip custom client code for large imports—use Solr's DataImportHandler (DIH) to pull directly from your database, or the post command-line tool with batch mode. These tools are optimized for high-volume indexing.
• Compress requests: Enable gzip compression on your client to reduce network overhead. Solr supports gzip by default—just set the Accept-Encoding: gzip header and compress your request payloads.
• Parallelize client requests: Run multiple client threads to send batches to Solr concurrently. Match the number of threads to your Solr server's updateHandler.threadPool.size (default is 10; you can bump it to 20-30 if your CPU has enough cores).

Start with the lowest-effort changes first (batch size, commit settings) then move to bigger tweaks like sharding or hardware upgrades. Always test each change incrementally to measure its impact—every environment is a little different!

内容的提问来源于stack exchange，提问作者user3309305