Apache Ignite集群部署问题咨询：替换Coherence缓存后的异常排查

Hey there! Let’s work through your Apache Ignite issues one by one to get your cache running reliably, plus figure out that database schema migration plan.

Fixing Distributed Cluster Startup Issues #

First off, your current discovery configuration has a critical mismatch:
You’re using TcpDiscoveryMulticastIpFinder but manually listing server IPs — this IpFinder is designed for multicast-based automatic node discovery, not static IP lists. If your environment blocks multicast (common in cloud/enterprise networks) or you need fixed node addresses, switch to TcpDiscoveryStaticIpFinder instead.

Correct Static Discovery Configuration #

<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
    <property name="ipFinder">
        <bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.static.TcpDiscoveryStaticIpFinder">
            <property name="addresses">
                <list>
                    <!-- Format: [host]:[port] — default discovery port is 47500 -->
                    <value>1xx.xxx.xx.xxx:47500</value>
                    <value>1xx.xxx.xx.xxx:47500</value>
                    <value>1xx.xxx.xx.xxx:47500</value>
                    <value>1xx.xxx.xx.xxx:47500</value>
                </list>
            </property>
        </bean>
    </property>
</bean>

Also, verify:

• All servers have ports 47500 (discovery) and 47100 (data communication) open in firewalls.
• Check Ignite startup logs on failing nodes for errors like "Failed to join cluster" or connection timeouts — these will pinpoint exact network issues.

Troubleshooting Single Node Crash After 2 Days #

Most unexpected stops stem from resource limits or misconfiguration. Here’s what to check:

• Check Ignite logs: Look for OOM errors, GC overload, or unhandled exceptions in work/log/ignite.log right before the crash.
• Tune JVM parameters: Ignite needs sufficient heap memory — set -Xms8G -Xmx8G (adjust based on your data size) and use -XX:+UseG1GC to avoid long GC pauses that can kill the process.
• Check system resources: Verify if the OS’s OOM killer terminated the Ignite process (look for logs in /var/log/syslog or /var/log/messages). This happens when the server runs out of physical memory.
• Review cache settings: Ensure you haven’t set an aggressive expiryPolicy or evictionPolicy that accidentally triggers node shutdown, and confirm the node is running in server mode (not client mode, which stops if no servers are available).

Migrating Database Schema to Ignite Cache #

You have two reliable approaches to sync your database schema and data with Ignite:

1. Cache Store (Read/Write-Through) #

This keeps cache and database in sync automatically. Implement a CacheStore to handle data loading/writing between Ignite and your DB.

@Bean
public CacheConfiguration<String, YourEntity> cacheConfig() {
    CacheConfiguration<String, YourEntity> cfg = new CacheConfiguration<>("your-db-cache");
    // Link to your custom CacheStore implementation
    cfg.setCacheStoreFactory(FactoryBuilder.factoryOf(YourDbCacheStore.class));
    cfg.setReadThrough(true); // Auto-load missing data from DB
    cfg.setWriteThrough(true); // Auto-sync cache writes to DB
    cfg.setWriteBehindEnabled(true); // Optional: Async writes for better performance
    return cfg;
}

Create YourDbCacheStore by extending CacheStoreAdapter and overriding load(), write(), and delete() methods to interact with your database.

2. Bulk Data Loading with Data Streamer #

For initial schema migration, use IgniteDataStreamer to batch-load all existing DB data into Ignite:

@Autowired
private Ignite ignite;

@PostConstruct
public void loadInitialData() {
    try (IgniteDataStreamer<String, YourEntity> streamer = ignite.dataStreamer("your-db-cache")) {
        streamer.perNodeBufferSize(2048); // Tune batch size for speed
        // Fetch all data from your database repository
        List<YourEntity> dbEntities = yourDatabaseRepository.findAll();
        dbEntities.forEach(entity -> streamer.addData(entity.getId(), entity));
    }
}

To map your schema for Ignite SQL queries, define a table matching your DB schema:

CREATE TABLE IF NOT EXISTS YourEntity (
    id VARCHAR PRIMARY KEY,
    column1 VARCHAR,
    column2 INT
) WITH "template=partitioned, cache_name=your-db-cache";

Best Practices for Long-Term Ignite Stability #

• Enable Native Persistence: Prevents data loss on node restarts and reduces memory pressure by offloading data to disk:

  <bean class="org.apache.ignite.configuration.IgniteConfiguration">
      <property name="dataStorageConfiguration">
          <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
              <property name="defaultDataRegionConfiguration">
                  <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
                      <property name="persistenceEnabled" value="true"/>
                  </bean>
              </property>
          </bean>
      </property>
  </bean>
  

• Monitor Nodes: Use JMX or Ignite’s built-in metrics to track memory usage, GC activity, and cluster health.
• Configure Failure Detection: Set failureDetectionTimeout in your Ignite config to let the cluster quickly identify and recover from dead nodes.

内容的提问来源于stack exchange，提问作者sri