Great question—let's unpack this step by step so you get a clear picture of what's happening and whether this setup works.

1. Will each cluster node store data locally? #

Absolutely. When you configure the RocksDBStateBackend with a file:///tmp/checkpoints path, each TaskManager node will write its own portion of the checkpoint data to its own local /tmp/checkpoints directory. There's no sharing here—each node's checkpoint files are isolated to that machine's filesystem.

2. Can this setup run normally? #

It depends entirely on your cluster setup:

• Single-node cluster (only one TaskManager): This will work fine. All checkpoint data lives on the same machine, so the JobManager can access it when needed for recovery, and there's no cross-node dependency.
• Multi-node cluster: This is a problematic setup that will fail in practice, for several key reasons:
  • Recovery failure: When the JobManager needs to restore from a checkpoint, it needs to access checkpoint files from every TaskManager. But since each file is stored on a different node's local filesystem, the JobManager (and other TaskManagers) can't read files from another node's /tmp/checkpoints path.
  • Data loss risk: If any TaskManager node crashes, its local checkpoint files are lost. This makes the entire checkpoint invalid, since Flink needs all parts of the checkpoint to restore the job state.
  • Incremental checkpoint issues: You've enabled incremental checkpoints with the second true parameter in RocksDBStateBackend. Incremental checkpoints rely on shared storage to track and reuse previous snapshot files. With local filesystems, this sharing isn't possible—each node's incremental snapshots can't be accessed by others, leading to broken checkpoint logic or failures.

Key Recommendation #

For any production multi-node Flink cluster, never use a local filesystem for checkpoint storage. You should always use a distributed, shared filesystem like HDFS, S3, or GCS. These systems provide centralized, accessible storage for all checkpoint data, ensuring fault tolerance and reliable recovery.

If you're just testing on a single machine, using file:///tmp/checkpoints is okay for temporary experiments—but don't rely on it for any clustered or production workloads.

内容的提问来源于stack exchange，提问作者Brutal_JL