K8s环境下搭建无主从全实例数据互访Redis集群的问题

Hey there! Let's tackle your two Redis problems and guide you toward your goal of building a masterless Redis cluster where all instances replicate data to each other.

1. Why Your Redis ReplicaSet Pods Keep Restarting After Setting slave-read-only no #

Root Cause #

ReplicaSets are built for stateless workloads where all pods are identical. When you set slave-read-only no without defining a clear master-slave replication topology, here's what breaks:

• Each Redis pod tries to operate without a defined role (master/slave), leading to conflicts as they can't establish valid replication connections.
• Redis requires explicit replicaof (or slaveof in older versions) directives to set up replication. Without this, pods fail to resolve their role and crash-loop repeatedly.
• ReplicaSets don't preserve stable pod identities, so even if you manually set replication, the relationships break when pods restart or scale.

Fix #

Switch to a StatefulSet—it’s designed for stateful services like Redis:

• StatefulSets give pods stable network identities (e.g., redis-0.redis-service.default.svc.cluster.local) and persistent storage, which are critical for maintaining replication links.
• You can configure unique replication rules per pod (based on its index) to build a controlled replication chain.

Sample snippet for a StatefulSet pod template:

containers:
- name: redis
  image: redis:latest
  command: ["redis-server"]
  args:
    - "--appendonly yes"
    - "--slave-read-only no"
    # Let redis-0 be the initial master; other pods replicate from it
    - "{{ if ne .Pod.Name \"redis-0\" }}--replicaof redis-0.redis-service 6379{{ end }}"

2. Why Your App Can't Fetch Master Node Info from Redis Sentinel #

Let's walk through common issues and fixes:

a. Verify Sentinel's Master Detection Status #

First, check if Sentinel has correctly identified the master node. Exec into a Sentinel pod:

kubectl exec -it sentinel-0 -- redis-cli -p 26379

Run this Redis command inside the pod to list monitored masters:

sentinel masters

• If no master shows up, your master-slave replication isn’t working. Check the master pod’s logs for connection errors, and ensure slave pods can reach the master via its stable StatefulSet DNS name.

b. Fix App Client Configuration #

Make sure your app’s Redis client is set up for Sentinel mode with these details:

• Sentinel Service Address: Use the Kubernetes Service name (e.g., sentinel-service.default.svc.cluster.local)—never hardcode pod IPs, as they change.
• Sentinel Port: Default is 26379.
• Master Name: Match the name defined in your Sentinel config (usually mymaster in the official example).

Example Python client setup (using redis-py):

from redis.sentinel import Sentinel

sentinel = Sentinel([('sentinel-service', 26379)], socket_timeout=0.1)
master = sentinel.master_for('mymaster', socket_timeout=0.1)

c. Validate Kubernetes Network & Service Setup #

• Confirm the Sentinel Service uses the correct label selector to target Sentinel pods:

apiVersion: v1
kind: Service
metadata:
  name: sentinel-service
spec:
  ports:
  - port: 26379
    name: sentinel
  selector:
    app: redis-sentinel

• Check if network policies are blocking traffic between your app and Sentinel pods. Temporarily disable policies to rule this out.

3. Building Your Desired Masterless, Full-Replication Redis Cluster #

If you want all instances to hold full data copies with no fixed master-slave hierarchy, Redis’ native master-slave has limitations. Here are two practical options:

Option 1: Bidirectional Master-Slave Replication #

Configure each pod to replicate from every other pod (using StatefulSet for stable identities). Key notes:

• Enable replica-ignore-same-replica yes to prevent loop replication.
• Use diskless replication (repl-diskless-sync yes) to reduce I/O overhead.
• Caveat: This setup can face data consistency issues during network partitions—best for non-critical workloads.

Option 2: Redis Cluster (Scalable Alternative) #

While Redis Cluster uses data sharding (not full replication), it offers automatic failover and scalability. If you prioritize resilience and scale over full data copies on every node, this is the official recommended solution.

内容的提问来源于stack exchange，提问作者veerendra2