Chronicle Queue滚动周期文件首片段被误读为前周期内容问题
关于Chronicle Queue滚动周期切换时重复读取条目的问题
我使用配置为RollCycles.MINUTELY滚动周期的Chronicle Queue,代码中通过ExcerptTailer检测周期切换,重置序列列表并调用TAILER.moveToCycle(currentCycle)方法。当前存在两个滚动周期文件,前一文件结束于seqNum=1001,下一文件起始于seqNum=0。运行时发现下一文件的首个条目被当作前周期内容读取;调用moveToCycle会再次读取该条目,注释此方法则不会重复读取。
使用的代码如下:
public static void main(String[] args){ ChronicleQueue QUEUE = SingleChronicleQueueBuilder.single("./chronicle/roll") .rollCycle(RollCycles.MINUTELY).build(); ExcerptTailer TAILER = QUEUE.createTailer(); ArrayList<Long> seqNums = new ArrayList<>(); //this reads all roll cycles starting from first and carries on to next rollcycle. //busy spinner that spins non-stop trying to read from queue int currentCycle = TAILER.cycle(); System.out.println(TAILER.cycle()); while(true){ //if it moves over to new cycle, start over the sequencing (fresh start for next day) int cycleCheck = TAILER.cycle(); long indexCheck = TAILER.index(); System.out.println(cycleCheck); System.out.println("idx: "+indexCheck); if (currentCycle != cycleCheck){ LOGGER.warn("Changing to new roll cycle, from: "+currentCycle+" to: "+cycleCheck+". Clearing list of size "+seqNums.size()); seqNums.clear(); // this may cause a memory issue see: https://stackoverflow.com/a/6961397/16034206 currentCycle = cycleCheck; TAILER.moveToCycle(currentCycle); cycleCheck = TAILER.cycle(); indexCheck = TAILER.index(); System.out.println("cycle: "+cycleCheck); System.out.println("idx: "+indexCheck); } //TODO:2nd option, on starting the chronicle runner, always move to end, and wait for next day's cycle to start if (TAILER.readDocument(w -> w.read("packet").marshallable( m -> { long seqNum = m.read("seqNum").readLong(); int size = seqNums.size(); if (size > 0){ int idx; if ((idx = seqNums.indexOf(seqNum)) >= 0){ LOGGER.warn("Duplicate seqNum: "+seqNum+" at idx: "+idx); }else{ long previous = seqNums.get(size-1); long gap = seqNum - previous; if (Math.abs(gap) > 1L){ LOGGER.error("sequence gap at seqNum: "+previous+" and "+seqNum+"! Gap of "+gap); } } } seqNums.add(seqNum); System.out.println(m.read("moldUdpHeader").text()); } ))){ ; }else { TAILER.close(); break; } //breaks out from spinner if nothing to be read. //a named tailer could be used to pick up from where is left off. } }
两种情况的输出截图:
- 调用
moveToCycle时的输出:
- 注释
moveToCycle时的输出:
内容的提问来源于stack exchange,提问作者pebble unit
相关产品推荐
相关产品推荐

