基于Invisible XML拆分解析期刊书目OCR文本条目的技术问询

问题背景 #

我手里有一份OCR识别的期刊书目文本，里面的条目是结构化的，想通过Invisible XML标准提取并解析这些条目。每个条目的规则很明确：以条目编号开头，后面跟一个或多个空格，再是被换行拆分的描述文本。

示例输入 #

1  2  Hype.  1990?- 1993.  Frequency:  Bimonthly.  River  Edge, 

NJ.  Published  by  Word  Up!  Video,  Inc.  Last  issue  66  pages. 
Height  28  cm.  Line  drawings;  Photographs  (some  in  color); 
Commercial  advertising;  Table  of  contents.  Previous  editor(s): 
Marica  A.  Cole.  ISSN  1056-4632.  LC  card  no.  sn91-1965. 
OCLC  no.  23715422.  Subject  focus  and/or  Features:  Hip  hop 
culture,  Music,  Rap  music. 

WHi  v.l,  n.6;  v.2,  n.5  Pam  01-5450  Aug,  1992;  Aug,  1993 

6561  The  Zora  Neale  Hurston  Forum.  1986-.  Frequency: 
Semiannual.  Ruth  T.  Sheffey,  Editor,  The  Zora  Neale  Hurston 
Forum,  P.O.  Box  550,  Morgan  State  University,  Baltimore, 

MD  21239.  $15  for  individuals  and  institutions.  Telephone: 
(301)  444-3435.  Published  by  Zora  Neale  Hurston  Society. 

Last  issue  69  pages.  Last  volume  142  pages.  Height  23  cm. 
Photographs;  Table  of  contents.  ISSN  1051-6867.  LC  card  no. 
90-649339.  OCLC  no.  15610848.  Subject  focus  and/or  Features:  Hurston,  Zora  Neale,  Literature,  Literary  criticism. 
MdBMC  v.l,  n.l-v.8,  n.2  Special  Collections  Fall,  1986-Spring, 

1994 

TxDw  v.l,  n.l;  v.2,  n.l  Woman’s  Collection  Fall,  1986;  Fall,  1987 
WU  v.l,  n.l-  AP/Z893/N345  Fall,  1986
6562  Zwanna:  Son  of  Zulu.  1993-.  Frequency:  Unknown. 
Nabile  P.  Hage,  Editor,  Zwanna,  P.O.  Box  38261,  Atlanta,  GA 
30334.  Published  by  Dark  Zulu  Lies  Comics,  Inc.  Last  issue  32 
pages.  Height  28  cm.  Line  drawings  (some  in  color);  Commercial  advertising.  OCLC  no.  28389961.  Subject  focus  and/or 
Features:  Comic  books,  strips,  etc. 

WHi  v.l,  n.l  Pam  00-305  Apr/May,  1993 

初始Invisible XML语法 #

一开始我写了下面的语法，但用CoffeePot处理器运行时出现了歧义：

data: entry+ .
entry: -#a, entrynum, " "+, content .
entrynum: -digit+ .
digit: ["1"-"9"] .
content: ~[]+; -#a+ .

歧义解析结果 #

错误的解析把本该属于新条目的6561当成了第一个条目的内容，导致条目拆分完全混乱：

<data xmlns:ixml="http://invisiblexml.org/NS" ixml:state="ambiguous">
  <entry>
    <entrynum>1</entrynum>
    <content>2 Hype. 1990?- 1993. Frequency: Bimonthly. River Edge, NJ. Published by Word Up! Video,
      Inc. Last issue 66 pages. Height 28 cm. Line drawings; Photographs (some in color); Commercial
      advertising; Table of contents. Previous editor(s): Marica A. Cole. ISSN 1056-4632. LC card
      no. sn91-1965. OCLC no. 23715422. Subject focus and/or Features: Hip hop culture, Music, Rap
      music. WHi v.l, n.6; v.2, n.5 Pam 01-5450 Aug, 1992; Aug, 1993 6561 The Zora Neale Hurston
      Forum. 1986-. Frequency: Semiannual. Ruth T. Sheffey, Editor, The Zora Neale Hurston Forum,
      P.O. Box 550, Morgan State University, Baltimore, MD 21239. $15 for individuals and
      institutions. Telephone: (301) 444-3435. Published by Zora Neale Hurston Society. Last issue
      69 pages. Last volume 142 pages. Height 23 cm. Photographs; Table of contents. ISSN 1051-6867.
      LC card no. 90-649339. OCLC no. 15610848. Subject focus and/or Features: Hurston, Zora Neale,
      Literature, Literary criticism. MdBMC v.l, n.l-v.8, n.2 Special Collections Fall, 1986-Spring,
      1994 TxDw v.l, n.l; v.2, n.l Woman’s Collection Fall, 1986; Fall, 1987 WU v.l, n.l-
      AP/Z893/N345 Fall, 1986</content>
  </entry>
  <entry>
    <entrynum>6562</entrynum>
    <content>Zwanna: Son of Zulu. 1993-. Frequency: Unknown. Nabile P. Hage, Editor, Zwanna, P.O.
      Box 38261, Atlanta, GA 30334. Published by Dark Zulu Lies Comics, Inc. Last issue 32 pages.
      Height 28 cm. Line drawings (some in color); Commercial advertising. OCLC no. 28389961.
      Subject focus and/or Features: Comic books, strips, etc. WHi v.l, n.l Pam 00-305 Apr/May, 1993
    </content>
  </entry>
</data>

解决方案 #

1. 先解决条目拆分的歧义 #

问题出在初始语法没限定条目编号必须出现在行首。修改语法，明确新条目是行首的数字序列，让内容在遇到下一个行首编号时停止：

data: entry+ .
entry: -#a, entrynum, " "+, content .
entrynum: -[1-9]+ .
content: (~[] - entry-start)+; -#a+ .
entry-start: -#a, [1-9]+, " "+ .

• entry-start定义了新条目的起始模式：行首的数字+空格
• content匹配所有内容，直到遇到entry-start，这样就不会把下一个条目的编号包含进来

2. 进一步结构化解析内容 #

接下来可以把条目内容拆分成具体字段，比如标题、出版时间、频率、馆藏信息等，让解析结果更有用：

data: entry+ .
entry: -#a, entrynum, " "+, title, ". ", details+ .
entrynum: -[1-9]+ .
title: (~[] - ".")+ .

details: date-range | frequency | publisher | physical-info | identifiers | subjects | holdings .

date-range: (~[] - ".")+, ". " .
frequency: "Frequency: ", (~[] - ".")+, ". " .
publisher: "Published by ", (~[] - ".")+, ". " .
physical-info: ("Last issue " | "Height " | "Line drawings" | "Photographs")+, (~[] - ".")+, ". " .
identifiers: ("ISSN " | "LC card no. " | "OCLC no. "), (~[] - ".")+, ". " .
subjects: "Subject focus and/or Features: ", (~[] - ".")+, ". " .
holdings: (~[] - (entry-start | "."))+ .

这个语法会把每个条目拆解成<title>、<frequency>、<publisher>等结构化元素，而不是把所有内容塞进一个<content>里。

预期解析效果 #

修改后的语法会正确拆分所有条目，并且输出结构化的XML，比如：

<data>
  <entry>
    <entrynum>1</entrynum>
    <title>2  Hype</title>
    <date-range>1990?- 1993</date-range>
    <frequency>Frequency: Bimonthly</frequency>
    <publisher>Published by Word Up! Video, Inc</publisher>
    <!-- 其他字段会依次解析出来 -->
  </entry>
  <entry>
    <entrynum>6561</entrynum>
    <title>The Zora Neale Hurston Forum</title>
    <date-range>1986-</date-range>
    <!-- 其他字段... -->
  </entry>
</data>

内容的提问来源于stack exchange，提问作者tat