如何在JS中正确计算含连字符、制表符的文本长度？解决前后端差异

问题背景 #

我有一段文本，Notepad和后端统计其长度为1026字符（文本包含连字符），但JavaScript的length属性计算结果为1024字符，导致客户端校验可通过，但后端会拦截，需要修复这一不一致问题。

文本示例 #

London is the capital and largest city of England and the United Kingdom, with a population of just under 9 million.[1] It stands on the River Thames in south-east England at the head of a 50-mile (80 km) estuary down to the North Sea, and has been a major settlement for two millennia.[9] The City of London, its ancient core and financial centre, was founded by the Romans as Londinium and retains its medieval boundaries.[note 1][10] The City of Westminster, to the west of the City of London, has for centuries hosted the national government and parliament. Since the 19th century,[11] the name "London" has also referred to the metropolis around this core, historically split between the counties of Middlesex, Essex, Surrey, Kent, and Hertfordshire,[12] which since 1965 has largely comprised Greater London,[13] which is governed by 33 local authorities and the Greater London Authority.[note 2][14]
As one of the world's major global cities,[15] London exerts a strong influence on its arts, entertainment, fashion,

JavaScript变量存储示例 #

const text =
  'London is the capital and largest city of England and the United Kingdom, with a population of just under 9 million.[1] It stands on the River Thames in south-east England at the head of a 50-mile (80 km) estuary down to the North Sea, and has been a major settlement for two millennia.[9] The City of London, its ancient core and financial centre, was founded by the Romans as Londinium and retains its medieval boundaries.[note 1][10] The City of Westminster, to the west of the City of London, has for centuries hosted the national government and parliament. Since the 19th century,[11] the name "London" has also referred to the metropolis around this core, historically split between the counties of Middlesex, Essex, Surrey, Kent, and Hertfordshire,[12] which since 1965 has largely comprised Greater London,[13] which is governed by 33 local authorities and the Greater London Authority.[note 2][14]\n' +
  '\n' +
  "As one of the world's major global cities,[15] London exerts a strong influence on its arts, entertainment, fashion,";

console.log(String(text).length);

问题原因与修复方案 #

核心原因 #

JavaScript的String.length统计的是UTF-16编码单元的数量，而Notepad和后端（通常基于Unicode码点或UTF-8字节）统计的是Unicode码点数量。差异大概率来自文本中的特殊字符——比如长连字符（—，U+2014）或智能引号这类由两个UTF-16编码单元组成的字符，在JS里会被算成2个长度，但在按码点统计的工具里算1个，最终导致总数差2（1026-1024=2）。

修复方案 #

要让客户端和后端统计逻辑一致，需要让JS按Unicode码点数量计算长度，而非UTF-16编码单元：

• 方案1：使用扩展运算符转换为数组统计

  function getCodePointLength(str) {
    return [...str].length;
  }
  
  console.log(getCodePointLength(text)); // 结果与后端/Notepad一致
  

• 方案2：使用Intl.Segmenter（现代浏览器支持）

  function getCodePointLength(str) {
    const segmenter = new Intl.Segmenter('en', { granularity: 'grapheme' });
    return Array.from(segmenter.segment(str)).length;
  }
  

• 方案3：兼容旧环境的正则表达式

  function getCodePointLength(str) {
    // 匹配所有Unicode码点，包括代理对
    const regex = /[\s\S]/gu;
    return (str.match(regex) || []).length;
  }
  

额外注意事项 #

• 如果后端是按UTF-8字节数统计，需要转换为统计UTF-8字节长度：

  function getUTF8ByteLength(str) {
    return new TextEncoder().encode(str).length;
  }
  

• 客户端校验时直接使用上述自定义函数替代String.length，确保和后端规则完全对齐。

内容的提问来源于stack exchange，提问作者DevOverflow