Elasticsearch 6.0.1开启skip_duplicates去重报错问题咨询

Hey there, let's sort out this problem you're having with the completion suggester and duplicate suggestions!

Why You're Seeing the Error #

The skip_duplicates option you tried to use is actually a feature that was added in Elasticsearch 6.1.0. Since you're running version 6.0.1, this parameter doesn't exist yet in your version—hence the error when you try to enable it.

Alternative Workarounds for 6.0.1 #

Since upgrading might not be an immediate option, here are a few ways to get unique suggestions:

• Preprocess your data before indexing
  Take care of duplicates at the source. Before sending data to Elasticsearch, use a script or ETL tool to filter out duplicate values in your "suggest" field. This way, only unique entries get indexed, so your suggester will naturally return unique results.

• Deduplicate results in your application
  If you can't modify the input data, handle duplicates after getting the suggester response. In your app code, extract the suggestion values, throw them into a set (which automatically removes duplicates), then return the cleaned-up list to your users. For example, in Python you could do something like:

  # Example: Extract and deduplicate suggestions
  raw_suggestions = response['suggest']['your_suggester_name'][0]['options']
  unique_suggestions = list({opt['text']: opt for opt in raw_suggestions}.values())
  

• Upgrade to Elasticsearch 6.1.0 or newer
  If your environment allows it, upgrading to a version that supports skip_duplicates is the most straightforward fix. Once you're on 6.1+, you can use the parameter exactly as described in the docs. Just make sure to back up your data first and test for any compatibility issues with your existing code.

内容的提问来源于stack exchange，提问作者woshitom