Lucene.Net调用J2N HashSet AddIfNotPresent时出现诡异NullReferenceException

该NullReferenceException异常并非频繁出现，难以复现，但通常在IIS进程从空闲状态恢复后触发（启动程序后闲置一段时间，再访问涉及Lucene.Net搜索的页面时出现）。对IIS相关知识不太熟悉，希望能得到解决帮助。

异常堆栈 #

Exception: System.NullReferenceException: Object reference not set to an instance of an object.
   at J2N.Collections.Generic.HashSet`1.AddIfNotPresent(T value)
   at Lucene.Net.Index.IndexReader.SubscribeToGetCacheKeysEvent(GetCacheKeysEvent getCacheKeysEvent)
   at Lucene.Net.Search.FieldCacheImpl.Cache`2.Get(AtomicReader reader, TKey key, Boolean setDocsWithField)
   at Lucene.Net.Search.FieldCacheImpl.GetInt64s(AtomicReader reader, String field, IInt64Parser parser, Boolean setDocsWithField)
   at Lucene.Net.Search.FieldComparer.Int64Comparer.SetNextReader(AtomicReaderContext context)
   at Lucene.Net.Search.TopFieldCollector.OneComparerNonScoringCollector.SetNextReader(AtomicReaderContext context)
   at Lucene.Net.Search.IndexSearcher.Search(IList`1 leaves, Weight weight, ICollector collector)

调用Lucene API的代码 #

var sectionResults = await Task.WhenAll(sectionsData
  .Select(data => lucene.ExploreQuery(data.Sorting, data.PageTypeFilter, null, data.CategoryId, null, null, null, null, data.Keyword))
  .Select(f => lucene.RetrieveResults(1, 4, f.Sort, null, f.Filter)));

初步怀疑 #

怀疑可以通过增加应用池空闲超时时间来降低异常发生频率。

查看AddIfNotPresent方法 #

已经查看了AddIfNotPresent方法，但不清楚运行时哪个引用为空，且该方法属于库代码无法修改：

private bool AddIfNotPresent(T value)
{
    if (_buckets == null)
    {
        Initialize(0);
    }

    int hashCode;
    int bucket;
    int collisionCount = 0;
    Slot[] slots = _slots;

    IEqualityComparer<T>? comparer = _comparer;

    if (comparer == null)
    {
        hashCode = value == null ? 0 : InternalGetHashCode(value.GetHashCode());
        bucket = hashCode % _buckets!.Length;

        if (default(T)! != null) // TODO-NULLABLE: default(T) == null warning (https://github.com/dotnet/roslyn/issues/34757)
        {
            for (int i = _buckets[bucket] - 1; i >= 0; i = slots[i].next)
            {
                if (slots[i].hashCode == hashCode && EqualityComparer<T>.Default.Equals(slots[i].value, value))
                {
                    return false;
                }

                if (collisionCount >= slots.Length)
                {
                // The chain of entries forms a loop, which means a concurrent update has happened.
                    throw new InvalidOperationException(SR.InvalidOperation_ConcurrentOperationsNotSupported);
                }
                collisionCount++;
            }
        }
        else
        {
        // Object type: Shared Generic, EqualityComparer<TValue>.Default won't devirtualize
        // https://github.com/dotnet/coreclr/issues/17273
        // So cache in a local rather than get EqualityComparer per loop iteration
            IEqualityComparer<T> defaultComparer = EqualityComparer<T>.Default;

            for (int i = _buckets[bucket] - 1; i >= 0; i = slots[i].next)
            {
                if (slots[i].hashCode == hashCode && defaultComparer.Equals(slots[i].value, value))
                {
                    return false;
                }

                if (collisionCount >= slots.Length)
                {
                // The chain of entries forms a loop, which means a concurrent update has happened.
                    throw new InvalidOperationException(SR.InvalidOperation_ConcurrentOperationsNotSupported);
                }
                collisionCount++;
            }
        }
    }
    else
    {
        hashCode = value == null ? 0 : InternalGetHashCode(comparer.GetHashCode(value));
        bucket = hashCode % _buckets!.Length;

        for (int i = _buckets[bucket] - 1; i >= 0; i = slots[i].next)
        {
            if (slots[i].hashCode == hashCode && comparer.Equals(slots[i].value, value))
            {
                return false;
            }

            if (collisionCount >= slots.Length)
            {
                // The chain of entries forms a loop, which means a concurrent update has happened.
                    throw new InvalidOperationException(SR.InvalidOperation_ConcurrentOperationsNotSupported);
            }
            collisionCount++;
        }
    }

    int index;
    if (_freeList >= 0)
    {
        index = _freeList;
        _freeList = slots[index].next;
    }
    else
    {
        if (_lastIndex == slots.Length)
        {
            IncreaseCapacity();
            // this will change during resize
            slots = _slots;
            bucket = hashCode % _buckets.Length;
        }
        index = _lastIndex;
        _lastIndex++;
    }
    slots[index].hashCode = hashCode;
    slots[index].value = value;
    slots[index].next = _buckets[bucket] - 1;
    _buckets[bucket] = index + 1;
    _count++;
    _version++;

    return true;
}

更新：找到问题根源 #

Tim Schmelter的建议非常有用。Lucene.Net的IndexReader.cs中的以下代码显示，线程不安全的实例变量getCacheKeysEvents会在多线程上下文下导致异常：

[ExcludeFromRamUsageEstimation]
private readonly ISet<WeakEvents.GetCacheKeysEvent> getCacheKeysEvents = new JCG.HashSet<WeakEvents.GetCacheKeysEvent>();
internal void SubscribeToGetParentReadersEvent(WeakEvents.GetParentReadersEvent getParentReadersEvent)
{
    if (getParentReadersEvent is null)
        throw new ArgumentNullException(nameof(getParentReadersEvent));
    if (getParentReadersEvents.Add(getParentReadersEvent))
        getParentReadersEvent.Subscribe(OnGetParentReaders);
}

internal void SubscribeToGetCacheKeysEvent(WeakEvents.GetCacheKeysEvent getCacheKeysEvent)
{
    if (getCacheKeysEvent is null)
        throw new ArgumentNullException(nameof(getCacheKeysEvent));
    if (getCacheKeysEvents.Add(getCacheKeysEvent))
        getCacheKeysEvent.Subscribe(OnGetCacheKeys);
} 

内容的提问来源于stack exchange，提问作者Ryan Xu