Great question—this is a common deep dive for folks who want to go beyond just using Word2Vec and understand its core mechanics. Let’s formalize this clearly, since there’s a standard, rigorous definition tied directly to how word embeddings operate.

Core Foundation #

First, recall that Word2Vec transforms every word ( w ) in its vocabulary into a dense, real-valued vector ( \mathbf{v}_w \in \mathbb{R}^d ) (where ( d ) is the embedding’s dimension, e.g., 100 or 300). The most_similar function’s job is to find the top ( n ) words whose vectors are closest to a target reference vector.

Formal Rigorous Definition #

Let’s start with the simplest case: you have a reference word ( w_{\text{ref}} ), and you want the ( n ) most similar words.

First, define your target query vector ( \mathbf{q} )—for a single reference word, this is just its embedding: ( \mathbf{q} = \mathbf{v}{w{\text{ref}}} ).

The function returns the set:
[
\text{most_similar}(\mathbf{q}, n) = \left{ w_i \in \mathcal{V} \mid w_i \neq w_{\text{ref}}, \text{rank}(\mathbf{v}_{w_i}) \leq n \right}
]

Here’s what each part means:

• ( \mathcal{V} ): The full vocabulary of words the Word2Vec model was trained on.
• ( \text{rank}(\mathbf{v}{w_i}) ): The position of ( \mathbf{v}{w_i} ) when all vocabulary vectors are sorted in descending order of their similarity to ( \mathbf{q} ).
• The similarity metric used 99% of the time (and the default in libraries like Gensim) is cosine similarity:
  [
  \text{cosine}(\mathbf{q}, \mathbf{v}{w_i}) = \frac{\mathbf{q} \cdot \mathbf{v}{w_i}}{|\mathbf{q}|2 \cdot |\mathbf{v}{w_i}|_2}
  ]

Edge Cases & Extensions #

• Excluding the reference word: Most implementations automatically remove the input word from results (hence ( w_i \neq w_{\text{ref}} ))—since a word is trivially most similar to itself.
• Analogy tasks: When you use most_similar for analogies (e.g., "king - man + woman"), the query vector becomes a linear combination: ( \mathbf{q} = \mathbf{v}{\text{king}} - \mathbf{v}{\text{man}} + \mathbf{v}_{\text{woman}} ). The same cosine similarity ranking applies to find the closest match.
• Alternative metrics: While rare, some variants might use Euclidean distance, but cosine similarity is preferred because it normalizes for vector magnitude. This focuses on the direction of vectors, which aligns better with Word2Vec’s training goal (semantically similar words have aligned vectors).

Why This Is the Standard #

Word2Vec’s training objectives (CBOW or Skip-Gram) optimize vectors so that words appearing in similar contexts end up close in the embedding space. Cosine similarity is the natural fit here because it measures the angle between vectors—smaller angles mean more aligned semantic meanings, regardless of how "large" the vectors are.

内容的提问来源于stack exchange，提问作者Alexei