如何解读Word2Vec权重的形状？及Keras嵌入层相关技术疑问

Hey there! Let's unpack your questions one by one to clear up the confusion:

1. What does the 1350 in pretrained_weights.shape = (1350, 100) mean? #

That 1350 is simply the total number of unique words in your Word2Vec vocabulary.

• When you trained your Word2Vec model with min_count=1, it included every word that appeared at least once in your sentences dataset.
• Each row in pretrained_weights corresponds to the 100-dimensional vector for one unique word.
• You can confirm this by running:

  print(len(word_model.wv.vocab))  # This should output 1350
  

So pretrained_weights is just a matrix where each row index maps to the word's position in the Word2Vec vocabulary (which is exactly what word_model.wv.vocab['test'].index returns).

2. Why does keras_model.predict(...) return a 1350-dimensional vector instead of your expected 7200-dimensional one? #

This comes down to understanding your Keras model's structure and what each layer outputs:

First, let's recap the Embedding layer's core role:

• input_dim=vocab_size (1350) defines the range of possible input word indices (0 to 1349).
• output_dim=100 means each input index gets converted to a 100-dimensional vector (matching your Word2Vec vectors).

If your full model is supposed to output a 7200-dimensional vector (e.g., for a multi-class task with 7200 categories), your model must have additional layers after the Embedding layer that transform the 100-dimensional word vector into the 7200-dimensional final output.

The reason you're seeing a 1350-dimensional output right now is likely one of these scenarios:

• You're testing a model that only includes the Embedding layer plus an extra Dense(1350) layer (double-check your model definition!).
• You might have misinterpreted the output shape. For example, if the output is (1, 1350), that's a batch of 1 sample with 1350 features—not a 1350-length vector in the way you're thinking.

Quick checks to verify: #

1. Print your model's full structure to see all layers and their output shapes:

   keras_model.summary()
   

2. Compare the Word2Vec vector and the Embedding layer's output for the same word—they should be nearly identical (since you loaded the pretrained weights):

   # Get Word2Vec vector for 'test'
   w2v_vector = word_model.wv['test']
   # Get Embedding layer output for 'test'
   keras_vector = keras_model.predict(np.array([word_model.wv.vocab['test'].index])).flatten()
   # Check similarity
   print(np.allclose(w2v_vector, keras_vector))  # Should return True
   

Once you confirm your model structure, you'll see that the 7200-dimensional output only comes from the final layer of your complete model—you're just seeing an intermediate layer's output right now.

内容的提问来源于stack exchange，提问作者Logan Wilson