跨词嵌入库相似度对比及媒体意识形态偏差统计检验技术问询

Great questions—let’s break these down clearly for your research needs:

• Short answer: Yes, but with critical caveats.
• The core issue is that word embedding spaces are not inherently comparable unless you align them first. Here’s why:
  • Even if you use the same algorithm, two models trained on different corpora learn vector spaces with arbitrary axis orientations. A word’s vector in Model A might point in a totally unrelated direction than the same word in Model B—raw cosine similarity between these unaligned vectors doesn’t reflect true semantic similarity.
  • To make scores comparable, you need to use procrustes alignment (a statistical method to rotate/scale one embedding space to match the structure of another). Tools like gensim have built-in functions (e.g., gensim.models.keyedvectors.WordEmbeddingsKeyedVectors.align_to) to handle this alignment. Once aligned, cosine similarity scores across the two models become meaningful to compare.

Your research approach is feasible, but you’ll need to address key steps to ensure valid, interpretable results:

• First, align your two models. Even with identical algorithms and parameters, the US and UK corpora will produce unaligned vector spaces. Skipping alignment means any observed difference in similarity scores could stem from random axis orientation, not actual ideological bias. Align first before making any comparisons.
• Second, quantify statistical uncertainty. Cosine similarity is a single point estimate—you need to test if the difference between US and UK scores is statistically significant. Try these methods:
  • Bootstrapping: Resample your corpora (with replacement) dozens/hundreds of times, train a new model each iteration, compute the similarity for your target word pair, then build a confidence interval for the difference between US and UK scores. If the interval doesn’t include zero, you can claim a significant difference.
  • Permutation tests: Shuffle corpus labels (swap US/UK tags for random article subsets), retrain models, and count how often the observed similarity difference occurs by chance. If it’s rare (e.g., p < 0.05), the difference is statistically significant.
• Third, control for confounders. US and UK media differ in more than ideology—like regional dialect, topic focus, or event coverage. To isolate ideological bias:
  • Match your corpora for factors like article length, publication dates, and topic distribution (use topic modeling to ensure both have similar proportions of politics, sports, etc.).
  • Test multiple word pairs tied to your ideological hypothesis, not just one—single pairs can be noisy due to idiosyncratic usage.
• Practical tip for gensim/word2vec/fasttext: Use the same random seed for both model trains to reduce unnecessary variability. Also, ensure your corpora are large enough—small datasets lead to unstable embeddings that don’t reflect true semantic patterns.

内容的提问来源于stack exchange，提问作者SanMelkote