多组合匹配成绩数据集:Stata/Python/R实现方案咨询
问题描述
我有两个不完整的成绩数据集,需要通过匹配规则关联它们:
- Dataset 1(学生平均成绩表):
| id1 | AverageExam | AverageInternal |
|---|---|---|
| 1 | 198.7 | 180 |
| 2 | 154.7 | 170 |
注:
AverageExam是学生的(数学成绩+物理成绩+生物成绩)/3,对应数据来自Dataset 2
- Dataset 2(科目成绩记录表,无学生ID):
| schoolid | ExamType | ExamGrade |
|---|---|---|
| 1 | Maths | 200 |
| 1 | Maths | 180 |
| 1 | Physics | 200 |
| 1 | Physics | 180 |
| 1 | Biology | 196 |
| 1 | Biology | 180 |
我的思路是:在每个schoolid下,计算三科成绩所有可能组合的平均值,再将Dataset 1的记录与Dataset 2的schoolid匹配。我更熟悉Stata,但之前尝试循环函数没成功,想知道:
- 能否用Stata实现这个需求?
- 如果Stata不合适,Python或R是否更适合?
解决方案
一、Stata实现方法
完全可以用Stata实现,核心是用cross命令生成所有成绩组合,再计算平均值后匹配,具体步骤如下:
1. 拆分Dataset 2为单科目数据集
先将Dataset 2按科目拆分,生成三个独立的科目成绩文件:
* 提取并保存数学成绩 preserve keep if ExamType == "Maths" rename ExamGrade MathsGrade save "maths.dta", replace restore * 提取并保存物理成绩 preserve keep if ExamType == "Physics" rename ExamGrade PhysicsGrade save "physics.dta", replace restore * 提取并保存生物成绩 preserve keep if ExamType == "Biology" rename ExamGrade BiologyGrade save "biology.dta", replace restore
2. 生成所有成绩组合并计算平均值
通过cross命令按schoolid交叉合并三个科目数据集,生成所有可能的成绩组合,再计算对应平均成绩:
* 合并数学与物理成绩 use "maths.dta", clear cross using "physics.dta", common(schoolid) * 再合并生物成绩 cross using "biology.dta", common(schoolid) * 计算AverageExam gen AverageExam = (MathsGrade + PhysicsGrade + BiologyGrade)/3 * 保留必要变量 keep schoolid MathsGrade PhysicsGrade BiologyGrade AverageExam save "all_combinations.dta", replace
3. 匹配Dataset 1与组合数据集
加载Dataset 1,根据schoolid和AverageExam与组合数据集匹配(如果存在浮点精度误差,可先用round(AverageExam, 0.1)处理后再匹配):
use "dataset1.dta", clear * 精确匹配,仅保留两边都匹配的记录 merge m:1 schoolid AverageExam using "all_combinations.dta", keep(match)
二、Python实现方法(备选)
如果数据集规模较大,Python的pandas处理组合生成和匹配会更高效灵活:
import pandas as pd # 读取数据集 df1 = pd.read_csv("dataset1.csv") df2 = pd.read_csv("dataset2.csv") # 拆分各科目成绩 maths = df2[df2["ExamType"] == "Maths"].rename(columns={"ExamGrade": "MathsGrade"}) physics = df2[df2["ExamType"] == "Physics"].rename(columns={"ExamGrade": "PhysicsGrade"}) biology = df2[df2["ExamType"] == "Biology"].rename(columns={"ExamGrade": "BiologyGrade"}) # 按schoolid生成所有成绩组合 combinations = pd.merge(maths, physics, on="schoolid") combinations = pd.merge(combinations, biology, on="schoolid") # 计算平均成绩 combinations["AverageExam"] = (combinations["MathsGrade"] + combinations["PhysicsGrade"] + combinations["BiologyGrade"])/3 # 匹配Dataset 1 result = pd.merge(df1, combinations, on=["schoolid", "AverageExam"], how="inner")
三、R实现方法(备选)
R的tidyverse套件可以通过宽表转换+行展开快速生成组合:
library(dplyr) library(tidyr) # 读取数据 df1 <- read.csv("dataset1.csv") df2 <- read.csv("dataset2.csv") # 生成所有成绩组合并计算平均值 combinations <- df2 %>% pivot_wider(names_from = ExamType, values_from = ExamGrade, values_fn = list) %>% unnest(c(Maths, Physics, Biology)) %>% mutate(AverageExam = (Maths + Physics + Biology)/3) # 匹配Dataset 1 result <- inner_join(df1, combinations, by = c("schoolid", "AverageExam"))
内容的提问来源于stack exchange,提问作者Nuno Can
相关产品推荐
相关产品推荐

