如何使用JavaScript对视频而非摄像头运行Teachable Machine姿态估计推理

核心错误原因 #

• 原生HTML <video> 元素不存在canvas属性，你替换的video.canvas实际值为undefined，传入model.estimatePose方法后触发了读取高度属性的报错，只有Teachable Machine封装的Webcam实例才自带canvas属性。
• model.estimatePose本身原生支持直接传入video、img、canvas类的DOM元素作为输入，不需要额外套canvas。

分步修改点 #

1. 修正预测方法入参 #

修改predict函数中调用estimatePose的代码，直接传入video元素：

const { pose, posenetOutput } = await model.estimatePose(video);

2. 补全canvas宽高配置 #

在init函数中给canvas设置和video一致的宽高，避免绘制错位：

const canvas = document.getElementById("canvas");
canvas.width = 540;
canvas.height = 360;
ctx = canvas.getContext("2d");

3. 修正姿态绘制逻辑 #

修改drawPose方法，直接将当前视频帧绘制到canvas上，不需要访问不存在的video.canvas：

function drawPose(pose) {
    ctx.drawImage(video, 0, 0, canvas.width, canvas.height);
    if (pose) {
        const minPartConfidence = 0.5;
        tmPose.drawKeypoints(pose.keypoints, minPartConfidence, ctx);
        tmPose.drawSkeleton(pose.keypoints, minPartConfidence, ctx);
    }
}

4. 补全分类结果展示逻辑 #

原来的代码只生成了分类文本没有写入页面，修改predict中的循环逻辑：

for (let i = 0; i < maxPredictions; i++) {
    const classPrediction =
        prediction[i].className + ": " + prediction[i].probability.toFixed(2);
    labelContainer.childNodes[i].innerHTML = classPrediction;
}

5. 优化循环调用（可选） #

你同时绑定了video的play事件触发loop，又手动调用了requestAnimationFrame(loop)，会导致重复执行循环，删掉video.addEventListener('play', loop )这一行即可。

修改后完整代码 #

<video id="video" style="width: 540px; height: 360px;" muted>
    <source src="GTSolo2.mp4" type="video/mp4" />
</video>
<div>Teachable Machine Pose Model</div>
<br>
<button type="button" onclick="init()">Start</button>
<div><canvas id="canvas"></canvas></div>
<div id="label-container"></div>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@1.3.1/dist/tf.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@teachablemachine/pose@0.8/dist/teachablemachine-pose.min.js"></script>
<!-- AJA adds the osc lib-->
<script type="text/javascript" src="node_modules/osc-js/lib/osc.min.js"></script>
<script type="text/javascript">
    const URL = "https://teachablemachine.withgoogle.com/models/rpV37-6o2/";
    let model, ctx, labelContainer, maxPredictions, video;

    async function init() {
        const modelURL = URL + "model.json";
        const metadataURL = URL + "metadata.json";
        model = await tmPose.load(modelURL, metadataURL);
        maxPredictions = model.getTotalClasses();

        video = document.getElementById("video");
        video.width = 540;
        video.height = 360;
        await video.play();
        window.requestAnimationFrame(loop);

        const canvas = document.getElementById("canvas");
        canvas.width = 540;
        canvas.height = 360;
        ctx = canvas.getContext("2d");
        labelContainer = document.getElementById("label-container");
        for (let i = 0; i < maxPredictions; i++) {
            labelContainer.appendChild(document.createElement("div"));
        }
    }

    async function loop(timestamp) {
        await predict();
        window.requestAnimationFrame(loop);
    }

    async function predict() {
        const { pose, posenetOutput } = await model.estimatePose(video);
        const prediction = await model.predict(posenetOutput);

        for (let i = 0; i < maxPredictions; i++) {
            const classPrediction =
                prediction[i].className + ": " + prediction[i].probability.toFixed(2);
            labelContainer.childNodes[i].innerHTML = classPrediction;
        }
        drawPose(pose);
    }

    function drawPose(pose) {
        ctx.drawImage(video, 0, 0, canvas.width, canvas.height);
        if (pose) {
            const minPartConfidence = 0.5;
            tmPose.drawKeypoints(pose.keypoints, minPartConfidence, ctx);
            tmPose.drawSkeleton(pose.keypoints, minPartConfidence, ctx);
        }
    }
</script>

内容的提问来源于stack exchange，提问作者Eden Evans