Unity+Vuforia+MediaPipe虚拟试穿:摄像头画面获取异常排查
Unity + Vuforia 获取摄像头画面失败求助
我正在使用Unity搭配Vuforia AR SDK开发虚拟试穿应用。由于Vuforia不支持人体追踪,计划通过WebSocket建立Unity与Python的连接,将Unity端Vuforia的摄像头画面发送至Python,借助MediaPipe处理后返回关键关节坐标用于人体追踪。
目前已成功建立Unity与Python的WebSocket连接,可正常收发数据,但在尝试获取并发送Vuforia摄像头画面时遇到问题。参照Vuforia官方文档编写了名为CameraImageAccess的脚本(挂载到场景中的Vuforia ARCamera上),代码如下:
using UnityEngine; using UnityEngine.UI; using Vuforia; using Image = Vuforia.Image; public class CameraImageAccess : MonoBehaviour { const PixelFormat PIXEL_FORMAT = PixelFormat.RGB888; const TextureFormat TEXTURE_FORMAT = TextureFormat.RGB24; public RawImage RawImage; Texture2D mTexture; bool mFormatRegistered; void Start() { // Register Vuforia Engine life-cycle callbacks: VuforiaApplication.Instance.OnVuforiaStarted += OnVuforiaStarted; VuforiaApplication.Instance.OnVuforiaStopped += OnVuforiaStopped; if (VuforiaBehaviour.Instance != null) VuforiaBehaviour.Instance.World.OnStateUpdated += OnVuforiaUpdated; } void OnDestroy() { // Unregister Vuforia Engine life-cycle callbacks: if (VuforiaBehaviour.Instance != null) VuforiaBehaviour.Instance.World.OnStateUpdated -= OnVuforiaUpdated; VuforiaApplication.Instance.OnVuforiaStarted -= OnVuforiaStarted; VuforiaApplication.Instance.OnVuforiaStopped -= OnVuforiaStopped; if (VuforiaApplication.Instance.IsRunning) { // If Vuforia Engine is still running, unregister the camera pixel format to avoid unnecessary overhead // Formats can only be registered and unregistered while Vuforia Engine is running UnregisterFormat(); } if (mTexture != null) Destroy(mTexture); } /// /// Called each time the Vuforia Engine is started /// void OnVuforiaStarted() { mTexture = new Texture2D(0, 0, TEXTURE_FORMAT, false); // A format cannot be registered if Vuforia Engine is not running RegisterFormat(); } /// /// Called each time the Vuforia Engine is stopped /// void OnVuforiaStopped() { // A format cannot be unregistered after OnVuforiaStopped UnregisterFormat(); if (mTexture != null) Destroy(mTexture); } /// /// Called each time the Vuforia Engine state is updated /// void OnVuforiaUpdated() { var image = VuforiaBehaviour.Instance.CameraDevice.GetCameraImage(PIXEL_FORMAT); // There can be a delay of several frames until the camera image becomes available if (Image.IsNullOrEmpty(image)) return; Debug.Log("\nImage Format: " + image.PixelFormat + "\nImage Size: " + image.Width + " x " + image.Height + "\nBuffer Size: " + image.BufferWidth + " x " + image.BufferHeight + "\nImage Stride: " + image.Stride + "\n"); // Override the current texture by copying into it the camera image flipped on the Y axis // The texture is resized to match the camera image size image.CopyToTexture(mTexture, true); RawImage.texture = mTexture; RawImage.material.mainTexture = mTexture; } /// /// Register the camera pixel format /// void RegisterFormat() { // Vuforia Engine has started, now register camera image format var success = VuforiaBehaviour.Instance.CameraDevice.SetFrameFormat(PIXEL_FORMAT, true); if (success) { Debug.Log("Successfully registered pixel format " + PIXEL_FORMAT); mFormatRegistered = true; } else { Debug.LogError("Failed to register pixel format " + PIXEL_FORMAT + "\n the format may be unsupported by your device;" + "\n consider using a different pixel format."); mFormatRegistered = false; } } /// /// Unregister the camera pixel format /// void UnregisterFormat() { Debug.Log("Unregistering camera pixel format " + PIXEL_FORMAT); VuforiaBehaviour.Instance.CameraDevice.SetFrameFormat(PIXEL_FORMAT, false); mFormatRegistered = false; } }
运行Unity Play模式后,Vuforia初始化正常,脚本开始运行,但始终输出以下日志:
ConfigureVideoBackground UnityEngine.Debug:Log (object) Vuforia.Internal.Utility.UnityLogger:LogInfo (string) Vuforia.Internal.Utility.Log:Info (string) Vuforia.MonoCameraConfiguration:ConfigureVideoBackground (UnityEngine.Matrix4x4) Vuforia.MonoCameraConfiguration:CheckForSurfaceChanges (UnityEngine.Matrix4x4) Vuforia.CameraController:UpdateCamera (UnityEngine.Matrix4x4) Vuforia.Internal.Core.StateProcessor:ProcessState (Vuforia.IVuState,System.Collections.Generic.IDictionary`2<int, Vuforia.Internal.Observers.IObserver>,Vuforia.Internal.Rendering.IVideoBackgroundRenderer,Vuforia.Internal.Rendering.IVBAugmentationRenderer,Vuforia.ICameraController,Vuforia.Internal.Core.IWorldOriginProvider,Vuforia.Internal.Observers.DeviceObserver) Vuforia.Internal.Core.Engine:UpdateState () Vuforia.Internal.Core.Engine:OnUnityUpdate () System.Delegate:DynamicInvoke (object[]) Vuforia.Utility.ExtensionMethods.DelegateHelper:InvokeDelegate (System.Delegate,object[]) Vuforia.Utility.ExtensionMethods.DelegateHelper:InvokeWithExceptionHandling (System.Action) Vuforia.Internal.Core.UnityLifeCycleListener:Update ()
虽然RegisterFormat()函数中显示"Successfully registered pixel format RGB888",确认Vuforia支持该格式,但OnVuforiaUpdated()函数中本该输出的图像信息日志并未出现,说明Vuforia无法获取摄像头图像。我已尝试更新Vuforia SDK、新建3D/AR移动端项目,问题仍未解决,恳请提供排查或解决建议。
内容的提问来源于stack exchange,提问作者omyeh23145
相关产品推荐
相关产品推荐

