如何通过Vuforia SDK获取设备相机的亮度与光强？

Hey there! Great question—Vuforia doesn’t expose a direct out-of-the-box API for grabbing camera luminance or light intensity, but you can absolutely pull this off by combining Vuforia’s camera callbacks with native platform code. Let me break this down for both Android and iOS, the main targets for AR apps.

Android Implementation #

Vuforia lets you access the underlying camera session, so you can hook into preview frames and extract luminance data directly from the frame buffer. Here's how:

• Register a Frame Observer
  First, set up Vuforia to deliver YUV format frames (since the Y channel holds luminance data, it’s more efficient than RGB). Then register a frame observer to catch each frame:

  CameraDevice.getInstance().setFrameFormat(Frame.PIXEL_FORMAT.YUV, true);
  CameraDevice.getInstance().registerFrameObserver(this, Frame.PIXEL_FORMAT.YUV);
  

• Calculate Average Luminance in the Callback
  Implement the FrameObserver interface, then process each frame to compute average luminance from the Y channel:

  @Override
  public void onFrameChanged(Frame frame) {
      if (frame.getPixelFormat() == Frame.PIXEL_FORMAT.YUV) {
          byte[] yuvData = frame.getPixels();
          int width = frame.getWidth();
          int height = frame.getHeight();
          
          // Sum up all Y channel values (first width*height bytes)
          long totalLuminance = 0;
          for (int i = 0; i < width * height; i++) {
              totalLuminance += yuvData[i] & 0xFF; // Convert byte to unsigned integer
          }
          float avgLuminance = totalLuminance / (float)(width * height);
          // avgLuminance ranges from 0 (completely dark) to 255 (fully bright)
      }
  }
  

• Alternative: Get Lux Values with Android Camera2 API
  If you need precise light intensity in lux, you can use Android’s native Camera2 API alongside Vuforia. Once Vuforia initializes the camera, grab the camera ID, then use Camera2 to access metrics like STATISTICS_LENS_SHADING_MAP or calculate lux from exposure and ISO values. Just be careful not to interfere with Vuforia’s camera control.

iOS Implementation #

On iOS, Vuforia uses AVFoundation under the hood, so you can tap into AVFoundation’s tools to get luminance or light intensity:

• Hook into AVFoundation Capture Session
  Access Vuforia’s underlying AVCaptureSession and add a video data output to receive frame data:

  AVCaptureSession* captureSession = [VuforiaRenderer instance].videoCaptureDevice.captureSession;
  AVCaptureVideoDataOutput* videoOutput = [[AVCaptureVideoDataOutput alloc] init];
  
  // Configure output to deliver YUV frames
  [videoOutput setVideoSettings:@{(NSString*)kCVPixelBufferPixelFormatTypeKey: @(kCVPixelFormatType_420YpCbCr8BiPlanarFullRange)}];
  [videoOutput setSampleBufferDelegate:self queue:dispatch_get_main_queue()];
  
  [captureSession addOutput:videoOutput];
  

• Compute Luminance from Sample Buffers
  Implement the AVCaptureVideoDataOutputSampleBufferDelegate protocol to process each frame and calculate average luminance:

  - (void)captureOutput:(AVCaptureOutput *)output didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection {
      CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
      CVPixelBufferLockBaseAddress(imageBuffer, 0);
      
      // Access the Y plane (luminance data)
      void* yPlane = CVPixelBufferGetBaseAddressOfPlane(imageBuffer, 0);
      size_t yWidth = CVPixelBufferGetWidthOfPlane(imageBuffer, 0);
      size_t yHeight = CVPixelBufferGetHeightOfPlane(imageBuffer, 0);
      
      // Calculate average luminance
      uint8_t* yData = (uint8_t*)yPlane;
      long totalLuminance = 0;
      for (int i = 0; i < yWidth * yHeight; i++) {
          totalLuminance += yData[i];
      }
      float avgLuminance = totalLuminance / (float)(yWidth * yHeight);
      
      CVPixelBufferUnlockBaseAddress(imageBuffer, 0);
  }
  

• Alternative: Get Direct Lux Values
  For actual light intensity in lux (iOS 10+), you can directly access the lightLevel property of Vuforia’s underlying AVCaptureDevice:

  AVCaptureDevice* device = [VuforiaRenderer instance].videoCaptureDevice.captureDevice;
  float luxLevel = device.lightLevel;
  // luxLevel ranges from 0 (dark) to higher values for brighter environments
  

Key Tips to Keep in Mind #

• Performance: Calculating average luminance every frame can eat up CPU. Consider downsampling the frame data or computing it every 2-3 frames to optimize.
• Vuforia Version: Ensure you’re using a recent Vuforia version (9.x or later)—older versions might have different camera access APIs.
• Permissions: Don’t forget to request camera permissions in your app’s manifest (Android) or Info.plist (iOS)—Vuforia requires this anyway, but it’s good to double-check.

内容的提问来源于stack exchange，提问作者Nuibb