如何编程捕获Apple VoiceOver朗读网页（含交互响应）的音频输出？

Great question! Capturing VoiceOver's spoken output—including the speech triggered by user interactions like clicking your Submit button—is absolutely possible, but it combines system-level audio capture with accessibility automation. Let’s walk through how to pull this off for your HTML example.

Core Approach #

VoiceOver’s audio is part of your Mac’s system output, so first we need to capture that system audio to a WAV stream. Then, we’ll use accessibility APIs to automate VoiceOver: load your web page, trigger interactions (like focusing the name field or clicking Submit), and make sure VoiceOver reads the relevant content during capture.

Step 1: Capture System Audio to WAV #

On macOS, you can use AVFoundation to capture system audio. First, you’ll need to set up an audio capture session targeting the system’s "Loopback" audio device (this lets you record what’s being played out, not just microphone input).

Here’s a simplified Swift snippet to get you started:

import AVFoundation

func startAudioCapture(to outputPath: String) throws -> AVCaptureSession {
    let session = AVCaptureSession()
    session.sessionPreset = .high

    // Get the system loopback audio device
    guard let audioDevice = AVCaptureDevice.default(.audioDataRecorder, for: .audio, position: .unspecified) else {
        throw NSError(domain: "AudioCapture", code: -1, userInfo: [NSLocalizedDescriptionKey: "Could not find loopback device"])
    }

    // Create audio input
    let audioInput = try AVCaptureDeviceInput(device: audioDevice)
    if session.canAddInput(audioInput) {
        session.addInput(audioInput)
    }

    // Create WAV output file
    let outputURL = URL(fileURLWithPath: outputPath)
    let audioOutput = AVCaptureAudioFileOutput()
    if session.canAddOutput(audioOutput) {
        session.addOutput(audioOutput)
    }

    // Start recording
    audioOutput.startRecording(to: outputURL, recordingDelegate: nil)
    session.startRunning()

    return session
}

// Usage:
// let captureSession = try startAudioCapture(to: "/path/to/output.wav")
// ... do VoiceOver actions ...
// captureSession.stopRunning()

Step 2: Automate VoiceOver & Web Interactions #

Next, we need to load your HTML page, trigger interactions, and make VoiceOver read the content. We’ll use WebKit to load the page and Accessibility APIs (via AXUIElement) to control VoiceOver.

First, optimize your HTML for better VoiceOver support (optional but recommended):

<!doctype html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <title>VoiceOver Test</title>
</head>
<body>
    <label for="name">Name</label>
    <input id="name" type="text" aria-label="Enter your full name" />
    <button type="submit" aria-label="Submit your name">Submit</button>
</body>
</html>

Then, here’s how to automate VoiceOver with the web page:

import WebKit
import ApplicationServices

func automateVoiceOverWithWebPage() {
    // Load the HTML page in WKWebView
    let webView = WKWebView(frame: CGRect(x: 0, y: 0, width: 800, height: 600))
    let htmlPath = Bundle.main.path(forResource: "test", ofType: "html")!
    let htmlContent = try! String(contentsOfFile: htmlPath)
    webView.loadHTMLString(htmlContent, baseURL: nil)

    // Wait for page to load (add proper async handling in production)
    Thread.sleep(forTimeInterval: 2)

    // Get VoiceOver's UI element
    guard let voiceOverPID = getVoiceOverPID(),
          let voiceOverElement = AXUIElementCreateApplication(voiceOverPID) else {
        print("Could not access VoiceOver")
        return
    }

    // Enable VoiceOver if it's off
    var isEnabled: CFBoolean = false
    AXUIElementCopyAttributeValue(voiceOverElement, kAXEnabledAttribute as CFString, &isEnabled)
    if !isEnabled {
        AXUIElementSetAttributeValue(voiceOverElement, kAXEnabledAttribute as CFString, kCFBooleanTrue)
        Thread.sleep(forTimeInterval: 1)
    }

    // Focus the name input and trigger VoiceOver to read it
    let nameInput = webView.value(forKey: "accessibilityElement") as! AXUIElement
    AXUIElementSetAttributeValue(voiceOverElement, kAXFocusedUIElementAttribute as CFString, nameInput)
    Thread.sleep(forTimeInterval: 2) // Wait for VoiceOver to finish reading

    // Simulate clicking the Submit button and read its response
    let submitButton = webView.accessibilityElement(at: 2) // Adjust index based on your page
    AXUIElementPerformAction(submitButton, kAXPressAction as CFString)
    Thread.sleep(forTimeInterval: 2)
}

func getVoiceOverPID() -> pid_t? {
    let workspace = NSWorkspace.shared
    guard let voiceOverApp = workspace.runningApplications.first(where: { $0.bundleIdentifier == "com.apple.VoiceOver" }) else {
        return nil
    }
    return voiceOverApp.processIdentifier
}

Step 3: Combine Audio Capture & Automation #

Put it all together: start audio capture, run the VoiceOver automation, then stop capture to save the WAV file.

func runFullCapture() {
    do {
        let captureSession = try startAudioCapture(to: "/Users/yourname/Documents/voiceover_output.wav")
        automateVoiceOverWithWebPage()
        captureSession.stopRunning()
        print("Audio capture saved successfully!")
    } catch {
        print("Error: \(error.localizedDescription)")
    }
}

// Run the capture
runFullCapture()

Key Notes #

• Permissions: Your app needs Accessibility permissions (enable in System Settings > Privacy & Security > Accessibility) to control VoiceOver, and Microphone permissions (even though we’re capturing system audio, some APIs require this).
• Async Handling: The example uses Thread.sleep for simplicity—replace this with proper completion handlers or dispatch queues in production code.
• VoiceOver Behavior: VoiceOver’s reading depends on your page’s accessibility markup. Make sure elements have clear aria-labels or proper semantic HTML (like <label> elements linked to inputs) for accurate speech.

内容的提问来源于stack exchange，提问作者Jonathan