Starred Gists
Starred Gists订阅

subvox prompt

we want to build a prototype mac app with my airpods pros

For AirPods Pro 3, Apple exposes motion, audio, and heart-rate data to third-party developers, but not every sensor’s raw output.

1. Developer-accessible sensors

Sensor Developer access API Raw data?
Accelerometer (motion) ✅ Yes Core Motion Processed
Gyroscope / rotation ✅ Yes Core Motion Processed
Head orientation ✅ Yes Core Motion Processed
Gravity vector ✅ Yes Core Motion Processed
External microphones ✅ Yes AVAudioEngine Processed audio
Heart-rate sensor ✅ Yes HealthKit Heart-rate measurements
Speech-detecting accelerometer ❌ No direct access Private No
Inward-facing microphone ❌ No separate access Private No
Skin-detection sensor ⚠️ Indirect System-managed No
Touch/force sensor ⚠️ Limited Media controls No
UWB chip (charging case) ❌ No direct access System-managed No

Apple documents the hardware and supported interfaces.

2. What Core Motion actually exposes

The most interesting API is CMHeadphoneMotionManager.

You can stream:

import CoreMotion
let manager = CMHeadphoneMotionManager()
manager.startDeviceMotionUpdates(to: .main) { motion, error in
    guard let motion else { return }
    // Acceleration
    let acceleration = motion.userAcceleration
    // Rotation velocity
    let rotation = motion.rotationRate
    // Orientation
    let attitude = motion.attitude
    // Gravity
    let gravity = motion.gravity
    print(acceleration, rotation, attitude, gravity)
}

This works on supported AirPods with iOS and macOS.

Important: These are processed motion measurements, not the raw high-frequency signals from the speech-detecting accelerometer.

Documentation: CMHeadphoneMotionManager

3. Can we detect subvocal speech?

Here’s how I’d rank the available signals:

Signal Potential for silent speech
Speech accelerometer (private) ⭐⭐⭐⭐⭐
Inward microphone (private) ⭐⭐⭐⭐
Head acceleration (public) ⭐⭐
Head rotation (public) ⭐
External microphone (public) ⭐⭐⭐ for whispering

The problem is sampling frequency and signal processing.

Head-tracking motion data is designed to measure relatively slow head movements, not the high-frequency vibrations associated with speech.

Even if jaw movement produces measurable vibrations, Apple’s processing may remove them.

4. One additional opportunity

Apple supports high-quality Bluetooth recording from compatible AirPods.

That makes whisper recognition worth investigating before attempting silent articulation.

Apple's high-quality Bluetooth recording API

5. What I’d build first

A native Swift app that simultaneously captures:

  1. AirPods microphone audio.
  2. Three-axis acceleration.
  3. Three-axis rotation.
  4. Head orientation.

Record synchronized streams while performing three experiments: normal speech, whispering, and completely silent mouthing.

Then measure whether the motion streams contain statistically distinguishable information about articulation.

My assessment: AirPods Pro 3 are sufficient to prototype whisper-based input. Whether they can support genuine subvocal input through public APIs remains unproven.

The fastest way to find out is a sensor-recording prototype, not an LLM integration.

... mac app that has a screen that can process accelerometer data to text

参与讨论