Derivative works — US Patent 11929073

Defensive Disclosure and Prior Art Generation for U.S. Patent 11,929,073

Publication Date: May 8, 2026
Subject: Derivative Embodiments and Obvious Variations of Hybrid Speech Arbitration Systems
Reference: U.S. Patent 11,929,073 B2 ("the '073 patent")

This document serves as a defensive publication of technical disclosures related to the art described in U.S. Patent 11,929,073. The following descriptions are intended to enter the public domain and be considered prior art for any future patent applications in this domain. The concepts disclosed herein are presented as logical and obvious extensions, substitutions, and combinations that a person of ordinary skill in the art of speech recognition, distributed computing, and machine learning would find apparent.

Derivative Variations Based on Core Claims

The following disclosures expand upon the core concepts of the '073 patent, particularly the "short-circuit" arbitration method where a local speech recognition result is evaluated for sufficiency before a cloud-based result is received.

I. Derivatives of Claim 1 (Method)

1. Material & Component Substitution: Neuromorphic & In-Memory Computing

Enabling Description: The method of claim 1 is implemented on a computing device where the "embedded ASR/NLU module" is not a traditional von Neumann processor but a specialized neuromorphic processing unit (NPU) or an in-memory computing (IMC) accelerator. The NPU, using spiking neural networks (SNNs), processes the incoming speech data with extremely low latency and power consumption. The "first plurality of features" includes bio-inspired signals such as spike-timing-dependent plasticity (STDP) metrics and neural firing rates, which serve as high-fidelity confidence indicators. The determination of whether to "short-circuit" is based on the stability and convergence of the SNN's output for the utterance. This substitutes conventional CPUs/DSPs with hardware that mimics biological neural processing to achieve the same functional goal of fast, local arbitration.

Mermaid Diagram:

graph TD
    A[Speech Signal Input] --> B{Neuromorphic Processor};
    B --> C[Generate Spiking Neural Representation];
    C --> D{Local SNN-based ASR/NLU};
    D --> E[First Recognition Result & STDP Metrics];
    E --> F{Arbitrator: Is Firing Pattern Stable?};
    F -- Yes --> G[Select Local Result & Act];
    F -- No --> H{Wait for Cloud Result};
    A --> I[Send to Cloud Service];
    I --> H;

2. Operational Parameter Expansion: Hypersonic Vehicle Cockpit Operation

Enabling Description: The arbitration method is adapted for extreme, high-stress, and high-vibration environments, specifically within the cockpit of a hypersonic vehicle (Mach 5+). The "first speech data" is acquired via a contact-based bone conduction microphone integrated into the pilot's helmet to mitigate extreme ambient noise and airframe vibration. The "computing device" is a radiation-hardened, passively cooled edge computer. The "short-circuit" classifier is trained specifically to handle clipped, stressed, or oxygen-mask-muffled speech patterns. The confidence threshold for selecting the local result is dynamically adjusted based on real-time biometric data from the pilot (e.g., heart rate, G-force load), biasing towards faster local execution for critical flight commands ("Pull up," "Eject") regardless of a slightly lower confidence score.

Mermaid Diagram:

sequenceDiagram
    participant Pilot
    participant BoneConductionMic
    participant RadHardEdgeCPU
    participant CloudService (Ground Control)

    Pilot->>BoneConductionMic: "Engage scramjet!"
    BoneConductionMic->>RadHardEdgeCPU: Digitized Speech Data
    RadHardEdgeCPU->>RadHardEdgeCPU: Process with Stress-Trained ASR
    RadHardEdgeCPU->>RadHardEdgeCPU: Calculate Confidence (Factoring G-Force)
    alt High Confidence OR Critical Command
        RadHardEdgeCPU->>Pilot: Execute Command (Audio/Haptic Feedback)
    else Low Confidence AND Non-Critical
        RadHardEdgeCPU->>CloudService: Send Data for Analysis
        CloudService-->>RadHardEdgeCPU: Return High-Fidelity Result
        RadHardEdgeCPU->>Pilot: Execute Refined Command
    end

3. Cross-Domain Application: Precision Agriculture (AgTech)

Enabling Description: The arbitration method is applied to an autonomous agricultural drone or "agribot" fleet. A farmer issues a voice command like "Spray sector gamma for blight, pattern delta." The command is captured by a ruggedized microphone on the farmer's handheld device. The "first speech recognition result" is generated on the agribot's local NVIDIA Jetson-class processor, which has access to local map data, installed pesticide types, and current GPS coordinates ("user data"). The "cloud computing service" is a central farm management server that holds historical yield data and satellite imagery. The short-circuit arbitrator decides if the command is simple and locally resolvable (e.g., "Stop spraying"). For complex commands involving chemical mixtures or historical data ("Analyze last season's protein levels and adjust spray"), the system waits for the cloud result to prevent costly errors.

Mermaid Diagram:

flowchart LR
    subgraph Agribot
        A[Voice Command] --> B[Local ASR/NLU];
        B -- Accesses --> C[Local Maps & Payload Data];
        B --> D{Short-Circuit Arbitrator};
    end
    subgraph Farm HQ
        E[Central Cloud Server]
        E -- Accesses --> F[Historical Yield & Satellite Data];
    end
    A --> E;
    D -- High Confidence --> G[Execute Command Immediately];
    D -- Low Confidence --> H{Wait for Cloud Confirmation};
    E --> H;
    H --> I[Execute Verified Command];

4. Integration with Emerging Tech: AI-Driven Federated Learning

Enabling Description: The arbitration method is integrated into a federated learning framework across a fleet of devices (e.g., vehicles). The local "short-circuit classifier" is itself a machine learning model. When the arbitrator opts to wait for the cloud result, the local result, the cloud result, and a ground-truth label (e.g., from user correction or implicit confirmation) are used as a training triplet. This triplet is not sent to the cloud. Instead, it is used locally to compute a gradient update for the short-circuit classifier model. These gradients, not the raw data, are securely aggregated in the cloud to train a global model, which is then pushed back to the devices. This continuously improves the local arbitrator's accuracy over time without compromising user privacy, using AI to optimize the core arbitration logic itself.

Mermaid Diagram:

stateDiagram-v2
    [*] --> Idle
    Idle --> LocalProcessing: Voice Input
    LocalProcessing: Generate Local Result R_E
    LocalProcessing --> ShortCircuit: Features F_E
    ShortCircuit: if Conf(R_E) > T_1 then select R_E
    ShortCircuit: else wait for R_C

    state fork_state <<fork>>
    ShortCircuit --> fork_state
    fork_state --> Selected_Local: High Confidence
    fork_state --> WaitingForCloud: Low Confidence

    WaitingForCloud --> ReceivedCloudResult: R_C arrives
    ReceivedCloudResult: Compare R_E and R_C
    ReceivedCloudResult --> UpdateModel: User confirms correct result
    UpdateModel: Compute model gradient
    UpdateModel --> Idle: Send gradient to cloud aggregator

    Selected_Local --> Idle

5. The "Inverse" or Failure Mode: Graceful Degradation Arbitration

Enabling Description: This version of the method is designed for network-unreliable or safety-critical environments (e.g., a subway system or industrial plant). The system operates in a "low-power" or "gracefully degraded" mode when network connectivity is lost or intermittent. In this mode, the "wait for the second speech recognition result" step is disabled entirely. The arbitrator's logic is inverted: it always selects the first (local) speech recognition result but attaches a "confidence level" metadata tag (e.g., "High," "Medium," "Low-Unverified"). Actions based on "Low-Unverified" results are restricted to non-critical functions (e.g., "What time is it?"). Critical functions ("Stop the assembly line") require a "High" confidence score or a secondary, non-verbal confirmation. This ensures the system remains functional but safe when the cloud is unreachable.

Mermaid Diagram:

graph TD
    A[Speech Input] --> B[Local ASR/NLU];
    B --> C{Network Connectivity Check};
    C -- Connected --> D{Standard Short-Circuit Arbitration};
    C -- Disconnected --> E{Graceful Degradation Mode};
    D -- High Confidence --> F[Select Local & Act];
    D -- Low Confidence --> G[Wait for Cloud];
    E --> H[Select Local Result];
    H --> I{Tag with Confidence Level};
    I --> J{Is Action Critical?};
    J -- Yes --> K[Require Secondary Confirmation];
    J -- No --> L[Execute Action];

II. Combination Prior Art Scenarios

These scenarios combine the teachings of the '073 patent with existing, open-source standards to create novel, non-obvious, and publicly disclosed systems.

1. Combination with Matter IoT Standard

Disclosure: The hybrid arbitration system of the '073 patent is integrated into a smart home hub that is compliant with the Matter open-source IoT standard. A user's voice command is captured by a Matter-enabled device (e.g., a smart speaker). The device's local processor, running a lightweight ASR engine, generates the first result. The "first plurality of features" is augmented with contextual data from the Matter fabric, such as the state of nearby lights, locks, and thermostats. The short-circuit decision uses this rich local context. For example, the command "Lock the door" when the Matter fabric reports the door sensor is already closed, will yield a very high confidence score for the local result. If the command is ambiguous ("Set the mood"), the system waits for the cloud result, which can process more complex natural language. The final selected command is then executed as a standard Matter command broadcast over the local Thread or Wi-Fi network.
Enabling Description: A developer would use the Matter SDK to build a device firmware. The firmware would include a local ASR engine (e.g., Picovoice, TensorFlow Lite for Microcontrollers) and the short-circuit arbitration logic. The arbitrator module would have an API to query the state of other Matter devices (clusters and attributes) on the local network. A high-confidence local result triggers an immediate call to the appropriate Matter command function within the SDK (e.g., chip::Controller::DoorLockCluster::LockDoor). A low-confidence result initiates a request to a cloud NLU service (like Rasa or a proprietary one), and the response from the cloud is then translated into the corresponding Matter command.

2. Combination with Android Open Source Project (AOSP)

Disclosure: The arbitration method is implemented as a core service within the Android Open Source Project (AOSP) framework, providing a standardized API for all apps. This service, tentatively named HybridRecognitionManager, performs the short-circuit arbitration. The "on-device" result is generated by Android's built-in SpeechRecognizer service, which can leverage personal data like on-device contacts and app names. The "cloud" result is solicited from a pluggable, user-selected backend (e.g., Google Assistant, Alexa). The HybridRecognitionManager exposes a callback, onImmediateResult(Result result, boolean isFinal), which allows an application to get the low-latency local result first. The isFinal flag is false. If the arbitrator decides to wait, it later invokes a second callback, onFinalResult(Result result), with the arbitrated best result. This allows app developers to, for example, tentatively populate a UI field with the immediate result and then confirm or correct it with the final result, improving perceived performance system-wide.
Enabling Description: This would involve adding a new system service to AOSP. The implementation would reside in /frameworks/base/services/core/java/com/android/server/hybridrecognition/. It would manage the lifecycle of both the local SpeechRecognizer and connections to external cloud recognizers. The service would use a DNN classifier (like the one described in the '073 patent) for the short-circuit decision. The API would be exposed through /frameworks/base/core/java/android/speech/HybridRecognitionManager.java, providing methods like startListening(Intent intent, RecognitionListener listener) and defining the new listener interface.

3. Combination with the Robot Operating System (ROS)

Disclosure: The system of claim 14 is embodied as a standardized ROS 2 node package called hybrid_asr_arbitrator. This package provides a service for arbitrating between two speech recognition topics. The "first speech recognition result" is subscribed from a topic published by a local, on-robot ASR node (e.g., kaldi_ros). The "cloud computing service" is another ROS node that acts as a bridge, forwarding the audio to a cloud service and publishing the result on a separate topic. The hybrid_asr_arbitrator node listens to the local result topic. Upon receiving a message, it performs the short-circuit decision based on the result and its associated features (confidence scores, etc.). If the local result is selected, it is immediately re-published on a final /final_transcript topic. If not, it waits for a message on the cloud result topic, performs the final comparison, and then publishes the winner to /final_transcript. This allows any roboticist to easily integrate a robust, low-latency hybrid speech system into any ROS-based robot.
Enabling Description: The package would be a C++ or Python ROS 2 package. The main node would subscribe to two topics of type speech_recognition_msgs/SpeechRecognitionCandidates. It would implement the short-circuit classifier logic. The classifier model (e.g., a trained ONNX model) would be included in the package. The node would have configurable parameters for the confidence threshold (T1) and the timeout for waiting for the cloud result. The output would be a std_msgs/String message on the /final_transcript topic. The package would be released on GitHub and could be installed via apt like any other ROS package.