US 12406663

Analysis of U.S. Patent 12,406,663

Washington, D.C. - An analysis of United States Patent 12,406,663, titled "Routing of user commands across disparate ecosystems," reveals a system for integrating voice commands from a vehicle with various smart home or Internet of Things (IoT) ecosystems. The patent, issued to Cerence Operating Co., outlines a method for a vehicle's assistant to intelligently route user commands to the correct external system, such as a home automation platform.

Patent Number	12,406,663
Title	Routing of user commands across disparate ecosystems
Assignee	Cerence Operating Co.
Inventors	Prateek Kathpal, Brian Arthur Rubin
Filing Date	December 17, 2021
Issue Date	September 2, 2025
Abstract	A system for routing commands issued by a passenger of a vehicle to a Smart Home and/or an Internet of Things (IoT) ecosystem via a connection manager. Issued commands are obtained from utterances using speech recognition and analyzed using natural language understanding and natural language processing. Using the output of the natural understanding analysis, the connection manager determines where to send the command by identifying a target Smart Home and/or IoT ecosystem.

As of April 26, 2026, a search of the dockets for the Court of Appeals for the Federal Circuit (CAFC) for the year 2026 did not reveal any cases involving US Patent 12,406,663.

Overview of Independent Claims:

This patent contains three independent claims which form the core of the invention.

Independent Claim 1 describes a system within a vehicle that can process a user's spoken commands. This system includes a "recognition module" with hardware processors that receives utterances from the vehicle's speakers. This module is designed to identify a specific command and determine which of the user's existing smart home systems (the "target ecosystem") the command is intended for. A key component is a natural language understanding (NLU) module that interprets the meaning of the spoken words to identify the correct ecosystem. The claim specifies that a "connection manager" then sends the command to that target ecosystem. A crucial feature of this claim is the system's ability to learn and improve; it receives feedback from the smart home system about the command and uses this feedback to update its NLU models.

Independent Claim 3 outlines a method, or a series of steps, for routing these commands. The process begins with receiving a spoken command in the vehicle, along with "contextual data" such as the vehicle's location or the time of day. The spoken words are converted to text. The system then determines the correct smart home ecosystem for the command by analyzing both the text and the contextual information. For example, a command like "turn on the lights" might be routed to the user's home lighting system if the vehicle is nearing the user's house in the evening. The command is then sent to the target system, and the method includes receiving a confirmation that the command was successfully carried out.

Independent Claim 12 describes a broader system that accomplishes a similar goal. It details a recognition module within a vehicle that processes voice commands from an occupant. This system uses contextual data associated with the command to identify the correct "target ecosystem," which is defined as a non-vehicle system in the user's home that can execute commands remotely. A "connection manager" is then responsible for routing the command to that identified ecosystem. This claim emphasizes the integration of the in-vehicle system with external smart home environments.

Litigation Search for U.S. Patent 12,406,663

Washington, D.C. - As of May 8, 2026, a search of federal court dockets has revealed that U.S. Patent No. 12,406,663 is asserted in a patent infringement lawsuit. The patent's assignee, Cerence Operating Co., has filed a complaint against Amazon.com, Inc. and its subsidiaries.

Details of the litigation are as follows:

• Case Title: Cerence Operating Company v. Amazon.com, Inc. et al.
• Plaintiff: Cerence Operating Company
• Defendants: Amazon.com, Inc., Amazon.com Services LLC, and Amazon Web Services, Inc.
• Jurisdiction: U.S. District Court for the Eastern District of Texas
• Case Number: 2:26-cv-00373
• Filing Date: May 4, 2026
• Status: The case is currently in its initial stages, with the complaint having been recently filed.

The complaint alleges that Amazon's products and services, including its Echo devices and Alexa services, infringe upon multiple Cerence patents related to voice recognition and natural language processing technologies. U.S. Patent 12,406,663 is specifically listed as one of the asserted patents in the complaint. This lawsuit is one of two filed by Cerence against Amazon on the same day, targeting a range of technologies including speech recognition, text-to-speech, and acoustic modeling.

Proceedings overview

The USPTO ODP API returns no AIA trial proceedings for this patent as of the most recent ingest. Web search did not surface any additional PTAB activity. Therefore, there are no PTAB proceedings on file for U.S. Patent 12,406,663, and all claims are currently untested in an AIA trial. This means a defendant would need to pursue an initial PTAB challenge if they wished to invalidate the claims through this venue.

Strategic summary

All claims of U.S. Patent 12,406,663 are currently untested in PTAB proceedings. No claims have been canceled or sustained through an AIA trial. Consequently, there is no estoppel landscape to consider under § 315(e)(2), and all prior-art grounds are still available for potential challenges. The absence of PTAB activity could indicate that the patent has not yet been aggressively asserted in a manner that would typically trigger such challenges, or it is too new to have garnered such attention.

Recommended next steps

As there is no PTAB activity on file, a defendant facing assertion of U.S. Patent 12,406,663 would need to initiate a new AIA trial (e.g., an Inter Partes Review or Post-Grant Review) if they wish to challenge the patentability of its claims before the PTAB.

Inventors

• Prateek Kathpal (Cerence Operating Co.)
• Brian Arthur Rubin (Cerence Operating Co.)

Original assignee

Cerence Operating Co. is the original assignee named on the issued patent. Cerence's primary line of business is providing AI-powered solutions for connected vehicles, including speech recognition, natural language understanding, and voice biometrics. They ship products embodying the claims, such as the CERENCE Drive 2.0 framework, which provides ASR and NLU services to vehicles. Cerence Operating Co. is an active, operating company.

Assignment timeline

• 2022-01-04 (executed) / recorded 2022-01-04 — Reel 058541/0717

  • Conveyance: Assignment
  • Assignor: Kathpal, Prateek; Rubin, Brian Arthur
  • Assignee: Cerence Operating Company, Massachusetts
  • Correspondent: Matthew J. Van Eman, Cerence Inc., 15 Wayside Road, Burlington, MA 01803.
  • Context: Internal transfer from inventors to operating company.

• 2024-04-12 (executed) / recorded 2024-04-15 — Reel 067417/0303

  • Conveyance: Security Agreement
  • Assignor: Cerence Operating Company
  • Assignee: Wells Fargo Bank, N.A., as Collateral Agent, North Carolina
  • Correspondent: Matthew J. Van Eman, Cerence Inc., 15 Wayside Road, Burlington, MA 01803. This correspondent also appears on the 2022-01-04 assignment.
  • Context: Collateral for a financing agreement.

• 2024-12-31 (executed) / recorded 2025-01-02 — Reel 069797/0422

  • Conveyance: Release
  • Assignor: Wells Fargo Bank, National Association
  • Assignee: Cerence Operating Company, Massachusetts
  • Correspondent: Matthew J. Van Eman, Cerence Inc., 15 Wayside Road, Burlington, MA 01803. This correspondent also appears on the 2022-01-04 and 2024-04-15 assignments.
  • Context: Release of security interest.

Timeline diagram

timeline
    title Ownership of US 12406663
    2021 : Filed by Cerence Operating Co.
    2022 : Assigned from inventors to Cerence
    2024 : Security agreement to Wells Fargo Bank
         : Security agreement released
    2025 : Patent Issued
    2026 : Litigation against Amazon filed

NPE / troll-pattern signals

1. Shell-entity transfer — not present. The patent has consistently been held by Cerence Operating Company, an active operating company, or temporarily by Wells Fargo as a collateral agent.
2. Known asserter in the chain — not present. Cerence Operating Company is not identified on public NPE lists as of the current date.
3. Repeat correspondent across the chain — present. Matthew J. Van Eman of Cerence Inc. is listed as the correspondent on all three recorded assignments (Reel 058541/0717, Reel 067417/0303, and Reel 069797/0422).
4. Cascading transfers — not present. The recorded transfers are an initial assignment from inventors, followed by a security interest and its subsequent release, all involving the same operating company.
5. Pre-litigation transfer — not present. The last recorded transfer (release of security interest) was on 2025-01-02, while the litigation against Amazon was filed on 2026-05-04. This is outside the 6-month window for a "pre-litigation transfer" signal.
6. Bankruptcy fire-sale — not present. Cerence Operating Company is an active operating company, not in bankruptcy.
7. Privateering — unclear. While Cerence is an operating company, the nature of its assertion against Amazon, a competitor in voice assistant technology, could be perceived as defensive or competitive. However, without further information (e.g., SEC filings detailing licensing agreements or public statements about assertion strategy), it's not definitively a privateering arrangement.
8. Defensive aggregator (anti-NPE) — not present. The patent is currently held by Cerence Operating Company, not a defensive aggregator.

Verdict

Operating-company assertion
Cerence Operating Company, the original and current assignee of US Patent 12,406,663, is an active company that develops and markets products directly embodying the claims, such as vehicle voice assistants. The ongoing litigation against Amazon.com, Inc. et al. (Case Number: 2:26-cv-00373) indicates assertion by an operating company against a competitor. The recorded assignments show internal transfers and a security interest, all consistent with a standard operating company's patent management practices, with the same correspondent consistently handling the recordings.

Analysis of Prior Art for U.S. Patent 12,406,663

Under 35 U.S.C. § 102, an invention cannot be patented if it was already patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. For U.S. Patent 12,406,663, with a priority date of December 21, 2020, any relevant prior art must predate this. The following patents, cited as prior art during the examination of the '663 patent, are considered most relevant.

---

1. U.S. Patent 9,734,839 B1: "Routing natural language commands to the appropriate applications"

• Full Citation: US Patent 9,734,839 B1
• Assignee: Amazon Technologies, Inc.
• Publication Date: August 15, 2017
• Filing Date: June 20, 2012
• Brief Description: This patent discloses a system that receives a natural language command and routes it to an appropriate application for execution. It describes a "routing manager" that analyzes the user's utterance to determine the user's intent and selects a third-party application to handle the command. The system can use contextual information, such as user location or time of day, to assist in the routing decision.
• Potential Anticipation of Claims:
  • Claim 1 & 12: This patent appears to describe the core elements of claims 1 and 12, namely a system with a module for receiving an utterance (command), interpreting its meaning (natural language understanding), and a manager for transmitting the command to a target system (application). The '839 patent's "routing manager" is analogous to the "connection manager" in the '663 patent. The use of contextual data for interpretation is also taught.
  • Claim 3: The method described in claim 3, including receiving an utterance and contextual data, translating to text, determining a target, and transmitting the command, is substantially described in the '839 patent. While the '839 patent is not limited to a vehicle environment, its general teachings of routing commands based on interpretation and context are highly relevant.

---

2. U.S. Patent 11,289,075 B1: "Routing of natural language inputs to speech processing applications"

• Full Citation: US Patent 11,289,075 B1
• Assignee: Amazon Technologies, Inc.
• Publication Date: March 29, 2022
• Filing Date: December 13, 2019
• Brief Description: This patent describes a system for routing a natural language input to one of a plurality of speech-processing applications. A router component determines which application is best suited to process the input based on the content of the input itself and potentially other factors. The system is designed to handle commands for various domains, such as music, smart home devices, or information queries.
• Potential Anticipation of Claims:
  • Claim 1 & 12: The '075 patent discloses a system that receives a natural language input (utterance) and routes it to a specific application (ecosystem). The "router" in this patent performs a similar function to the "recognition module" and "connection manager" of the '663 patent by identifying the user's intent and selecting the appropriate destination for the command. Its filing date precedes the '663 patent's priority date.
  • Claim 3: The method of receiving an input, determining a target application from a plurality of options, and transmitting the input for processing is a central teaching of this patent. This aligns closely with the steps laid out in claim 3 of the '663 patent, even though the context is not explicitly limited to an in-vehicle system interacting with smart homes.

---

3. U.S. Patent Application Publication 2016/0179462 A1: "Connected device voice command support"

• Full Citation: US Patent Application Publication 2016/0179462 A1
• Assignee: Intel Corporation
• Publication Date: June 23, 2016
• Filing Date: December 22, 2014
• Brief Description: This application describes a system where a primary device (like a smartphone or in-vehicle infotainment system) can receive a voice command intended for a secondary, connected device (such as a smart home appliance). The primary device processes the voice command to identify the target device and the intended action, and then transmits the command to the target device for execution.
• Potential Anticipation of Claims:
  • Claim 1 & 12: This publication teaches a system that receives an utterance on one device (analogous to the vehicle head unit), identifies a target ecosystem (the secondary device), and forwards a command. The concept of a primary device acting as an intermediary to control various other connected devices is a key overlap with the invention claimed in the '663 patent.
  • Claim 3: The method of receiving a voice command, processing it to determine a target device and action, and then transmitting it is clearly outlined. This process mirrors the steps in claim 3. The application specifically mentions an in-vehicle system as a potential primary device, strengthening its relevance as prior art. While it may not explicitly detail updating NLU models based on feedback, the foundational routing process is present.

Obviousness Analysis of U.S. Patent 12,406,663 under 35 U.S.C. § 103

Washington, D.C. - An analysis of the claims of U.S. Patent 12,406,663 ("the '663 patent") in light of prior art available before its priority date of December 21, 2020, indicates that the claims may be rendered obvious under 35 U.S.C. § 103. The analysis centers on the combination of known elements from separate prior art references which, when combined, appear to teach all elements of the independent claims of the '663 patent. A person having ordinary skill in the art (PHOSITA) in the fields of in-vehicle infotainment, voice recognition, and IoT device control would have been motivated to combine these teachings to achieve a predictable result.

The core inventive concept of the '663 patent is a system within a vehicle that uses natural language understanding (NLU) and contextual data (e.g., vehicle location) to interpret a user's voice command and route it to the correct external smart home or IoT ecosystem. The system then uses feedback from that ecosystem to improve its NLU models.

Several combinations of prior art references could render the claims obvious. One strong combination is U.S. Patent 9,734,839 to Amazon Technologies, Inc. ("'839 patent") in view of U.S. Patent Application Publication 2014/0365228 to Honda Motor Co., Ltd. ("'228 application") and U.S. Patent Application Publication 2014/0136202 to GM Global Technology Operations LLC ("'202 application").

---

Analysis of Independent Claim 1

Claim 1 recites a system with a recognition module and a connection manager in a vehicle. The recognition module receives an utterance, uses an NLU module to interpret its meaning and identify a target smart home ecosystem, and the connection manager transmits the command. Crucially, the system receives feedback from the ecosystem and updates its NLU models.

1. Primary Reference: US 9,734,839 ('839 patent)
   The '839 patent, titled "Routing natural language commands to the appropriate applications," teaches the fundamental concept of the '663 patent. The '839 patent discloses a system that receives a natural language command, uses a routing component to determine which of several available applications (analogous to the '663 patent's "ecosystems") is the intended target, and forwards the command to that application. This directly teaches the core elements of receiving an utterance, interpreting it to identify a target, and transmitting a command to that target.

2. Secondary Reference: US 2014/0365228 ('228 application)
   The '839 patent does not explicitly place its system within a vehicle or use vehicle-specific context. The '228 application, titled "Interpretation of ambiguous vehicle instructions," remedies this. It explicitly teaches a system for interpreting voice commands within a vehicle and using contextual information, such as vehicle location ("the present location of the host vehicle"), to resolve ambiguity.

   • Motivation to Combine '839 and '228: A PHOSITA would have been motivated to implement the command-routing system of the '839 patent within the vehicle environment described in the '228 application. By 2020, integrating external applications and services into vehicle head units was a well-established trend. A PHOSITA would have seen a clear and predictable benefit in extending the in-vehicle voice assistant of the '228 application to control not just vehicle functions but also the wide array of external applications described in the '839 patent (e.g., smart home devices). This combination would allow a user to seamlessly interact with their digital life from their car, addressing a known market demand.

3. Tertiary Reference: US 2014/0136202 ('202 application)
   The combination of the '839 and '228 references teaches most of Claim 1, but lacks the explicit teaching of updating NLU models based on feedback from the target ecosystem. The '202 application, titled "Adaptation methods and systems for speech systems," supplies this missing element. It describes methods for adapting a speech processing system based on user interactions and confirmations, which serves as a form of feedback. This adaptation improves the accuracy of the system over time.

   • Motivation to Combine with '202: A PHOSITA, having combined the '839 and '228 references to create an in-vehicle command router, would naturally seek to improve its performance. The use of feedback loops and model adaptation, as taught by the '202 application, was a standard and well-known technique in machine learning and speech recognition to enhance accuracy. Applying this adaptation method to the combined system would be a predictable step to make the command routing more reliable, thus rendering the final element of Claim 1 obvious.

---

Analysis of Independent Claim 3

Claim 3 outlines a method that mirrors the system of Claim 1 but explicitly includes the step of "receiving contextual data relating to the one or more utterances."

• The combination of the '839 patent and the '228 application renders this claim obvious. The '839 patent teaches the overall method of receiving a command, determining a target ecosystem, transmitting the command, and receiving confirmation. The '228 application explicitly teaches the missing step of using contextual data, specifically vehicle location and time of day, to aid in this determination. A PHOSITA would combine these methods for the same reasons articulated for Claim 1: to create a more powerful and versatile in-vehicle assistant capable of controlling external devices, which was a clear direction of technology development.

---

Analysis of Independent Claim 12

Claim 12 describes a system with a recognition module in a vehicle that uses contextual data to identify a target non-vehicle ecosystem and a connection manager to route the command. This claim is broader than Claim 1 as it does not require the feedback-based NLU model update.

• This claim is rendered obvious by the combination of the '839 patent and the '228 application alone. The '839 patent teaches the recognition module and connection manager for routing commands to target ecosystems (applications). The '228 application teaches placing this capability within a vehicle and using contextual data to do so. The motivation to combine these references is to apply a known command-routing technique to the automotive domain to enhance functionality, which would have been a straightforward and predictable design choice for a PHOSITA at the time.

Analysis of Patent Term and Related Applications for U.S. Patent 12,406,663

Washington, D.C. - A detailed review of the United States Patent and Trademark Office (USPTO) records for U.S. Patent No. 12,406,663, assigned to Cerence Operating Co., provides information regarding its term, related applications, and international family members.

Patent Term and Expiration

The standard term for a U.S. patent is 20 years from the earliest effective filing date. For U.S. Patent 12,406,663, the filing date is December 17, 2021.

• Patent Term Adjustment (PTA): There is no information available from the provided source to indicate that a Patent Term Adjustment was granted or denied. PTA is typically granted to compensate for delays caused by the USPTO during the patent prosecution process.
• Patent Term Extension (PTE): There is no indication of any Patent Term Extension for this patent. PTE is typically granted to compensate for regulatory delays in bringing a product to market, which is not applicable here.
• Projected Expiration Date: Based on the provided data, the projected expiration date is April 14, 2044. This adjusted expiration date suggests a significant patent term adjustment may have been applied, though the specific number of PTA days is not detailed in the source text. The standard 20-year term from the December 17, 2021, filing date would normally result in a 2041 expiration.

Related Applications and Patent Family

The prosecution history of the '663 patent shows connections to other applications, both domestically and internationally.

• Priority Application: The patent claims priority to U.S. Provisional Application No. 63/128,293, filed on December 21, 2020. This earlier filing date is critical for establishing the invention's novelty and non-obviousness against prior art.
• Continuation or Divisional Applications: The provided information does not list any continuation or divisional applications for U.S. Patent 12,406,663.
• Patent Family Members: This patent is part of a larger international patent family, indicating that Cerence Operating Co. has sought protection for this invention in multiple jurisdictions. The known family members include:
  • China: CN116830190A
  • Europe: EP4264599A1
  • WIPO (PCT): WO2022140178A1

These international filings underscore the perceived commercial importance of the technology for routing user commands from vehicles to smart home ecosystems across different global markets.

Defensive Disclosure and Prior Art Generation for U.S. Patent 12,406,663

Publication Date: May 8, 2026
Reference ID: DDPUB-2026-0508-A
Subject Matter: Systems and methods for contextual routing of commands between a primary control interface (e.g., a vehicle) and a plurality of disparate, remote-controlled ecosystems (e.g., Smart Home, Industrial IoT). This document describes derivative works, alternative embodiments, and expansions of the core concepts disclosed in U.S. Patent 12,406,663 to establish prior art against future incremental patent claims in this domain.

---

Derivatives of Independent Claim 1 (System Claim)

The core claim describes a system with a recognition module (NLU), a connection manager, and a feedback loop for updating NLU models, all operating from a vehicle head unit to control a user's smart home. The following disclosures expand upon this foundation.

1. Component Substitution: Federated Edge-Based NLU Processing

• Enabling Description: This embodiment replaces the centralized, cloud-based NLU module with a federated learning architecture executing on edge devices. The in-vehicle head unit, the user's mobile device, and a home hub each contain a local instance of the NLU model. A command issued in the vehicle is processed by the local NLU model. Instead of sending raw utterance data to the cloud, only the resulting model update gradients (e.g., deltas representing learned associations) are encrypted and shared with a cloud-based aggregator. The aggregator compiles updates from all user devices to create an improved global model, which is then pushed back to the edge devices. This approach enhances privacy by keeping raw user data on-device and reduces latency for command interpretation. The connection manager function remains to route the post-interpretation command to the target ecosystem API.
• Mermaid Diagram:

  sequenceDiagram
      participant V as Vehicle HU
      participant P as User's Phone
      participant H as Home Hub
      participant C as Cloud Aggregator
      participant E as Target Ecosystem API
  
      V->>V: User speaks command; Local NLU processes it
      V->>C: Sends encrypted model update gradients
      P->>C: Periodically sends local learning gradients
      H->>C: Periodically sends local learning gradients
      C->>C: Aggregates gradients to create new global model
      C-->>V: Pushes updated global model
      C-->>P: Pushes updated global model
      C-->>H: Pushes updated global model
      V->>E: Connection Manager routes command to API
      E-->>V: Returns command feedback
      V->>V: Local NLU model is updated with feedback
  

2. Cross-Domain Application: Surgical Operating Room (OR) Command Arbitration

• Enabling Description: The system is adapted for a surgical OR environment. The "head unit" is a sterile, microphone-equipped console. The "ecosystems" are disparate pieces of surgical equipment from different manufacturers (e.g., a da Vinci surgical robot, a Stryker endoscopy tower, a Philips patient monitoring system). A surgeon's spoken command, such as "increase insufflation pressure," is processed by an NLU module. The contextual data includes the current phase of the operation (e.g., "laparoscopic cholecystectomy - dissection phase") retrieved from the hospital's electronic health record (EHR) system. The NLU determines that during this phase, "insufflation pressure" refers to the CO2 insufflator managed by the endoscopy tower. The connection manager then formats the command into the proprietary protocol for the Stryker tower and transmits it. Feedback (e.g., confirmation of pressure change) is used to update the NLU models for surgical-phase-specific command routing.
• Mermaid Diagram:

  flowchart TD
      A[Surgeon's Utterance: "increase pressure"] --> B{NLU Module};
      C[EHR Context: "Cholecystectomy - Dissection Phase"] --> B;
      B --> D{Arbitration Engine};
      D -- "Pressure" in this context refers to insufflator --> E[Select Target: Stryker Endoscopy Tower];
      D -- Not surgical robot --> F((da Vinci Robot));
      D -- Not patient monitor --> G((Philips Monitor));
      E --> H[Connection Manager];
      H --> I[Format command for Stryker API];
      I --> J[Transmit to Endoscopy Tower];
      J --> K[Feedback: "Pressure set to 15 mmHg"];
      K --> B;
  

3. Cross-Domain Application: Aerospace Flight Deck Subsystem Routing

• Enabling Description: The invention is applied to a commercial aircraft flight deck. The pilot's voice commands are captured by the avionics system. The "disparate ecosystems" are distinct flight-critical subsystems such as the Flight Management System (FMS, e.g., Honeywell), the satellite communication system (SATCOM, e.g., Inmarsat), and the cabin environmental controls (e.g., Liebherr). A command like "contact maintenance on the ground" is interpreted by the NLU. The contextual data is the flight phase (e.g., "en-route cruise over Atlantic"). The NLU module, trained on flight operations manuals, determines this command should be routed to the SATCOM system via an ACARS data link message. The connection manager formats the request and transmits it. If the command was "set cabin temperature to 22 degrees," the same system would route it to the environmental control subsystem.
• Mermaid Diagram:

  graph LR
      subgraph Flight Deck
          A(Pilot Utterance)
      end
      subgraph Avionics Core
          B(NLU Module)
          C(Connection Manager)
      end
      subgraph Disparate Subsystems
          D[FMS]
          E[SATCOM]
          F[Cabin Control]
      end
      A --> B
      B -- Context: Flight Phase --> B
      B -- Intent: Comms --> C
      C --> E
      B -- Intent: Navigation --> C
      C --> D
      B -- Intent: Environment --> C
      C --> F
  

4. Integration with Emerging Tech: AI-Powered API Command Synthesis and Blockchain Auditing

• Enabling Description: This derivative integrates a large language model (LLM) and a blockchain ledger. When the NLU module identifies the target ecosystem, instead of using a pre-programmed command format, it passes the user's intent to a generative AI model. This AI model has been trained on the API documentation for hundreds of IoT ecosystems. It synthesizes the exact, syntactically correct API call (e.g., a complex JSON payload) required for that specific ecosystem in real-time. Simultaneously, the connection manager generates a transaction containing the original utterance, the identified intent, the target ecosystem, the synthesized command, and a timestamp. This transaction is hashed and recorded on a private, immutable blockchain (e.g., Hyperledger Fabric). The feedback from the ecosystem (success/failure) is recorded in a subsequent block, creating a secure, tamper-proof audit trail for all commands, critical for security and high-value asset control.
• Mermaid Diagram:

  sequenceDiagram
      participant V as Vehicle Assistant
      participant NLU as NLU Module
      participant LLM as Generative AI
      participant CM as Connection Manager
      participant BC as Blockchain Ledger
      participant API as Target Ecosystem API
  
      V->>NLU: Utterance: "Make my house secure"
      NLU->>LLM: Intent: "lock_doors, arm_alarm"
      LLM->>CM: Synthesized Command (JSON payload)
      CM->>BC: Log TX: {utterance, intent, command, timestamp}
      CM->>API: Execute synthesized command
      API-->>CM: Feedback: {status: success}
      CM->>BC: Log TX: {feedback, timestamp}
      CM-->>V: Confirmation
  

5. Inverse/Failure Mode: Graceful Degradation to Local Deterministic Control

• Enabling Description: The system is designed for high-reliability environments where network connectivity may be intermittent (e.g., remote areas, underground tunnels). The system operates in two modes. In "Cloud-Connected Mode," it functions as described in the patent, using cloud-based NLU and AI. When connectivity is lost, it enters "Local Deterministic Mode." In this mode, the vehicle's head unit relies on a small, embedded speech recognition engine and a pre-defined, cached ruleset. This ruleset maps specific, simple phrases directly to commands for critical ecosystems (e.g., "house lockdown" maps to pre-authenticated API calls to both the door lock and security system ecosystems). No complex NLU or contextual interpretation occurs. The system only supports a limited vocabulary of 10-20 critical commands. When connectivity is restored, it automatically switches back to Cloud-Connected Mode and syncs any state changes.
• Mermaid Diagram:

  stateDiagram-v2
      [*] --> Disconnected
      Disconnected --> Connected: Network Detected
      Connected --> Disconnected: Connection Lost
  
      state Connected {
          direction LR
          CloudNLU: Full-feature NLU
          Context: Real-time Data
          DynamicRouting: Route to any ecosystem
          CloudNLU --> Context --> DynamicRouting
      }
  
      state Disconnected {
          EmbeddedASR: Limited Vocabulary
          CachedRules: Pre-defined command map
          CriticalOnly: Supports only essential commands
          EmbeddedASR --> CachedRules --> CriticalOnly
      }
  

---

Derivatives of Independent Claim 3 (Method Claim)

The core claim outlines the method of receiving utterances and context, translating to text, determining a target, transmitting, and receiving confirmation.

1. Process Substitution: End-to-End Spoken Language Understanding (SLU)

• Enabling Description: This method variation replaces the distinct, sequential steps of Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) with a single, unified end-to-end Spoken Language Understanding (SLU) model. The input to this model is the raw audio waveform of the user's utterance. The output is a structured intent object that directly includes the target ecosystem, the command, and its parameters (e.g., {'target': 'SimpliSafe_API', 'action': 'set_state', 'parameters': {'state': 'armed_away'}}). This eliminates the intermediate text representation, reducing potential errors from ASR transcription and lowering overall latency. The contextual data (location, time) is provided as an additional input vector to the SLU model during inference, allowing it to directly influence the audio-to-intent mapping.
• Mermaid Diagram:

  flowchart TD
      subgraph Traditional Method
          A[Audio Waveform] --> B(ASR Module);
          B --> C[Text: "arm the security system"];
          C --> D{NLU Module};
          E[Context Data] --> D;
          D --> F[Structured Intent];
      end
      subgraph SLU Method
          G[Audio Waveform] --> H{End-to-End SLU Model};
          I[Context Data] --> H;
          H --> J[Structured Intent];
      end
      F --> K((Transmit to Ecosystem));
      J --> K;
  

2. Operational Parameter Expansion: High-Frequency Trading (HFT) Command Routing

• Enabling Description: The method is applied to a high-frequency trading environment where low latency is critical. A trader's spoken utterance (e.g., "sell fifty thousand at market") is captured. The "contextual data" is not geographic location but rather sub-second market data, including stock volatility, order book depth, and breaking news sentiment analysis scores from a live data feed. The method determines the "target ecosystem" to be one of several available execution algorithms (e.g., "Iceberg Algorithm," "TWAP Algorithm," "Aggressive Market-Taker Bot"). Based on the high volatility context, the NLU determines that the "Aggressive Market-Taker Bot" is the appropriate target to ensure immediate execution. The command is transmitted to that algorithmic trading system's API, and confirmation of the trade execution is received within milliseconds. The feedback loop trains the model to associate specific market conditions and utterance types with the best-performing execution algorithms.
• Mermaid Diagram:

  graph TD
      A[Trader Utterance: "sell 50k at market"] --> B{NLU};
      C[Market Data Stream: High Volatility, Low Liquidity] --> B;
      B --> D{Determine Target Algorithm};
      D -- Volatility is high --> E[Aggressive Market-Taker Bot];
      D -- Volatility is low --> F((TWAP Algorithm));
      E --> G[Transmit Order];
      G --> H[Confirmation: Fill at $123.45];
      H --> B;
  

---

Combination Prior Art Scenarios

These scenarios describe the integration of the core invention of US12406663 with existing, open-source standards to create novel but obvious combinations.

1. Combination with Matter Protocol

• Enabling Description: The system acts as a high-level "contextual bridge" for Matter-enabled devices. While the Matter standard provides a common IP-based application layer for device interoperability within a home "fabric," it does not specify how a user's ambiguous command should be routed. This invention is combined with Matter by having the in-vehicle NLU module resolve ambiguity and select a target Matter device or group. For example, the command "turn on the lights" from a vehicle approaching home would cause the Connection Manager to issue a standard Matter command to the group.living_room_lights within the home's Matter fabric. The feedback is the standard success/failure code from the Matter command. The system learns user preferences for which lights to turn on when arriving home, a layer of intelligence not inherent in the Matter specification itself.
• Mermaid Diagram:

  sequenceDiagram
      participant V as Vehicle Assistant
      participant NLU as Contextual NLU
      participant CM as Connection Manager (Matter Controller)
      participant M as Home Matter Fabric
  
      V->>NLU: Utterance: "turn on the lights"
      NLU->>NLU: Context: "approaching home, 7 PM"
      NLU->>CM: Intent: "activate_lights", Target: "group.entryway"
      CM->>M: Matter Command: write-attribute(endpoint=group.entryway, attribute=on-off, value=true)
      M-->>CM: Matter Status: SUCCESS
      CM-->>V: Confirmation
  

2. Combination with MQTT (Message Queuing Telemetry Transport)

• Enabling Description: The Connection Manager is implemented as an intelligent MQTT client/broker. The vehicle assistant publishes all voice commands as messages to a generic MQTT topic, e.g., vehicle/vin123/commands/raw. The NLU/Connection Manager module subscribes to this topic. After processing the utterance and its context, it determines the target ecosystem. It then re-publishes a new, structured message to an ecosystem-specific topic (e.g., home/alexa/commands or home/security/commands). Dedicated bridges on the home network subscribe to these topics and translate the MQTT messages into API calls for non-native MQTT devices. The feedback loop is implemented by subscribing to response topics (e.g., home/alexa/responses), allowing the NLU model to be updated based on command success or failure.
• Mermaid Diagram:

  flowchart LR
      subgraph Vehicle
          A[Utterance] --> B{Vehicle Assistant};
          B -- Publishes --> C(MQTT Topic: `.../commands/raw`);
      end
      subgraph Cloud/Hub
          D{NLU/Connection Manager} -- Subscribes --> C;
          D -- Processes --> D;
          D -- Publishes --> E(MQTT Topic: `.../ecosystem_A/command`);
          D -- Publishes --> F(MQTT Topic: `.../ecosystem_B/command`);
      end
      subgraph Home
          G[Ecosystem A Bridge] -- Subscribes --> E;
          H[Ecosystem B Bridge] -- Subscribes --> F;
          G --> I[Device A];
          H --> J[Device B];
      end
  

3. Combination with OAuth 2.0 and OpenID Connect

• Enabling Description: The system's user onboarding and authentication process is built entirely on the OAuth 2.0 and OpenID Connect (OIDC) standards. To link a new ecosystem (e.g., Google Home), the user initiates a process from the vehicle's head unit. The system acts as an OAuth 2.0 client and redirects the user (e.g., to their phone) to the ecosystem's authorization server. The user authenticates and grants permission. The ecosystem's server returns an authorization code, which the vehicle system's backend exchanges for an access token and a refresh token. The "authentication cache" described in the patent is specifically an encrypted database for storing these OAuth 2.0 refresh tokens. The Connection Manager attaches the valid access token to every API call sent to the target ecosystem, adhering to this universal standard for secure, delegated access. The user's identity across systems can be federated using their OIDC identity token.
• Mermaid Diagram:

  sequenceDiagram
      participant User as User
      participant VA as Vehicle Assistant (Client)
      participant AS as Ecosystem Auth Server
      participant API as Ecosystem Resource API
  
      User->>VA: "Link my smart home"
      VA->>User: Redirect to AS for login/consent
      User->>AS: Logs in, gives consent
      AS->>VA: Returns Authorization Code
      VA->>AS: Exchanges Auth Code for Tokens
      AS->>VA: Returns Access Token + Refresh Token
      VA->>VA: Securely stores Refresh Token
      Note over VA, API: Later, for a command...
      VA->>API: API Request + Bearer Access Token
      API-->>VA: API Response