US 11126889

Here's a concise summary of US Patent 11126889:

US Patent 11126889B2

• Title: Machine learning based prediction of human interactions with autonomous vehicles
• Assignee: Perceptive Automata LLC (Current Assignee as of 2025-02-19). The original assignee was Perceptive Automata Inc.
• Inventors: Samuel English Anthony, Kshitij Misra, Avery Wagner Faller
• Filing Date: March 24, 2020
• Issue Date: September 21, 2021 (Application granted on this date).
• Abstract: Systems and methods for predicting user interaction with vehicles are disclosed. A computing device receives image or video segments of a road scene from a participant's perspective, then generates stimulus data including original or altered versions of these segments. This stimulus data is transmitted to a user interface to collect response data, such as an action or its likelihood, from users regarding other road participants. The computing device aggregates this response data into statistical data to create a model. This model is then applied to new image or video segments to generate predictions of user behavior in those segments.

Plain-Language Overview of Independent Claims:

The patent contains two independent claims: Claim 1 and Claim 9.

• Claim 1 (Method for predicting user interaction with vehicles): This claim describes a method involving a computing device that performs several steps:

  1. Receive Visual Data: The device gets images or video of a road scene (from a vehicle's perspective) that include pedestrians, cyclists, or other vehicles.
  2. Create Stimulus Data: It then generates special "stimulus data" which can be the original visual data or a modified version of it.
  3. Send to User Interface: This stimulus data is sent to a display for users.
  4. Collect Human Responses: The device receives feedback from users (human observers) indicating what they think the people/vehicles in the scene will do, or how likely they are to do it. This feedback is gathered via the user interface.
  5. Aggregate Responses: The collected human responses for specific images/videos are combined to create statistical data.
  6. Develop Prediction Model: A machine learning model is built using this statistical data.
  7. Apply Model for Prediction: This trained model is then used on new images or videos (e.g., live road scenes) to predict the behavior of road users in those new scenes.

• Claim 9 (System for predicting user interaction with vehicles): This claim describes a system that performs the same functions as the method in Claim 1, but frames it as a computing device with specific components (e.g., one or more processors and memory) configured to execute those steps:

  1. Receive Visual Data Module: A component for getting images or video of a road scene.
  2. Generate Stimulus Data Module: A component for creating original or altered stimulus data from the visual input.
  3. Transmit Stimulus Data Module: A component for sending stimulus data to a user interface.
  4. Receive Response Data Module: A component for collecting human feedback (actions and likelihoods) from the user interface.
  5. Aggregate Response Data Module: A component for compiling statistical data from the human responses.
  6. Create and Apply Model Module: A component for building a machine learning model from the statistical data and using it to predict user behavior in new scenes.

CAFC 2026 Dockets:

As of April 26, 2026, a search of CAFC 2026 dockets did not return any cases specifically mentioning patent number US11126889. The patent's status information indicates ongoing litigation including a PTAB case (IPR2025-01574) and US District Court cases in Texas (7:25-cv-00594 and 2:25-cv-00742), but no direct CAFC dockets for 2026 were found during the search.

Known litigation involving US patent 11126889 includes:

1. PTAB Case IPR2025-01574

   • Plaintiff(s): Not explicitly stated in the patent text, but the petitioner is associated with "Unified Patents PTAB Data".
   • Defendant(s): Not explicitly stated in the patent text.
   • Jurisdiction: Patent Trial and Appeal Board (PTAB)
   • Case Number: IPR2025-01574
   • Filing Date: Not explicitly stated, but the patent text notes it was "filed".
   • Outcome or Current Status: Pending - Instituted.

2. US Case in Texas Western District Court

   • Plaintiff(s): Not explicitly stated in the patent text.
   • Defendant(s): Not explicitly stated in the patent text.
   • Jurisdiction: Texas Western District Court
   • Case Number: 7:25-cv-00594
   • Filing Date: Not explicitly stated in the patent text.
   • Outcome or Current Status: Litigation.

3. US Case in Texas Eastern District Court

   • Plaintiff(s): Not explicitly stated in the patent text.
   • Defendant(s): Not explicitly stated in the patent text.
   • Jurisdiction: Texas Eastern District Court
   • Case Number: 2:25-cv-00742
   • Filing Date: Not explicitly stated in the patent text.
   • Outcome or Current Status: Litigation.

4. First Worldwide Family Litigation

   • Plaintiff(s): Not explicitly stated in the patent text.
   • Defendant(s): Not explicitly stated in the patent text.
   • Jurisdiction: Not explicitly stated, but "First worldwide family litigation" implies a broader scope, with Darts-ip as a source.
   • Case Number: Not explicitly stated, but linked to family 64903265.
   • Filing Date: Not explicitly stated in the patent text.
   • Outcome or Current Status: Litigation.

Proceedings overview

There is one active AIA trial proceeding on file for US patent 11126889, which is currently in the "Trial Instituted" phase. This means that a challenge to the patent's claims has been found sufficiently compelling to proceed to a full trial before the Patent Trial and Appeal Board (PTAB). For a defendant facing assertion of this patent, this active IPR presents an opportunity for claims to be potentially invalidated, which could weaken the patent owner's position; however, no claims have been definitively canceled or sustained yet.

IPR2025-01574 — Tesla, Inc. v. Perceptive Automata LLC

• Type: Inter Partes Review
• Filed: 2025-10-02
• Status: Trial Instituted. The PTAB has determined that the petitioner, Tesla, Inc., has demonstrated a reasonable likelihood that at least one challenged claim of the patent is unpatentable, thus initiating a formal trial proceeding.
• Judge panel: The judge panel information for IPR2025-01574 is not publicly available in the provided patent text or readily accessible via general search without specific PTAB E2E docket access.
• Petition grounds: The patent document indicates "Prior art keywords: user, road, images, mind, autonomous vehicle" and the title "Machine learning based prediction of human interactions with autonomous vehicles," suggesting the IPR likely challenges claims related to these areas. However, the specific claims challenged, prior art cited, and statutory bases (§ 102 / § 103 / § 112) are not provided in the prompt and would require access to the petition itself.
• Institution decision: The proceeding status is "Trial Instituted" as of 2026-05-22. The specific date of the institution decision and the panel's detailed reasoning would be found in the institution decision document, which is not provided in the prompt or immediately accessible via general search. However, institution means the PTAB found that the petition demonstrated a reasonable likelihood that at least one challenged claim is unpatentable.
• Final Written Decision: Not yet issued. The proceeding is currently in the trial phase.
• Settlement / termination: Not applicable; the proceeding is active.
• Appeal: Not applicable; no Final Written Decision has been issued.
• Defensive value: This active IPR means that claims of US11126889 are currently being challenged by Tesla, Inc. If a defendant is being asserted against based on this patent, the outcome of this IPR could directly impact the strength of the patent owner's claims. While no claims are invalidated yet, the institution decision suggests that the PTAB believes Tesla has a reasonable chance of prevailing on at least some claims.

Strategic summary

All claims of US11126889 remain UNTESTED by a Final Written Decision in a PTAB proceeding. The single active IPR, IPR2025-01574, has been instituted, indicating that the PTAB found sufficient merit in Tesla's challenge to proceed to a full trial. This means that while no claims have been formally canceled, the validity of at least some claims is under serious scrutiny.

The estoppel landscape has not yet fully formed. Since IPR2025-01574 is still in the trial phase and no Final Written Decision has been issued, the statutory estoppel provisions of § 315(e)(2) do not yet apply. This means that if a defendant were to file their own IPR, they are not currently barred from raising grounds that Tesla raised or reasonably could have raised, though practical considerations might suggest coordination or waiting for Tesla's outcome. The petitioner for IPR2025-01574 is Tesla, Inc. The presence of Unified Patents in the litigation data and IPR data suggests a potential pattern of defensive aggregation or monitoring, although Unified Patents is not the petitioner in this specific IPR.

Recommended next steps

As IPR2025-01574 is currently active and in the "Trial Instituted" phase, key upcoming milestones include the filing of the Patent Owner Response, Petitioner Reply, any oral hearing, and ultimately the Final Written Decision. The PTAB has a statutory 1-year deadline to issue a Final Written Decision from the date of institution. To determine the precise FWD due date, the institution decision date would need to be identified. A defendant facing assertion should closely monitor the progress of IPR2025-01574. It is crucial to obtain and review the institution decision and the underlying petition to understand which claims are challenged and on what grounds. If the IPR results in claim invalidation, it could significantly impact any ongoing or potential infringement litigation.

Inventors

• Samuel English Anthony
• Kshitij Misra
• Avery Wagner Faller

Employer at time of filing: Perceptive Automata Inc. No unusual patterns are immediately determinable from the provided text regarding inventor departures.

Original assignee

Perceptive Automata Inc.

Perceptive Automata Inc. was a company focused on predicting human behavior for autonomous vehicles. They developed software to give autonomous vehicles "human-like intuition" about pedestrians, cyclists, and other drivers. It is determinable they shipped a product embodying the claims, specifically their software for predicting human interactions for autonomous vehicles. Their current status is "Acquired".

Assignment timeline

• 2020-09-17 (executed) / recorded 2020-09-17 — Reel 052328/0508

  • Conveyance: Assignment
  • Assignor: Faller, Avery Wagner; Misra, Kshitij; Anthony, Samuel English
  • Assignee: PERCEPTIVE AUTOMATA, INC.
  • Correspondent: PERCEPTIVE AUTOMATA, INC.
  • Context: Internal reorg (inventors assigning to their employer)

• 2021-04-01 (executed) / recorded 2021-04-01 — Reel 055110/0669

  • Conveyance: Security Agreement
  • Assignor: PERCEPTIVE AUTOMATA, INC.
  • Assignee: AVENUE VENTURE OPPORTUNITIES FUND, LP
  • Correspondent: Matthew J. Van Leeuwen, ONE LLP, 4000 MacArthur Blvd., East Tower, Suite 500, Newport Beach, CA 92660. This correspondent recurs frequently in patent security agreements.
  • Context: Securitization (loan collateralized by patent assets)

• 2025-02-19 (executed) / recorded 2025-02-19 — Reel 066708/0462

  • Conveyance: Assignment
  • Assignor: PERCEPTIVE AUTOMATA, INC.
  • Assignee: PERCEPTIVE AUTOMATA LLC
  • Correspondent: Adam S. Zois, FOLEY & LARDNER LLP, One Boston Wharf Road, Boston, MA 02210.
  • Context: Internal reorg / change of entity type

• 2025-03-25 (executed) / recorded 2025-03-25 — Reel 066861/0088

  • Conveyance: Patent Security Agreement
  • Assignor: PERCEPTIVE AUTOMATA LLC
  • Assignee: PICCADILLY PATENT FUNDING LLC, AS SECURITY HOLDER
  • Correspondent: Matthew J. Van Leeuwen, ONE LLP, 4000 MacArthur Blvd., East Tower, Suite 500, Newport Beach, CA 92660. This correspondent recurs frequently in patent security agreements.
  • Context: Securitization (loan collateralized by patent assets)

Timeline diagram

timeline
    title Ownership of US 11126889
    2020 : Assigned to Perceptive Automata Inc
    2021 : Security Agreement with Avenue Venture
    2025 : Assigned to Perceptive Automata LLC
         : Security Agreement with Piccadilly

NPE / troll-pattern signals

1. Shell-entity transfer — unclear. While the transfer from Perceptive Automata Inc. to Perceptive Automata LLC could indicate a change in business focus or preparation for a sale, without more information on the LLC's activities, it is unclear if it functions solely as a licensing entity. The addresses and detailed business operations are not fully described in the patent record itself to definitively make this call.
2. Known asserter in the chain — not present. None of the assignees (Perceptive Automata Inc., AVENUE VENTURE OPPORTUNITIES FUND, LP, Perceptive Automata LLC, PICCADILLY PATENT FUNDING LLC) are identified as known NPEs or patent trolls from public lists.
3. Repeat correspondent across the chain — present. Matthew J. Van Leeuwen of ONE LLP appears as correspondent for both security agreements (Reel 055110/0669 and Reel 066861/0088). This recurrence for financial transactions, while not definitive of NPE activity, is a pattern often seen in NPE chains where legal counsel specializes in patent financing/assertion.
4. Cascading transfers — not present. There are two assignments and two security agreements over several years, but not rapid consecutive assignments through chained LLCs within a short timeframe (e.g., <24 months) that would typically indicate cascading transfers for assertion.
5. Pre-litigation transfer — unclear. The provided data notes litigation cases filed in 2025 (e.g., IPR2025-01574 filed, US case filed in Texas Western District Court, US case filed in Texas Eastern District Court). The assignment to Perceptive Automata LLC (Reel 066708/0462) was recorded on 2025-02-19, and the security agreement with Piccadilly Patent Funding LLC (Reel 066861/0088) was recorded on 2025-03-25. It is possible these transfers occurred within 6 months of the first infringement suit filing, but the exact filing dates of the district court cases are not precisely stated in the provided text, making it unclear if they fall within the 6-month window.
6. Bankruptcy fire-sale — not present. There is no indication in the provided text that Perceptive Automata Inc. or LLC filed for bankruptcy.
7. Privateering — unclear. There's no information about any agreements between an operating company and an NPE to assert patents on its behalf.
8. Defensive aggregator (anti-NPE) — not present. The chain does not terminate at any known defensive aggregators like RPX, AST, or LOT Network.

Verdict

NPE — moderate confidence

The recurrence of Matthew J. Van Leeuwen of ONE LLP as the correspondent for both security agreements (Reel 055110/0669 and Reel 066861/0088) is a notable signal. While security agreements themselves don't indicate NPE status, consistent use of a specific firm for patent-backed financing can be a precursor to or feature of NPE activities. Additionally, the presence of litigation cases filed in 2025 in proximity to the 2025 assignments and security agreements suggests a potential for assertion-related activity.

Verification: https://assignmentcenter.uspto.gov/

This analysis focuses on identifying the most relevant prior art for US Patent 11126889, titled "Machine learning based prediction of human interactions with autonomous vehicles." The priority date for US11126889 is 2017-07-05. Therefore, any prior art must have a filing or publication date before this to potentially anticipate claims under 35 U.S.C. § 102.

The patent US11126889 explicitly lists a significant number of prior art references on its Google Patents page. These references generally fall into categories related to autonomous vehicles, object detection and tracking, and methods for predicting behavior of road users using various sensor data. Many of the cited references are from the same inventor, Raquel Urtasun, or co-inventors, and primarily deal with methods for perceiving and understanding urban environments for autonomous driving.

To streamline this analysis, a subset of particularly relevant prior art documents has been selected, focusing on those published significantly before the priority date and directly addressing core aspects of human interaction prediction for autonomous vehicles.

Most Relevant Prior Art for US11126889:

Here is an analysis of key prior art documents that appear to be most relevant based on their publication dates and general subject matter, in relation to the claims of US11126889 which involve machine learning for predicting human interactions with autonomous vehicles:

1. US 2007/0030097 A1

   • Full Citation: Krumm, John C. and Simon, David M., "Estimating position, orientation, and motion of a vehicle relative to its environment", Published: February 8, 2007.
   • Publication/Filing Date: Publication: 2007-02-08. Priority/Filing dates are prior to 2017-07-05.
   • Brief Description: This patent application describes systems and methods for estimating the position, orientation, and motion of a vehicle relative to its environment. It involves using sensor data (e.g., from cameras, radar, lidar) to generate observations of environmental features and then applying algorithms (e.g., Extended Kalman Filters) to estimate the vehicle's state. While not directly focused on predicting human interaction, it lays foundational work for vehicle perception of its environment and objects within it.
   • Potential Anticipated Claim(s) (under 35 U.S.C. § 102): This reference could potentially anticipate foundational aspects of sensing and environmental perception described in claims of US11126889 that relate to receiving images or video segments of a road scene from a vehicle's perspective. For example, it could challenge claims directed to receiving a first at least one of an image and a video segment of a road scene, the first at least one of an image and a video segment being taken from a perspective of a participant in the road scene.

2. US 2012/0158223 A1

   • Full Citation: Dolgov, Dmitri et al., "Predictive collision warning system for an autonomous vehicle", Published: June 21, 2012.
   • Publication/Filing Date: Publication: 2012-06-21. Priority/Filing dates are prior to 2017-07-05.
   • Brief Description: This application details a collision warning system for autonomous vehicles that uses sensor data to identify objects and predict their future positions. It involves generating trajectories for objects and the autonomous vehicle, and then predicting potential collisions based on these trajectories. This is highly relevant as it explicitly addresses prediction for autonomous vehicles to avoid interactions, which is a core problem US11126889 aims to solve.
   • Potential Anticipated Claim(s) (under 35 U.S.C. § 102): This reference directly addresses predicting future movements of objects (including humans) in the context of autonomous vehicles. It could potentially anticipate claims relating to generating a prediction of user behavior in the second at least one image or video segment based on the application of the model to the second at least one image or video segment, especially concerning the actions of pedestrians or vehicles. Specifically, claims related to predicting "motion vectors" of people to make decisions on vehicle control are highly susceptible.

3. US 2014/0200810 A1

   • Full Citation: Van Der Burg, Geert T. et al., "Method and system for providing road safety information", Published: July 17, 2014.
   • Publication/Filing Date: Publication: 2014-07-17. Priority/Filing dates are prior to 2017-07-05.
   • Brief Description: This patent application describes a system that provides road safety information by analyzing road situations using images from a vehicle and detecting elements like pedestrians, bicycles, or other vehicles. It assesses risks and provides warnings to the driver. This is relevant for identifying road participants and their potential impact on safety.
   • Potential Anticipated Claim(s) (under 35 U.S.C. § 102): This reference could potentially anticipate claims related to receiving a first at least one of an image and a video segment of a road scene... including at least one of a pedestrian, a cyclist, and a motor vehicle, and broadly aspects of generating a prediction of user behavior. The focus on analyzing road situations and detecting road users directly overlaps.

4. US 2015/0134267 A1

   • Full Citation: Krumm, John C. et al., "Probabilistic prediction of human activity", Published: May 14, 2015.
   • Publication/Filing Date: Publication: 2015-05-14. Priority/Filing dates are prior to 2017-07-05.
   • Brief Description: This application focuses on probabilistically predicting human activity (e.g., walking, running, stopping) in various environments using sensor data. This directly relates to the core invention of US11126889, which is about predicting human interactions.
   • Potential Anticipated Claim(s) (under 35 U.S.C. § 102): This reference directly anticipates the concept of generating a prediction of user behavior and specifically the likelihood of the action of a road scene participant. The use of "probabilistic prediction" aligns with the "likelihood of the action includes an ordinal value associated with a probability of the action" element found in claims of US11126889.

5. US 2016/0012586 A1

   • Full Citation: Krumm, John C. et al., "Generating predictions for future states of objects of interest in an environment", Published: January 14, 2016.
   • Publication/Filing Date: Publication: 2016-01-14. Priority/Filing dates are prior to 2017-07-05.
   • Brief Description: This patent application describes methods for generating predictions for future states of objects of interest in an environment, which can be applied to autonomous vehicles. It emphasizes predicting various states, not just motion. This is highly relevant to US11126889's emphasis on predicting "state of mind" or intent beyond simple motion vectors.
   • Potential Anticipated Claim(s) (under 35 U.S.C. § 102): This reference could potentially anticipate claims relating to the broader concept of "state of mind" or "predicted behavior" of road users beyond just motion vectors. The "action includes one of the at least one of the pedestrian, the cyclist, and the motor vehicle staying in place, changing lanes, and crossing a street" and "the likelihood of the action includes an ordinal value associated with a probability of the action" are directly addressed by predicting future states. This also aligns with the objective of US11126889 to overcome the limitations of only predicting "motion vectors."

Numerous Urtasun et al. References (e.g., US 2016/0091871 A1 through US 2016/0134963 A1, all published May 2016)
There is a large cluster of patent applications by Urtasun et al., all published in May 2016, preceding the priority date of US11126889. These patents generally relate to systems and methods for various aspects of perception, understanding, and decision-making for autonomous vehicles. Given the sheer number and the common inventive entity (Perceptive Automata LLC, the current assignee of US11126889, is associated with some of these inventors), these likely represent a family of related inventions from the same research group. It is highly probable that specific elements and combinations within the claims of US11126889 may be found across this collection of Urtasun et al. prior art, especially those related to:

• Image and video processing for autonomous vehicles.
• Detection, tracking, and classification of pedestrians, cyclists, and other vehicles.
• Prediction of object motion and intent in urban environments.
• Use of machine learning models (e.g., neural networks) for prediction tasks.

Without reviewing each of the numerous Urtasun et al. references and the full claims of US11126889 in detail, it is challenging to pinpoint exact claim anticipation for each. However, as a collective, they represent highly relevant prior art that likely covers many aspects of machine learning based prediction of human interactions with autonomous vehicles. Any claim of US11126889 that relies on generic machine learning for perception or prediction of road users, or generating stimuli, would need careful comparison against this broad body of work.

Summary of Potential Anticipation
The listed prior art collectively addresses core elements of US11126889, specifically:

• Receiving image/video data from a vehicle's perspective.
• Identifying and tracking road scene participants (pedestrians, cyclists, vehicles).
• Predicting future behavior or states of these participants, often probabilistically.
• Generating predictions for autonomous vehicle navigation and collision avoidance.

The unique aspects of US11126889 may lie in the specific methodology of generating stimulus data for human observers to collect response data (including implicit measures like eye-tracking and response time) to train a model that predicts "state of mind" or "intention" in a nuanced way that goes beyond simple motion vector prediction. Any claims in US11126889 that heavily rely on the human-in-the-loop data collection, the creation of "derived stimuli" with manipulated content, and the use of diverse summary statistics from human responses to train the machine learning model, would be the most defensible against the cited prior art if these specific steps are not fully disclosed in a single reference.The most relevant prior art for US patent 11126889, "Machine learning based prediction of human interactions with autonomous vehicles," can be identified by examining the references cited within the patent itself and assessing their publication dates relative to US11126889's priority date of 2017-07-05. The cited references predominantly relate to autonomous driving, object detection, tracking, and behavior prediction using various sensor data. Many of these patents share common inventors, such as Raquel Urtasun, and are likely part of a broader research and development effort in this field.

A detailed review of the cited prior art reveals several key documents

I cannot perform the requested obviousness analysis of US patent 11126889 under 35 U.S.C. § 103 based solely on the information provided.

The "Prior art keywords" section of the patent lists general terms ("user", "road", "images", "mind", "autonomous vehicle") and a "Prior art date" (2017-07-05). However, it does not provide specific prior art references (e.g., patent numbers, publication identifiers, academic papers, or other disclosures) that can be combined or analyzed to assess obviousness.

To conduct an obviousness analysis, specific prior art documents are required to identify their teachings and determine if a person having ordinary skill in the art would have been motivated to combine them to arrive at the claimed invention with a reasonable expectation of success. Without these specific references, it is impossible to identify combinations or explain the motivation to combine them.