Prior art

This analysis focuses on identifying the most relevant prior art for US Patent 11126889, titled "Machine learning based prediction of human interactions with autonomous vehicles." The priority date for US11126889 is 2017-07-05. Therefore, any prior art must have a filing or publication date before this to potentially anticipate claims under 35 U.S.C. § 102.

The patent US11126889 explicitly lists a significant number of prior art references on its Google Patents page. These references generally fall into categories related to autonomous vehicles, object detection and tracking, and methods for predicting behavior of road users using various sensor data. Many of the cited references are from the same inventor, Raquel Urtasun, or co-inventors, and primarily deal with methods for perceiving and understanding urban environments for autonomous driving.

To streamline this analysis, a subset of particularly relevant prior art documents has been selected, focusing on those published significantly before the priority date and directly addressing core aspects of human interaction prediction for autonomous vehicles.

Most Relevant Prior Art for US11126889:

Here is an analysis of key prior art documents that appear to be most relevant based on their publication dates and general subject matter, in relation to the claims of US11126889 which involve machine learning for predicting human interactions with autonomous vehicles:

1. US 2007/0030097 A1

   • Full Citation: Krumm, John C. and Simon, David M., "Estimating position, orientation, and motion of a vehicle relative to its environment", Published: February 8, 2007.
   • Publication/Filing Date: Publication: 2007-02-08. Priority/Filing dates are prior to 2017-07-05.
   • Brief Description: This patent application describes systems and methods for estimating the position, orientation, and motion of a vehicle relative to its environment. It involves using sensor data (e.g., from cameras, radar, lidar) to generate observations of environmental features and then applying algorithms (e.g., Extended Kalman Filters) to estimate the vehicle's state. While not directly focused on predicting human interaction, it lays foundational work for vehicle perception of its environment and objects within it.
   • Potential Anticipated Claim(s) (under 35 U.S.C. § 102): This reference could potentially anticipate foundational aspects of sensing and environmental perception described in claims of US11126889 that relate to receiving images or video segments of a road scene from a vehicle's perspective. For example, it could challenge claims directed to receiving a first at least one of an image and a video segment of a road scene, the first at least one of an image and a video segment being taken from a perspective of a participant in the road scene.

2. US 2012/0158223 A1

   • Full Citation: Dolgov, Dmitri et al., "Predictive collision warning system for an autonomous vehicle", Published: June 21, 2012.
   • Publication/Filing Date: Publication: 2012-06-21. Priority/Filing dates are prior to 2017-07-05.
   • Brief Description: This application details a collision warning system for autonomous vehicles that uses sensor data to identify objects and predict their future positions. It involves generating trajectories for objects and the autonomous vehicle, and then predicting potential collisions based on these trajectories. This is highly relevant as it explicitly addresses prediction for autonomous vehicles to avoid interactions, which is a core problem US11126889 aims to solve.
   • Potential Anticipated Claim(s) (under 35 U.S.C. § 102): This reference directly addresses predicting future movements of objects (including humans) in the context of autonomous vehicles. It could potentially anticipate claims relating to generating a prediction of user behavior in the second at least one image or video segment based on the application of the model to the second at least one image or video segment, especially concerning the actions of pedestrians or vehicles. Specifically, claims related to predicting "motion vectors" of people to make decisions on vehicle control are highly susceptible.

3. US 2014/0200810 A1

   • Full Citation: Van Der Burg, Geert T. et al., "Method and system for providing road safety information", Published: July 17, 2014.
   • Publication/Filing Date: Publication: 2014-07-17. Priority/Filing dates are prior to 2017-07-05.
   • Brief Description: This patent application describes a system that provides road safety information by analyzing road situations using images from a vehicle and detecting elements like pedestrians, bicycles, or other vehicles. It assesses risks and provides warnings to the driver. This is relevant for identifying road participants and their potential impact on safety.
   • Potential Anticipated Claim(s) (under 35 U.S.C. § 102): This reference could potentially anticipate claims related to receiving a first at least one of an image and a video segment of a road scene... including at least one of a pedestrian, a cyclist, and a motor vehicle, and broadly aspects of generating a prediction of user behavior. The focus on analyzing road situations and detecting road users directly overlaps.

4. US 2015/0134267 A1

   • Full Citation: Krumm, John C. et al., "Probabilistic prediction of human activity", Published: May 14, 2015.
   • Publication/Filing Date: Publication: 2015-05-14. Priority/Filing dates are prior to 2017-07-05.
   • Brief Description: This application focuses on probabilistically predicting human activity (e.g., walking, running, stopping) in various environments using sensor data. This directly relates to the core invention of US11126889, which is about predicting human interactions.
   • Potential Anticipated Claim(s) (under 35 U.S.C. § 102): This reference directly anticipates the concept of generating a prediction of user behavior and specifically the likelihood of the action of a road scene participant. The use of "probabilistic prediction" aligns with the "likelihood of the action includes an ordinal value associated with a probability of the action" element found in claims of US11126889.

5. US 2016/0012586 A1

   • Full Citation: Krumm, John C. et al., "Generating predictions for future states of objects of interest in an environment", Published: January 14, 2016.
   • Publication/Filing Date: Publication: 2016-01-14. Priority/Filing dates are prior to 2017-07-05.
   • Brief Description: This patent application describes methods for generating predictions for future states of objects of interest in an environment, which can be applied to autonomous vehicles. It emphasizes predicting various states, not just motion. This is highly relevant to US11126889's emphasis on predicting "state of mind" or intent beyond simple motion vectors.
   • Potential Anticipated Claim(s) (under 35 U.S.C. § 102): This reference could potentially anticipate claims relating to the broader concept of "state of mind" or "predicted behavior" of road users beyond just motion vectors. The "action includes one of the at least one of the pedestrian, the cyclist, and the motor vehicle staying in place, changing lanes, and crossing a street" and "the likelihood of the action includes an ordinal value associated with a probability of the action" are directly addressed by predicting future states. This also aligns with the objective of US11126889 to overcome the limitations of only predicting "motion vectors."

Numerous Urtasun et al. References (e.g., US 2016/0091871 A1 through US 2016/0134963 A1, all published May 2016)
There is a large cluster of patent applications by Urtasun et al., all published in May 2016, preceding the priority date of US11126889. These patents generally relate to systems and methods for various aspects of perception, understanding, and decision-making for autonomous vehicles. Given the sheer number and the common inventive entity (Perceptive Automata LLC, the current assignee of US11126889, is associated with some of these inventors), these likely represent a family of related inventions from the same research group. It is highly probable that specific elements and combinations within the claims of US11126889 may be found across this collection of Urtasun et al. prior art, especially those related to:

• Image and video processing for autonomous vehicles.
• Detection, tracking, and classification of pedestrians, cyclists, and other vehicles.
• Prediction of object motion and intent in urban environments.
• Use of machine learning models (e.g., neural networks) for prediction tasks.

Without reviewing each of the numerous Urtasun et al. references and the full claims of US11126889 in detail, it is challenging to pinpoint exact claim anticipation for each. However, as a collective, they represent highly relevant prior art that likely covers many aspects of machine learning based prediction of human interactions with autonomous vehicles. Any claim of US11126889 that relies on generic machine learning for perception or prediction of road users, or generating stimuli, would need careful comparison against this broad body of work.

Summary of Potential Anticipation
The listed prior art collectively addresses core elements of US11126889, specifically:

• Receiving image/video data from a vehicle's perspective.
• Identifying and tracking road scene participants (pedestrians, cyclists, vehicles).
• Predicting future behavior or states of these participants, often probabilistically.
• Generating predictions for autonomous vehicle navigation and collision avoidance.

The unique aspects of US11126889 may lie in the specific methodology of generating stimulus data for human observers to collect response data (including implicit measures like eye-tracking and response time) to train a model that predicts "state of mind" or "intention" in a nuanced way that goes beyond simple motion vector prediction. Any claims in US11126889 that heavily rely on the human-in-the-loop data collection, the creation of "derived stimuli" with manipulated content, and the use of diverse summary statistics from human responses to train the machine learning model, would be the most defensible against the cited prior art if these specific steps are not fully disclosed in a single reference.The most relevant prior art for US patent 11126889, "Machine learning based prediction of human interactions with autonomous vehicles," can be identified by examining the references cited within the patent itself and assessing their publication dates relative to US11126889's priority date of 2017-07-05. The cited references predominantly relate to autonomous driving, object detection, tracking, and behavior prediction using various sensor data. Many of these patents share common inventors, such as Raquel Urtasun, and are likely part of a broader research and development effort in this field.

A detailed review of the cited prior art reveals several key documents