US 10200654

US Patent 10,200,654 Summary:

Title: Systems and methods for real time manipulation and interaction with multiple dynamic and synchronized video streams in an augmented or multi-dimensional space.

Assignee: ZUGARA Inc.

Inventors: Matthew SZYMCZYK, Alex Goldberg, Hans Forsman, Jack Benoff, Aaron Von Hungen, Blake Callens, Mark Weisheit, and Andrey Tkachuk.

Filing Date: February 27, 2013.

Issue Date: February 5, 2019.

Abstract: The patent proposes systems and methods for real-time manipulation of objects within multiple dynamic and augmented video streams that are synchronized in an augmented reality (AR) or multi-dimensional space. This approach allows a user of a first AR system to connect with users of a second (similar or different) AR system, sharing their initial and/or augmented video streams. Through real-time synchronization, participants can see, share, manipulate, and interact with augmented content/objects in their own or another user's augmented video streams.

Plain-Language Overview of Independent Claims:

• Independent Claim 1 (System): This claim describes a system that enables collaborative augmented reality. It includes a video camera providing a real-world video stream of a participant in a video chat. A "multiple stream sync module" receives an augmented video stream (containing 3D virtual objects from a database) from another participant. This module combines the received augmented video with the local real-world video in real-time to create a "multiplexed video stream." From this combined stream, one or more 3D virtual objects are extracted, and their 3D coordinates can be translated (moved) within the virtual space based on actions of the local participant. The resulting multiplexed stream is displayed, and a key feature is that both the local participant and the other participant can independently translate the coordinates of these 3D virtual objects within the shared 3D space.

• Independent Claim 32 (Method): This claim outlines a method for achieving the collaborative augmented reality described in Claim 1. It involves providing a real-world video stream from a participant's camera and receiving an augmented video stream with 3D virtual objects from another participant. The method then multiplexes these streams in real-time to generate a combined video stream. It further includes extracting virtual objects, translating their 3D coordinates based on a participant's action, displaying the combined stream, and allowing both the local and other participants to translate the coordinates of the virtual objects within the shared multi-dimensional space.

Legal Status Note:
As of April 26, 2026, the patent is indicated as "Active" and is set to expire on July 27, 2033.
The patent family is currently involved in litigation, including:

• A PTAB (Patent Trial and Appeal Board) case, IPR2026-00266, which is pending.
• A US case filed in the Texas Eastern District Court (case 2:25-cv-00437).
• Additionally, the patent notes that the first worldwide family litigation has been filed.

Searches for "10200654" in the USPTO database and CAFC 2026 dockets did not yield additional specific case details beyond what is already stated in the patent document itself. Therefore, the litigation information provided here relies on the details directly available within the patent text.