US 9860450

US patent 9860450, titled "Method and apparatus to correct digital video to counteract effect of camera shake," has the following details:

• Title: Method and apparatus to correct digital video to counteract effect of camera shake
• Assignee: Clear Imaging Research LLC (Current and Original)
• Inventor: Fatih M. Ozluturk
• Filing Date: 2017-02-13
• Issue Date (Publication Date): 2018-01-02
• Abstract: The patent describes a method and apparatus for correcting camera shake in digital video using signal processing techniques. It involves capturing a sequence of digital images (video), detecting imaging device movement with motion sensors, and recording this motion information. A processor then determines a correcting filter based on the motion data and user input, modifies the image sequence, combines the modified images to produce a final corrected video, and displays it.

Plain-Language Overview of Independent Claims:

• Claim 1 (Method): This claim describes a method used in an imaging device. It involves:

  1. Capturing a video (sequence of images) with an image sensor and storing it.
  2. Detecting and synchronously storing motion information of the device during the capture of one or more images using motion sensors.
  3. A processor calculating vertical and horizontal shift values for one or more images based on this motion information.
  4. The processor then modifying these images using the calculated shift values.
  5. Combining the modified images into a final video.
  6. Storing this final video.

• Claim 14 (Imaging Device): This claim describes an imaging device itself. It comprises:

  1. An image sensor for capturing and storing a video (sequence of images).
  2. One or more motion sensors to detect and synchronously store motion information of the device during image capture.
  3. A processor configured to:
     • Determine vertical and horizontal shift values for images based on motion information.
     • Modify images based on these shift values.
     • Combine the modified images to create a final video.
  4. A memory configured to store the final video.

• Claim 28 (Method with Compression): This claim describes a method similar to Claim 1, but explicitly includes video compression. It involves:

  1. Capturing a video (sequence of images) with an image sensor.
  2. Detecting motion information for one or more images during capture using motion sensors.
  3. A processor calculating vertical and horizontal shift values for one or more images based on this motion information.
  4. The processor modifying these images based on the shift values.
  5. Combining the modified images and applying a video compression technique to obtain a final video.
  6. Storing this final video in memory.

• Claim 29 (Imaging Device with Compression): This claim describes an imaging device similar to Claim 14, but explicitly includes video compression. It comprises:

  1. An image sensor for capturing a video (sequence of images).
  2. One or more motion sensors to detect motion information of the device during image capture.
  3. A processor configured to:
     • Determine vertical and horizontal shift values for images based on motion information.
     • Modify images based on these shift values.
     • Combine the modified images and apply a video compression technique to obtain a final video.
  4. A memory configured to store the final video.

Litigation Information:

The Google Patents entry for US9860450B2 indicates that the patent family has litigation. Specifically, there is a "US case filed in Court of Appeals for the Federal Circuit" with a link to case 26-1485. There is also a PTAB case IPR2026-00181 filed (Pending). The CAFC 2026 dockets provided in the search results list scheduled cases for May and June 2026, but the patent number 9860450 or its related case number 26-1485 is not explicitly visible in the snippets from the May or June schedules. Therefore, while litigation at the CAFC is confirmed by the patent record, specific details for scheduled hearings in May or June 2026 are not immediately available from the provided docket snippets.

Known litigation involving US patent 9860450:

1. US case filed in Court of Appeals for the Federal Circuit

   • Jurisdiction: Court of Appeals for the Federal Circuit
   • Case number: 26-1485
   • Filing Date: Not explicitly stated in the provided text.
   • Outcome or Current Status: Critical. Plaintiff(s) and Defendant(s) are not explicitly stated in the provided text.

2. US case filed in California Southern District Court

   • Jurisdiction: California Southern District Court
   • Case number: 3:25-cv-00221
   • Filing Date: Not explicitly stated in the provided text.
   • Outcome or Current Status: Litigation. Plaintiff(s) and Defendant(s) are not explicitly stated in the provided text.

3. PTAB case IPR2026-00181

   • Jurisdiction: PTAB
   • Case number: IPR2026-00181
   • Filing Date: Not explicitly stated in the provided text.
   • Outcome or Current Status: Pending. Petitioner, Plaintiff(s), and Defendant(s) are not explicitly stated in the provided text.

4. US case filed in California Southern District Court

   • Jurisdiction: California Southern District Court
   • Case number: 3:23-cv-00673
   • Filing Date: Not explicitly stated in the provided text.
   • Outcome or Current Status: Litigation. Plaintiff(s) and Defendant(s) are not explicitly stated in the provided text.

5. PTAB case IPR2020-01394

   • Jurisdiction: PTAB
   • Case number: IPR2020-01394
   • Filing Date: Not explicitly stated in the provided text.
   • Outcome or Current Status: Not Instituted - Procedural. Petitioner, Plaintiff(s), and Defendant(s) are not explicitly stated in the provided text.

6. US case filed in Texas Eastern District Court

   • Jurisdiction: Texas Eastern District Court
   • Case number: 2:19-cv-00326
   • Filing Date: Not explicitly stated in the provided text.
   • Outcome or Current Status: Litigation. Plaintiff(s) and Defendant(s) are not explicitly stated in the provided text.

7. US case filed in Texas Eastern District Court

   • Jurisdiction: Texas Eastern District Court
   • Case number: 2:25-cv-00240
   • Filing Date: Not explicitly stated in the provided text.
   • Outcome or Current Status: Litigation. Plaintiff(s) and Defendant(s) are not explicitly stated in the provided text.

Proceedings overview

One AIA trial proceeding has been filed against US patent 9860450. This proceeding, IPR2026-00181, resulted in a discretionary denial of institution, meaning no claims were invalidated or sustained on the merits. This outcome provides a strong defensive posture for the patent owner at this early stage of challenge.

IPR2026-00181 — Google LLC v. Clear Imaging Research LLC

• Type: Inter Partes Review
• Filed: 2026-01-29
• Status: Discretionary Denial — The petition for IPR was denied institution based on the Board's discretion, meaning the trial on the merits did not proceed.
• Judge panel: Information not publicly available in the provided data.
• Petition grounds: Information not publicly available in the provided data.
• Institution decision: Denied (date not specified in provided data; last modified 2026-05-26). The petition was denied institution on a discretionary basis, as indicated by the status "Discretionary Denial." This implies the Board chose not to institute the review for reasons other than the merits of the obviousness/anticipation arguments, potentially due to parallel litigation, timing, or other factors.
• Final Written Decision (if issued): Not applicable, as institution was denied.
• Settlement / termination: Not applicable, as institution was denied.
• Appeal: Not applicable, as no Final Written Decision was issued.
• Defensive value: The patent owner successfully prevented the institution of an IPR, meaning the patent's claims remain untested on the merits by this petitioner in this proceeding. This strengthens the patent owner's position against a future challenge from Google LLC (or its privies) on grounds that could have reasonably been raised in this petition, subject to estoppel rules.

Strategic summary

Currently, all claims of US9860450 remain UNTESTED in AIA trial proceedings, as the single filed IPR (IPR2026-00181) was denied institution on discretionary grounds. This means that, from a PTAB perspective, no claims have been canceled or sustained on the merits. The patent has not been narrowed through IPR.

Regarding the estoppel landscape, since IPR2026-00181 was denied institution on a discretionary basis, the full estoppel provisions of 35 U.S.C. § 315(e)(2) may not apply in the same way as with a FWD. However, the petitioner (Google LLC) and its privies may still be precluded from raising the same grounds or grounds that could have reasonably been raised in a subsequent PTAB petition, even without a merits-based FWD, depending on how "reasonably could have raised" is interpreted in the context of a discretionary denial. This generally means the prior art and arguments presented in the IPR2026-00181 petition are likely unavailable to Google LLC and its privies for future PTAB challenges against this patent. For other potential defendants not in privity with Google LLC, all prior-art grounds remain available for challenging the patent in a new IPR or litigation.

As for pattern signals, only one IPR has been filed against this patent by Google LLC, which resulted in a discretionary denial. This does not indicate a pattern of aggressive PTAB appeals by the patent owner, nor does it immediately signal a consistent defensive aggregation effort, although Google LLC itself can act defensively. The discretionary denial suggests the PTAB is managing its docket or applying evolving discretionary institution standards.

Recommended next steps

Since IPR2026-00181 was denied institution, there is no Final Written Decision to link to for claim invalidation. No active proceedings are currently pending that would lead to trial-stage milestones. The absence of a merits-based PTAB decision means the patent's claims are entirely untested at the PTAB.

Inventors

• Fatih M. Ozluturk

Original assignee

Clear Imaging Research LLC is the original assignee named on the issued patent US9860450B2. It is an intellectual property company that focuses on patent monetization. It is currently operating and has an active litigation record.

Assignment timeline

• 2017-07-25 (executed) / recorded 2017-07-25 — Reel 039403/0338
  • Conveyance: Assignment of Assignor's Interest
  • Assignor: Ozluturk, Fatih M.
  • Assignee: Clear Imaging Research LLC
  • Correspondent: R. Michael J. Walsh, 600 S. Orlando Ave. Suite 211, Maitland, FL, 32751.
  • Context: Inventor assigned rights to the original assignee.
• 2021-04-19 (executed) / recorded 2021-04-19 — Reel 056492/0533
  • Conveyance: Assignment of Assignor's Interest
  • Assignor: Clear Imaging, LLC; Ozluturk & Co, LLC; Ozluturk, Fatih
  • Assignee: Clear Imaging Research, LLC
  • Correspondent: R. Michael J. Walsh, 600 S. Orlando Ave. Suite 211, Maitland, FL, 32751. (This correspondent recurs in this chain.)
  • Context: Internal reorganization / assignment from related entities and inventor to the patent-owning entity.

Timeline diagram

timeline
    title Ownership of US 9860450
    2017-02-13 : Filed
    2017-07-25 : Inventor assigned to Clear Imaging Research LLC
    2018-01-02 : Issued
    2021-04-19 : Assigned to Clear Imaging Research LLC (reorg)

NPE / troll-pattern signals

1. Shell-entity transfer — Not present. The patent was initially assigned from the inventor to Clear Imaging Research LLC, which appears to be a patent holding and assertion entity. There isn't a clear transfer from an operating company to a shell.
2. Known asserter in the chain — Not present. Clear Imaging Research LLC is an asserting entity, but it is not explicitly listed among the provided common NPEs (Acacia Research Corp, Marathon Patent Group, Intellectual Ventures, IPNav, Wi-LAN, Mosaid / Conversant, Vringo, Pendrell, Innovatio IP Ventures, MPHJ Technology, Lumen View Technology, Round Rock Research, Document Generation Corp, Erich Spangenberg entities).
3. Repeat correspondent across the chain — Present. R. Michael J. Walsh of 600 S. Orlando Ave. Suite 211, Maitland, FL, 32751, is listed as the correspondent on both recorded assignments (Reel 039403/0338 and Reel 056492/0533).
4. Cascading transfers — Not present. There are only two recorded assignments, both originating from the inventor or inventor-related entities to the assignee, Clear Imaging Research LLC.
5. Pre-litigation transfer — Unclear. The abstract indicates there is litigation associated with this patent family, but the exact filing dates of the first infringement suits specifically naming US9860450B2 are not available from the provided data to compare against the assignment dates.
6. Bankruptcy fire-sale — Not present. No evidence of the original assignee filing for bankruptcy.
7. Privateering — Not present. No information suggesting an operating company transferred the patent to an NPE to assert on its behalf.
8. Defensive aggregator (anti-NPE) — Not present. The chain does not terminate at any known defensive aggregators.

Verdict

NPE — moderate confidence. While Clear Imaging Research LLC is itself an asserting entity, it is not explicitly on the provided list of "known asserters." However, the patent was assigned from the inventor to Clear Imaging Research LLC, which appears to be a patent holding and assertion entity with an active litigation record, and the recurrence of the same correspondent across the assignments strongly suggests a consistent legal strategy for managing patent assets, which is a common characteristic of NPEs. This is further supported by the current assignee's primary line of business being patent monetization.

To identify the most relevant prior art for US patent 9860450, I will first examine the "Cited By" section of the Google Patents record for US9860450B2, and then look at the "Prior art keywords" and "Prior art date".

From the "Cited By" section on Google Patents, the patent US9860450B2 cites the following patent:

• US11490015B2 (Cited by examiner)

In addition, the "Related Parent Applications" and "Applications Claiming Priority" sections indicate a family of patents, including:

• US8331723B2
• US8154607B2
• US8922663B2
• US9013587B2
• US9338356B2
• US9800787B2

It is important to note that these are generally continuations or divisions of the same underlying invention, sharing a priority date, and thus typically do not serve as prior art against US9860450B2 in the sense of 35 U.S.C. § 102 anticipation, as they would have the same priority date. However, they provide context for the evolution of the invention. The relevant prior art for anticipation would typically come from the "Cited By (1)" section or from earlier, distinct patents or publications.

Let's examine the directly cited patent first, and then the "Prior art keywords" for any other potential leads if the direct citation isn't sufficient.

Analysis of Directly Cited Patent (US11490015B2):

• Full Citation: US11490015B2, "Method and apparatus for capturing digital video", issued November 1, 2022, to Clear Imaging Research, LLC.
• Publication/Filing Date: Priority date: 2004-03-25. Publication date: 2022-11-01.
• Brief Description: This patent describes a method and apparatus for capturing digital video, including detecting motion information during capture, determining shift values, modifying images based on these shifts, and combining them. This description is very similar to the abstract of US9860450B2.
• Potential Anticipation: US11490015B2 shares the same priority date (2004-03-25) as US9860450B2. Patents sharing the same priority date cannot anticipate each other under 35 U.S.C. § 102. It is common for related patents within the same family to cite each other, but this does not make them prior art in the anticipatory sense.

Since the direct citation is from the same patent family and does not act as anticipatory prior art, I will now consider the broader context of prior art as typically found in the "References Cited" section of a patent and guided by the provided "Prior art keywords" and "Prior art date." The Google Patents page for US9860450B2 shows a "Prior art date" of 2004-03-25, which corresponds to the priority date of the patent itself. To identify anticipatory prior art, we need to look for references prior to this date.

The Google Patents entry also lists "Families Citing this family (75)", which are patents that cite US9860450's family. This is distinct from prior art cited by US9860450. To find prior art that US9860450 itself cited, I need to look at the "References Cited" section, which is typically found within the full patent document. Given that the provided text is the full patent text, I will search within it for "References Cited" or similar sections.

Upon reviewing the provided full patent text for US9860450B2, there is no explicit "References Cited" section listing prior art patents or publications in the standard USPTO format (e.g., U.S. Patent Documents, Foreign Patent Documents, Other Publications). However, the background section implicitly refers to existing techniques and devices.

The patent mentions:

• "Commonly used post-processing techniques for digitally correcting blurred images typically involve techniques that seek to increase the sharpness or contrast of the image."
• "In the prior art, electro-mechanical devices for correcting image blur due to camera motion are built into some high quality lenses, variously called 'image stabilization', 'vibration reduction', or similar names by camera/lens manufacturers."

Without an explicit "References Cited" list in the provided document, identifying specific prior art patents or publications that were formally cited during the prosecution of US9860450B2 and could potentially anticipate its claims under 35 U.S.C. § 102 is not possible from the given text alone. The "Prior art keywords" (images, sequence, motion, image, imaging device) are broad terms describing the field, not specific prior art documents.

Therefore, based on the provided patent text, I cannot confidently list specific prior art references that were cited against this patent during its examination and that would serve to anticipate its claims under 35 U.S.C. § 102. The "Applications Claiming Priority" are related applications, not typically anticipatory prior art. The "Cited By" patents are those that cited this patent, not vice-versa, with the exception of US11490015B2 which is also a member of the same family and thus shares the same priority date.

To truly identify the most relevant prior art, a direct examination of the patent's official file wrapper on the USPTO website would be necessary, specifically looking at the Information Disclosure Statements (IDS) and office actions. However, the current instruction is to rely only on the provided full patent text and current search results. Since the provided patent text does not contain a "References Cited" section in the typical format, I cannot fulfill the request to list specific cited prior art and analyze its potential anticipation.

Therefore, I must state that I do not know the most relevant cited prior art for US patent 9860450 with high confidence based solely on the provided information, as the full list of prior art references that were formally considered during its prosecution is not present in the given text.

The patent US9860450B2 claims methods and an apparatus for correcting camera shake in digital video. This involves capturing a video, using motion sensors to detect and record device motion during capture, determining vertical and horizontal shift values based on this motion information, modifying the images based on these shifts, combining the modified images into a final video, and storing it. Claims 28 and 29 additionally incorporate video compression techniques.

To analyze the obviousness under 35 U.S.C. § 103, we need to consider if a person having ordinary skill in the art (PHOSITA) would have been motivated to combine prior art references to arrive at the claimed invention, with a reasonable expectation of success. Since I do not have access to a database of prior art specifically cited against US9860450B2 during its prosecution or a general patent prior art search tool, I cannot definitively identify specific prior art combinations.

However, based on the general description of the invention and common knowledge in the field of image processing and digital video stabilization prior to the patent's priority date of March 25, 2004, I can outline potential areas of prior art and likely motivations for combination.

General Areas of Prior Art (pre-March 25, 2004):

• Image Stabilization Techniques:
  • Optical Image Stabilization (OIS): Electro-mechanical devices moving lens elements to compensate for camera movement. The patent itself mentions these as prior art, noting their cost and impact on lens characteristics.
  • Digital Image Stabilization (DIS): Techniques involving analyzing motion within image sequences and computationally shifting or warping frames to counteract shake. This often involves motion estimation algorithms.
• Motion Sensing Technology:
  • Accelerometers and Gyroscopes: Sensors capable of detecting and recording motion (linear and angular acceleration/velocity) were available and increasingly integrated into consumer electronics by the early 2000s.
• Video Processing and Compression:
  • Techniques for capturing, processing, and compressing digital video were well-established, including methods for frame alignment, noise reduction, and various forms of video compression (e.g., MPEG standards).
• Image Deconvolution/Deblurring:
  • Algorithms for deblurring images by estimating a point spread function (PSF) or transfer function (e.g., using blind deconvolution or explicit motion data) were known in academic and specialized imaging fields.

Potential Obviousness Argument (Hypothetical):

A PHOSITA in March 2004, working on improving digital video quality and counteracting camera shake, would likely have been aware of:

1. Digital image stabilization methods that analyze pixel-level motion within a video stream to align frames.
2. Hardware-based motion sensors (accelerometers/gyroscopes) that could provide precise external measurements of camera movement.
3. The benefits of combining information from different sources for more robust results in signal processing.

Hypothetical Combination of Prior Art References:

Let's assume the existence of hypothetical prior art references at the time of the invention:

• Prior Art A (Digital Video Stabilization): Discloses a method and apparatus for stabilizing digital video by analyzing inter-frame motion vectors to computationally shift frames and combine them into a stabilized video. This reference, however, relies solely on image content analysis for motion estimation.
• Prior Art B (Motion Sensor Integration): Discloses the integration of accelerometers or gyroscopes into portable electronic devices (e.g., digital cameras) to detect device orientation or movement for various applications (e.g., screen rotation, rudimentary shake detection for single photos). This reference, however, does not explicitly detail using sensor data for pixel-level video stabilization.
• Prior Art C (Image Deblurring/Deconvolution Principles): Discloses the mathematical principles of using a known or estimated motion transfer function to deconvolve a blurred image and recover a sharper original. This reference might demonstrate this for single still images or in contexts not directly related to real-time video stabilization using motion sensors.

Motivation for Combination and Explanation of Obviousness:

A PHOSITA, seeing the limitations of solely relying on image-based motion estimation (e.g., difficulty with uniform motion, featureless regions, or fast/complex movements) (from Prior Art A), would be motivated to seek more robust and accurate motion information. Prior Art B would teach the availability and integration of motion sensors into imaging devices. It would be an obvious design choice for the PHOSITA to combine the direct motion measurements from the sensors (Prior Art B) with the digital video stabilization techniques (Prior Art A).

The motivation would be to improve the accuracy and robustness of video stabilization. The motion sensor data provides an objective, direct measurement of camera movement, which could be used to:

• Corroborate or refine image-based motion estimation, particularly in challenging scenarios.
• Directly calculate the required shift values (vertical and horizontal) for each frame, as taught by US9860450B2, rather than relying solely on computationally intensive and potentially less accurate image analysis.
• Provide a feed-forward mechanism for stabilization, predicting motion rather than reacting to it solely from pixel changes.

Prior Art C (deblurring principles) would further teach the underlying mathematical basis for using a "transfer function" representing motion to "undo" blur. While US9860450B2 describes deriving vertical and horizontal shift values rather than a full deconvolution filter, the concept of using measured motion to inform image correction for blur reduction is conceptually linked. A PHOSITA would readily understand that the motion information from the sensors could be translated into precise shift values to align video frames, effectively reducing blur caused by camera shake.

Regarding the claims with video compression (Claims 28 and 29), applying video compression to a final, corrected video is a standard practice in digital video processing to reduce file size and facilitate storage/transmission. A PHOSITA would consider it an obvious implementation choice to apply known video compression techniques to the output of any video processing pipeline, including one for stabilization. There would be no inventive step in adding a video compression step to an already stabilized video.

Therefore, the combination of these hypothetical prior art references, driven by the motivation to improve the accuracy and reliability of digital video stabilization, would likely render the claims of US9860450B2 obvious to a PHOSITA.

US patent 9860450 has a complex prosecution history, being part of a large family of applications. Based on the provided patent text and general patent law, the following details can be provided:

Patent Term Adjustments (PTA) and Patent Term Extensions (PTE):
Specific details regarding the amount of Patent Term Adjustment (PTA) applied to US9860450 are not explicitly stated in the provided patent text or the search results. PTA is typically granted to compensate for certain delays by the United States Patent and Trademark Office (USPTO) during the prosecution of a patent application. The Google Patents record indicates an "Anticipated expiration" of 2025-03-24, while the base 20-year term from its earliest priority date (March 25, 2004) would be March 25, 2024. This suggests a PTA of approximately one year was granted.

Patent Term Extensions (PTE) are distinct from PTA and are typically granted under 35 U.S.C. § 156 for patents covering products, such as pharmaceuticals, that require premarket regulatory review by agencies like the FDA. There is no indication from the patent's title, abstract, or description that US9860450 relates to a product subject to regulatory review, making a PTE highly unlikely.

Continuation and Divisional Applications:
US patent 9860450 is explicitly identified as a continuation application. Its "CROSS REFERENCE TO RELATED APPLICATIONS" section details a chain of parent applications:

• Continuation of Ser. No. 15/149,481, filed May 9, 2016.
• Which is a continuation of U.S. patent application Ser. No. 14/690,818, filed on Apr. 20, 2015 (issued as U.S. Pat. No. 9,338,356).
• Which is a continuation of U.S. patent application Ser. No. 14/532,654, filed on Nov. 4, 2014 (issued as U.S. Pat. No. 9,013,587).
• Which is a continuation of U.S. patent application Ser. No. 13/442,370, filed on Apr. 9, 2012 (issued as U.S. Pat. No. 8,922,663).
• Which is a continuation of U.S. patent application Ser. No. 12/274,032, filed on Nov. 19, 2008 (issued as U.S. Pat. No. 8,154,607).
• Which is a continuation of U.S. patent application Ser. No. 11/089,081, filed on Mar. 24, 2005 (issued as U.S. Pat. No. 8,331,723).
• This chain ultimately claims the benefit of U.S. Provisional Application Ser. No. 60/556,230, filed on Mar. 25, 2004.

The provided patent text and Google Patents information do not explicitly state if any divisional applications directly stem from US9860450 itself. However, the "Family Applications" list on Google Patents shows a large number of related applications that share the same priority date, which could include divisional applications from earlier patents in the chain, or further continuations.

Related Family Members:
The "Family Applications" section on Google Patents lists numerous related patents and applications sharing the same priority date of 2004-03-25. These include both granted patents and pending applications, such as:

• US15/858,339 (US10389944B2)
• US16/544,426 (US10880483B2)
• US17/066,882 (US11165961B2)
• US17/514,486 (US11627391B2)
• US17/692,988 (US11457149B2)
• US17/693,052 (US11490015B2)
• US17/952,934 (US11595583B2)
• US17/977,739 (US11627254B2)
• US18/114,784 (US11800228B2)
• US18/132,698 (US11812148B2)
• US18/132,673 (US11924551B2)
• US18/438,928 (US12132992B2)
• US18/929,275 (US20250056121A1, currently pending)

Projected Expiration Date:
The general patent term for utility patents filed on or after June 8, 1995, is 20 years from the earliest effective filing date, subject to any Patent Term Adjustment (PTA) or Patent Term Extension (PTE). US9860450 claims priority to U.S. Provisional Application Ser. No. 60/556,230, filed on March 25, 2004. Therefore, the base 20-year patent term would end on March 25, 2024.

However, the Google Patents entry for US9860450B2 lists the "Anticipated expiration" as 2025-03-24. This discrepancy indicates that approximately one year of Patent Term Adjustment (PTA) was likely granted, extending the patent's effective term beyond the standard 20 years from the priority date.

Furthermore, the Google Patents record states the legal status as "Expired - Fee Related". Given that the current date is June 1, 2026, and the anticipated expiration date was March 24, 2025, the patent has indeed expired. The "Fee Related" aspect suggests that the expiration occurred due to the non-payment of maintenance fees rather than reaching the full term based on the 20-year clock plus PTA.

The USPTO website provides tools for patent public search. I can confirm that patent number 9860450 is a valid US patent number format (6-8 digits). I will proceed with generating the defensive disclosure document based on the provided patent text and the confirmed validity of the patent number.

Defensive Disclosure for US Patent 9860450: Method and apparatus to correct digital video to counteract effect of camera shake

This document provides derivative variations of the core claims of US patent 9860450, aimed at establishing prior art for future incremental improvements in the field of digital video stabilization. The technical descriptions are intended to be sufficiently enabling for a person having ordinary skill in the art (PHOSITA) to reproduce or implement these variations.

Core Claims Addressed:

• Claim 1 (Method): A method for use in an imaging device comprising an image sensor, a processor, a memory, and one or more motion sensors, the method comprising: capturing a sequence of images (video), detecting motion information, determining vertical/horizontal shift values, modifying images based on shifts, combining to obtain a final video, and storing the final video.
• Claim 14 (Imaging Device): An imaging device comprising an image sensor, one or more motion sensors, a processor configured to perform the steps of Claim 1, and a memory configured to store the final video.
• Claim 28 (Method with Compression): Similar to Claim 1, but explicitly includes applying a video compression technique to obtain a final video.
• Claim 29 (Imaging Device with Compression): Similar to Claim 14, but explicitly includes a processor configured to apply a video compression technique to obtain a final video.

---

Derivative Variations

For Claim 1: Method for Video Stabilization with Motion Sensors

1. Material & Component Substitution: Utilizing Piezoelectric Motion Sensors and Ferroelectric Memory

• Enabling Description: The method of Claim 1 is implemented using miniature piezoelectric accelerometers and gyroscopes for detecting motion information. These sensors, based on the piezoelectric effect, generate electrical charge in response to mechanical stress, offering high sensitivity and low power consumption suitable for compact imaging devices. The captured sequence of images and detected motion information are stored in ferroelectric random-access memory (FeRAM), which provides non-volatile data storage with high write endurance and fast read/write speeds, enabling rapid buffering and processing of video frames and motion data. The processor's shift value determination and image modification operations are optimized for the specific data access patterns of FeRAM.

flowchart TD
    A[Capture Image Sequence (Image Sensor)] --> B{Detect Motion (Piezoelectric Sensors)};
    B --> C[Store Raw Data (FeRAM)];
    C --> D[Processor: Determine Shifts];
    D --> E[Processor: Modify Images (FeRAM access)];
    E --> F[Combine Modified Images];
    F --> G[Store Final Video (FeRAM)];

2. Operational Parameter Expansion: Ultra-High-Resolution, High-Frame-Rate Video Stabilization for Scientific Imaging

• Enabling Description: The method is adapted for stabilizing video streams captured at resolutions up to 8K UHD (7680x4320 pixels) and frame rates exceeding 240 frames per second (fps). Motion sensors, such as high-frequency micro-electro-mechanical systems (MEMS) gyroscopes and accelerometers, operate at sampling rates of 10 kHz or higher to accurately capture rapid, subtle movements. The processor, consisting of a parallel processing architecture (e.g., GPU clusters or multi-core DSPs), utilizes a pipelined algorithm for real-time determination of vertical and horizontal shift values, ensuring minimal latency. Image modification involves sub-pixel interpolation (e.g., bicubic or Lanczos resampling) to maintain image fidelity at extreme magnifications common in scientific applications, and the combining step aggregates these high-resolution, stabilized frames into an 8K/240fps final video.

graph TD
    A[Capture 8K@240fps Video] --> B{High-Freq MEMS Sensors};
    B --> C[Raw Motion & Image Data Stream];
    C --> D[Pipelined Parallel Processor];
    D -- Sub-pixel Interpolation --> E[Modified 8K Frames];
    E --> F[Combine & Output Final 8K@240fps Video];

3. Cross-Domain Application: Surgical Endoscopy Stabilization

• Enabling Description: The method is applied to stabilize video feeds from surgical endoscopes during minimally invasive procedures. The imaging device is a flexible endoscope with a miniature image sensor and embedded motion sensors (e.g., fiber optic gyroscopes or strain gauges near the distal tip) that detect minute movements caused by surgeon tremor or patient respiration. The motion information is used by a dedicated processor unit to determine precise vertical and horizontal shifts for each video frame. Image modification aligns the live endoscopic view, correcting for shake and enabling a steadier visual field for the surgeon. The final stabilized video can be displayed on a surgical monitor or recorded for post-operative analysis and training.

sequenceDiagram
    participant E as Endoscope (Sensor + Imager)
    participant P as Processor Unit
    participant M as Surgical Monitor

    E ->> P: Transmit Live Video Frames (Sequence)
    E ->> P: Transmit Motion Data (Synchronous)
    P ->> P: Determine Shift Values (V/H)
    P ->> P: Modify Video Frames (Align)
    P ->> P: Combine/Render Stabilized Video
    P ->> M: Display Stabilized Video

4. Integration with Emerging Tech: AI-Driven Predictive Stabilization with IoT Sensor Network

• Enabling Description: The method incorporates an AI-driven predictive model (e.g., a Recurrent Neural Network or Transformer-based architecture) for determining vertical and horizontal shift values. Instead of solely reacting to current motion, the AI model processes historical motion sensor data, real-time data from an array of IoT-enabled motion sensors (e.g., strategically placed accelerometers on the camera body, lens, and even the user's hand/gimbal), and contextual information (e.g., scene content analysis, ambient vibration patterns). This allows for proactive estimation of future camera movements and pre-computation of shifts, significantly reducing stabilization latency and artifacts. The IoT sensor network transmits data via low-power wireless protocols (e.g., Bluetooth Low Energy or Zigbee) to a central processing unit for aggregate motion analysis.

graph TD
    subgraph IoT Sensor Network
        S1[Hand Sensor] -- BLE/Zigbee --> R(Data Aggregator/Router)
        S2[Camera Body Sensor] -- BLE/Zigbee --> R
        S3[Lens Sensor] -- BLE/Zigbee --> R
    end
    R --> D[Raw Motion Data Stream];
    D -- Historical Data --> AI[AI Predictive Model (RNN/Transformer)];
    AI -- Predicted Shifts --> P(Processor Unit);
    P -- Real-time Video Frames --> P;
    P -- Proactive Correction --> F[Final Stabilized Video];

5. The "Inverse" or Failure Mode: Safe-Mode Low-Power Stabilization

• Enabling Description: The method includes a safe-mode operation for low-power or limited-functionality scenarios. When battery levels fall below a threshold or system resources are constrained, the imaging device automatically switches to a low-power stabilization mode. In this mode, motion detection is simplified, perhaps by reducing the sampling rate of motion sensors or activating only a subset of sensors. Shift value determination uses a coarser approximation algorithm (e.g., block-matching motion estimation instead of full pixel-level shifts), prioritizing computational efficiency over maximal image quality. The image modification might involve simpler averaging of frames or only correcting dominant translational motion, while rotational or perspective distortions are ignored. The final video is stored at a reduced resolution or bitrate, ensuring continued recording and basic stabilization until resources are restored or capture terminates.

stateDiagram-v2
    state Normal_Operation {
        [*] --> Capture_High_Res : Start
        Capture_High_Res --> Process_Full_Stabilization : Motion Detected
        Process_Full_Stabilization --> Store_High_Quality_Video : Done
    }
    state Low_Power_Mode {
        [*] --> Capture_Low_Res : Start
        Capture_Low_Res --> Process_Basic_Stabilization : Motion Detected
        Process_Basic_Stabilization --> Store_Reduced_Quality_Video : Done
    }
    Normal_Operation --> Low_Power_Mode : Low Battery / Resource Constraint
    Low_Power_Mode --> Normal_Operation : Power Restored / Resources Available

---

For Claim 14: Imaging Device for Video Stabilization with Motion Sensors

1. Material & Component Substitution: Modular Carbon Fiber Body with MEMS IMUs and Custom ASIC Processor

• Enabling Description: The imaging device comprises a modular housing constructed from carbon fiber composites, offering superior strength-to-weight ratio and vibration dampening compared to traditional metal or plastic bodies. The one or more motion sensors are integrated as a redundant array of miniaturized MEMS Inertial Measurement Units (IMUs), strategically placed within the camera body and lens mount for comprehensive 6-degrees-of-freedom (6-DoF) motion detection. The processor is a custom-designed Application-Specific Integrated Circuit (ASIC) optimized for real-time execution of the shift determination and image modification algorithms. This ASIC includes dedicated hardware accelerators for image resampling and blending, ensuring ultra-low power consumption and high processing throughput. The memory consists of stacked 3D NAND flash for high-density video storage, coupled with high-bandwidth GDDR6 for frame buffers.

classDiagram
    class ImagingDevice {
        +CarbonFiberHousing
        +ImageSensor[]
        +MEMSIMU[] MotionSensors
        +CustomASICProcessor
        +3DNANDMemory
        +GDDR6FrameBuffer
    }
    ImagingDevice --> ImageSensor
    ImagingDevice --> MEMSIMU
    ImagingDevice --> CustomASICProcessor
    ImagingDevice --> 3DNANDMemory
    ImagingDevice --> GDDR6FrameBuffer

2. Operational Parameter Expansion: Submersible, Pressure-Tolerant Imaging Device for Deep-Sea Exploration

• Enabling Description: The imaging device is designed for deep-sea exploration, encased in a titanium alloy pressure housing capable of withstanding pressures up to 100 MPa (corresponding to ~10,000 meters depth). The image sensor is a back-illuminated scientific CMOS (sCMOS) array optimized for low-light conditions prevalent in deep-sea environments. Motion sensors are hardened piezoelectric transducers and fiber-optic gyroscopes, hermetically sealed to prevent water ingress and maintain functionality under extreme pressure and low temperatures. The processor, also housed in the pressure vessel, employs a fault-tolerant, radiation-hardened architecture to ensure reliable operation. Shift determination and image modification algorithms are pre-calibrated for potential optical distortions unique to underwater imaging, such as chromatic aberration and refraction effects from viewing ports.

graph LR
    A[TitaniumPressureHousing] --> B[Back-Illuminated_sCMOS_ImageSensor];
    A --> C[Hardened_Piezoelectric_MotionSensors];
    A --> D[FiberOpticGyroscopes];
    A --> E[Fault-Tolerant_Radiation-Hardened_Processor];
    E -- Controls --> B;
    E -- Processes --> C;
    E -- Processes --> D;
    E -- Stores --> F[Memory_for_Final_Video];
    F -- Encased_within --> A;

3. Cross-Domain Application: Drone-Mounted Aerial Survey System

• Enabling Description: The imaging device is specifically configured for integration with unmanned aerial vehicles (UAVs) for aerial survey and mapping. It features a lightweight, gimbal-stabilized housing with an integrated high-resolution global shutter image sensor to minimize rolling shutter artifacts. The motion sensors comprise a high-precision IMU (including gyroscopes, accelerometers, and magnetometers) rigidly coupled to the image sensor, capable of detecting subtle drone vibrations and platform movements. The processor is a compact, low-power system-on-chip (SoC) with dedicated hardware acceleration for real-time video stabilization and geospatial metadata embedding. The device determines precise shift values to correct for residual gimbal imperfections and wind-induced drone movements, ensuring georeferencing accuracy of the captured video for photogrammetry applications.

flowchart TD
    A[UAV Platform] --> B(Gimbal-Stabilized Mount);
    B --> C[Drone-Mounted_ImagingDevice];
    C -- Contains --> D[Global_Shutter_ImageSensor];
    C -- Contains --> E[High-Precision_IMU_MotionSensors];
    C -- Contains --> F[Compact_SoC_Processor];
    F -- Determines Shifts --> D;
    F -- Modifies/Combines --> G[Memory_for_Final_Video];
    G -- Outputs --> H[Georeferenced_Stabilized_Video];

4. Integration with Emerging Tech: Edge AI-Enhanced Smart Surveillance Camera

• Enabling Description: The imaging device is a smart surveillance camera featuring an embedded Edge AI processor (e.g., a neural processing unit or dedicated AI accelerator). The motion sensors are high-accuracy MEMS IMUs. The Edge AI is trained to distinguish between intentional camera movements (e.g., pan/tilt by operator) and unintentional shake, dynamically adjusting stabilization parameters. Furthermore, it can perform semantic segmentation of the video stream to identify regions of interest (e.g., human subjects, vehicles). The processor prioritizes stabilization within these regions, potentially allowing a slight blur in static background areas if it optimizes computational resources or improves target tracking. Blockchain technology is integrated for secure, verifiable logging of camera operational parameters, motion events, and stabilization applied, ensuring chain of custody and tamper-proofing for forensic video evidence.

graph TD
    A[ImageSensor] --> B[VideoStream];
    M[MEMS_IMUs] --> C[MotionData];
    B & C --> E[Edge_AI_Processor];
    E -- Identifies Intentional/Unintentional Motion --> F(Stabilization Module);
    E -- Semantic Segmentation --> F;
    F --> G[Modified_Images];
    G --> H[Combined_Final_Video];
    H --> I[Memory];
    I --> J[Blockchain_Logger];
    E -- Logs Parameters & Events --> J;

5. The "Inverse" or Failure Mode: Redundant, Fail-Safe Micro-Camera System

• Enabling Description: The imaging device is a redundant micro-camera system for critical applications where video capture must continue despite component failure. It comprises an array of three identical, small-form-factor imaging modules, each with its own image sensor, motion sensors, and local processing unit. If one module detects an internal fault (e.g., motion sensor failure, processor error, or memory corruption), it signals a master controller. The master controller then dynamically reconfigures the system to use data from the remaining functional modules. In a single module failure, the system falls back to a 2-camera mode, potentially sacrificing some spatial resolution or stabilization robustness but maintaining core functionality. If two modules fail, the system operates in a minimal single-camera mode with only basic stabilization or raw capture, signaling a critical alert. This fail-safe design ensures graceful degradation rather than catastrophic failure.

stateDiagram-v2
    state Normal_Operation {
        [*] --> All_Modules_Active : System Start
        All_Modules_Active --> Full_Stabilization : No Faults
    }
    state Degraded_Mode_1 {
        [*] --> Module_1_Failed : Fault Detected
        Module_1_Failed --> Two_Modules_Active : Reconfigure
        Two_Modules_Active --> Reduced_Stabilization : Continue
    }
    state Degraded_Mode_2 {
        [*] --> Module_2_Failed : Fault Detected (from Degraded_Mode_1)
        Module_2_Failed --> One_Module_Active : Reconfigure
        One_Module_Active --> Basic_Stabilization_or_Raw : Continue
    }

    Normal_Operation --> Degraded_Mode_1 : Module 1 Failure
    Degraded_Mode_1 --> Degraded_Mode_2 : Module 2 Failure

---

For Claim 28: Method for Video Stabilization with Compression

1. Material & Component Substitution: Hybrid Hardware/Software Codec with Quantum-Dot Image Sensor

• Enabling Description: The method of Claim 28 utilizes a hybrid hardware/software video compression technique. After modifying and combining the images, the resulting video stream is fed into a dedicated hardware video encoder (e.g., an H.265/HEVC ASIC) for high-speed, power-efficient compression. Concurrently, a software-based post-processing engine (running on a general-purpose processor) applies perceptual quality enhancements or adaptive bitrate optimization not typically handled by basic hardware codecs. The initial image capture is performed by an image sensor utilizing quantum-dot technology, which offers superior color reproduction and wider dynamic range compared to traditional CMOS sensors, providing higher quality input for both stabilization and subsequent compression.

flowchart TD
    A[Capture Image Sequence (Quantum-Dot Sensor)] --> B[Detect Motion];
    B --> C[Determine Shifts];
    C --> D[Modify Images];
    D --> E[Combine Modified Images];
    E --> F{Hybrid Video Compression};
    F --> F1[Hardware Encoder (H.265 ASIC)];
    F --> F2[Software Post-Processing (Adaptive Bitrate)];
    F1 & F2 --> G[Final Compressed Video];
    G --> H[Store Final Video];

2. Operational Parameter Expansion: Real-time, Ultra-Low Latency Video Compression for Remote-Controlled Systems

• Enabling Description: The method is optimized for real-time video stabilization and compression with ultra-low latency, crucial for remote-controlled robotics or telepresence applications where feedback delay is critical. This involves capturing images at very high frame rates (e.g., 500+ fps), performing shift determination and image modification on a frame-by-frame basis with minimal buffering. The video compression technique used is a custom wavelet-based codec or a highly optimized low-latency H.264/H.265 profile, specifically designed to minimize end-to-end delay rather than solely focusing on file size. This may involve predictive coding across minimal frames or reducing group-of-pictures (GOP) sizes. Motion sensor data is timestamped with nanosecond precision and processed concurrently with image data to ensure synchronous shifts.

sequenceDiagram
    participant I as ImageSensor
    participant M as MotionSensors
    participant P as Processor (Low-Latency)
    participant C as Custom_Codec
    participant N as Network_Stream

    I ->> P: Frame_N (High FPS)
    M ->> P: MotionData_N (Synchronous)
    P ->> P: Determine/Modify(Frame_N, MotionData_N)
    P ->> C: Modified_Frame_N
    C ->> N: Compressed_Frame_N (Ultra-low latency)
    loop Continuous Operation
        I ->> P: Frame_N+1
        M ->> P: MotionData_N+1
        P ->> P: Determine/Modify(Frame_N+1, MotionData_N+1)
        P ->> C: Modified_Frame_N+1
        C ->> N: Compressed_Frame_N+1
    end

3. Cross-Domain Application: Live Sports Broadcasting Stabilization and Archival

• Enabling Description: The method is deployed in live sports broadcasting to stabilize dynamic camera footage (e.g., from handheld cameras, steadicams, or wired cams) and prepare it for real-time transmission and archival. The imaging device detects rapid, complex camera movements characteristic of sports coverage. Shift values are determined to keep athletes or the field of play centered and stable. The video compression technique applied is a professional broadcast-grade codec (e.g., MPEG-2 Transport Stream for live, JPEG 2000 or ProRes for archival), allowing for high-quality, low-artifact output suitable for television networks, while simultaneously enabling flexible bitrates for different distribution channels (e.g., 4K HDR for main broadcast, 1080p for streaming).

graph TD
    A[Live_Sports_Camera] --> B{Detect Motion};
    B --> C[Capture Video Stream];
    C & B --> D[Processor: Determine/Modify Shifts];
    D --> E[Combined_Stabilized_Video];
    E --> F{Broadcast-Grade_Compression};
    F -- Real-time --> G[Live_Broadcast_Feed];
    F -- High-Quality --> H[Archival_Storage];

4. Integration with Emerging Tech: Decentralized Video Stabilization with Blockchain for Content Integrity

• Enabling Description: The method extends to a decentralized network where multiple imaging devices (Claim 14 devices) collaboratively stabilize and compress video, with blockchain technology ensuring content integrity. Each imaging device captures video and motion data, performs initial shift determination, and applies a preliminary stabilization. Before final combination and compression, selected frames and associated metadata (motion vectors, stabilization parameters) are securely hashed and recorded on a blockchain. A network of distributed processors then performs the final image modification and combination, verifying the integrity of the input frames against the blockchain records. The final compressed video is also hashed and recorded, providing an immutable record of its origin and processing steps, valuable for journalistic integrity or legal evidence.

sequenceDiagram
    participant D1 as Device 1 (Imager+Sensors)
    participant D2 as Device 2 (Imager+Sensors)
    participant DP as Distributed Processors
    participant B as Blockchain Network

    D1 ->> DP: Send Raw Frames & Motion Data
    D2 ->> DP: Send Raw Frames & Motion Data
    DP ->> DP: Initial Shift & Stabilization (per device)
    DP ->> B: Hash Frames & Metadata (Record)
    B -->> DP: Verification Confirmation
    DP ->> DP: Final Combination & Compression
    DP ->> B: Hash Final Video (Record)
    B -->> DP: Final Video Integrity Confirmed
    DP ->> DP: Store Final Video

5. The "Inverse" or Failure Mode: Adaptive Compression for Bandwidth Constraint Degradation

• Enabling Description: The method incorporates an adaptive video compression technique that gracefully degrades quality under severe bandwidth constraints, a common failure mode in wireless video transmission. If the network uplink bandwidth drops below a critical threshold, the processor dynamically adjusts the compression parameters. This begins with increasing the quantization parameter (QP), reducing chroma subsampling, and then progressively lowering resolution or frame rate as conditions worsen. The stabilization algorithm may also adapt, reducing the complexity of shift determination or motion compensation to save processing cycles that can be reallocated to compression, ensuring a continuous (though lower quality) video stream is maintained rather than dropping frames entirely. This ensures a "minimum viable video" is always transmitted.

graph TD
    A[Detect Network Bandwidth] --> B{Bandwidth Threshold Exceeded?};
    B -- Yes --> C[Adaptive Compression Module];
    C --> C1[Increase QP];
    C --> C2[Reduce Chroma Subsampling];
    C --> C3[Lower Resolution/Frame Rate];
    C --> C4[Simplify Stabilization Algo];
    C1 & C2 & C3 & C4 --> D[Apply Compression];
    D --> E[Transmit Degraded Video];
    B -- No --> F[Standard Compression];
    F --> G[Transmit High-Quality Video];
    E & G --> H[Store Final Video];

---

For Claim 29: Imaging Device for Video Stabilization with Compression

1. Material & Component Substitution: Gallium Nitride (GaN) Power Electronics and Neural Network Processor

• Enabling Description: The imaging device integrates Gallium Nitride (GaN) power electronics for highly efficient power management, critical for compact, long-endurance devices requiring significant processing power for video stabilization and compression. The processor is a dedicated Neural Network Processor (NNP) or AI accelerator, specifically designed for executing deep learning models. This NNP handles both the determination of vertical and horizontal shift values (using a trained CNN for robust motion estimation) and aspects of the video compression pipeline (e.g., neural network-based image enhancement before encoding, or neural compression itself). The image sensor is a low-noise, high-sensitivity global shutter CMOS sensor. The memory includes both high-speed LPDDR5 for temporary frame storage and UFS 4.0 for high-throughput final video storage.

classDiagram
    class ImagingDevice {
        +GlobalShutterCMOS
        +MotionSensors
        +NNPProcessor
        +GaNPowerElectronics
        +LPDDR5Memory
        +UFS4_0Memory
    }
    ImagingDevice --> GlobalShutterCMOS
    ImagingDevice --> MotionSensors
    ImagingDevice --> NNPProcessor
    ImagingDevice --> GaNPowerElectronics
    ImagingDevice --> LPDDR5Memory
    ImagingDevice --> UFS4_0Memory

2. Operational Parameter Expansion: Cryogenic Imaging Device for Astronomical Observations

• Enabling Description: The imaging device is designed for use in cryogenic environments, such as space telescopes or specialized laboratory setups, operating at temperatures as low as 4 Kelvin. The image sensor is a specialized cryo-CMOS sensor, and motion sensors are superconducting quantum interference devices (SQUIDs) or other ultra-low temperature compatible accelerometers/gyroscopes, capable of detecting minute vibrations. The processor and memory components are radiation-hardened and selected for stable operation at cryogenic temperatures, potentially utilizing superconductive interconnects for extreme speed. The video compression technique is optimized for scientific data integrity, possibly employing lossless or near-lossless compression algorithms to preserve subtle astronomical features, and is performed by a dedicated cryo-compatible processing unit before transmission or storage.

graph TD
    A[CryogenicEnvironment] --> B(Cryo_ImagingDevice);
    B -- Contains --> C[Cryo-CMOS_ImageSensor];
    B -- Contains --> D[SQUID_MotionSensors];
    B -- Contains --> E[Radiation-Hardened_Cryo-Processor];
    E -- Processes --> C;
    E -- Processes --> D;
    E -- Applies --> F[Lossless/Near-Lossless_Compression];
    F --> G[Cryo-Compatible_Memory_for_Video];

3. Cross-Domain Application: Automated Agricultural Monitoring Drone

• Enabling Description: The imaging device is implemented as part of an autonomous agricultural monitoring drone system. The image sensor captures multispectral or hyperspectral video sequences of crops. Motion sensors (integrated IMU on the drone) detect precise movements from wind gusts or drone maneuvers. The processor determines shift values to stabilize the video, ensuring accurate spatial alignment of successive frames for vegetation index calculation. The video compression technique applied is optimized for efficient transmission of large datasets over wireless links in agricultural fields (e.g., using sparse coding or region-of-interest (ROI) compression to prioritize crop health indicators over background), sending compressed data to a ground station for analysis.

flowchart LR
    A[Autonomous_AgriDrone] --> B(Multispectral_ImageSensor);
    A --> C(Drone_IMU_MotionSensors);
    B & C --> D[Onboard_Processor_Unit];
    D --> E[Stabilization_Module];
    E --> F[ROI-Optimized_Video_Compression];
    F --> G[Wireless_Telemetry_Module];
    G --> H[Ground_Station_Analysis];

4. Integration with Emerging Tech: Real-time Digital Twin Generation with AI & VR/AR Output

• Enabling Description: The imaging device is a component of a system for generating real-time digital twins of physical environments, leveraging AI-driven stabilization and immersive output. The image sensor captures high-fidelity video, while high-precision motion sensors provide spatial tracking data. An onboard AI processor performs stabilization and concurrently generates 3D point clouds or mesh representations from the stabilized video. The processor is configured to combine these modified images and spatial data, applying a specialized video compression technique (e.g., a volumetric video codec or a codec optimized for 3D reconstruction data). The output is streamed for real-time visualization in a Virtual Reality (VR) or Augmented Reality (AR) headset, allowing users to interact with a stabilized, live digital twin of the environment.

sequenceDiagram
    participant I as ImageSensor
    participant M as MotionSensors
    participant P as AI_Processor (Onboard)
    participant V as VR/AR_Headset

    I ->> P: High-Fidelity_Video_Frames
    M ->> P: High-Precision_Motion_Data
    P ->> P: Stabilize_Video_Frames (AI-driven)
    P ->> P: Generate_3D_Point_Cloud (from stabilized video)
    P ->> P: Volumetric_Video_Compression
    P ->> V: Stream_Real-time_Digital_Twin_Data

5. The "Inverse" or Failure Mode: Forensic Event Recorder with Write-Once Read-Many (WORM) Memory

• Enabling Description: The imaging device acts as a forensic event recorder in high-risk environments (e.g., vehicle dashcam, industrial safety monitoring). Its primary "failure mode" consideration is data integrity against tampering or accidental deletion. The device uses a Write-Once Read-Many (WORM) memory for storing the final video, preventing any post-capture modification. If a major impact or system fault is detected by the motion sensors (e.g., an accelerometer reading exceeding a crash threshold), the processor immediately triggers a final, robust stabilization and compression cycle on the buffered video segment preceding and during the event. This specific event video is then quickly written to the WORM memory. The video compression technique employed is a fixed, high-quality, tamper-evident codec (e.g., a specific H.264 profile with embedded cryptographic hashes in metadata) to ensure the stored footage is admissible as evidence. Regular video capture continues with standard compression to a volatile buffer or a different storage medium.

stateDiagram-v2
    state Normal_Recording {
        [*] --> Buffer_Video_&_Motion : Start
        Buffer_Video_&_Motion --> Standard_Compression : Regular Ops
        Standard_Compression --> Volatile_Storage : Store
    }
    state Event_Recording {
        Event_Detected --> Trigger_Forensic_Capture : High G-Force / Fault
        Trigger_Forensic_Capture --> Final_Robust_Stabilization : Process Buffered Data
        Final_Robust_Stabilization --> Tamper_Evident_Compression : Apply Codec
        Tamper_Evident_Compression --> WORM_Memory_Storage : Store Permanently
    }
    Normal_Recording --> Event_Recording : Motion Sensor Alert

---

Combination Prior Art Scenarios

These scenarios describe how the concepts within US9860450 could be combined with existing open-source standards, thereby contributing to the prior art and rendering similar future developments obvious.

1. Integration with FFmpeg for General-Purpose Video Editing & Processing Workflows

• Scenario Description: An implementation wherein the methods described in Claim 1 and Claim 28 of US9860450 are provided as a set of open-source filter plugins or libraries for the FFmpeg multimedia framework. Specifically, motion information detected by the device's sensors (e.g., an IMU log file) is ingested alongside the video stream. A custom FFmpeg filter would then parse this motion data, calculate the vertical and horizontal shift values for each frame (as described in Claim 1), and apply these shifts using FFmpeg's existing video manipulation capabilities (e.g., setpts, tps, crop, pad filters, or custom pixel-level transformations). The modified images are combined, and the resulting stabilized video is then subjected to FFmpeg's vast array of video compression techniques (e.g., libx264 for H.264 or libvpx for VP9, as per Claim 28), allowing for a highly flexible and widely adaptable video stabilization and compression pipeline. This publicly available FFmpeg plugin, documented with enabling descriptions, renders any similar "ingest motion data for stabilization then compress" approach obvious for general video processing.

graph TD
    A[Raw Video (e.g., MP4)] --> B(FFmpeg Input);
    M[Motion Sensor Log (e.g., CSV)] --> B;
    B --> C{Custom FFmpeg Filter Plugin};
    C -- Parse Motion Data --> D[Calculate V/H Shifts];
    D -- Apply Shifts --> E[FFmpeg Video Manipulation (e.g., setpts, crop)];
    E --> F[Combined Stabilized Video Stream];
    F --> G{FFmpeg Encoder (e.g., libx264)};
    G --> H[Final Compressed Video File];

2. Real-time Stabilization for Robotics Platforms using Robot Operating System (ROS) and OpenCV

• Scenario Description: The principles of the imaging device (Claim 14) and its method (Claim 1) are implemented on a mobile robotics platform (e.g., a ground robot or drone) running the Robot Operating System (ROS). The imaging device's motion sensors publish their data (e.g., IMU messages) as standard ROS topics. The image sensor publishes raw video frames as another ROS topic. A ROS node, implemented using the OpenCV library for image processing, subscribes to both these topics. This node processes the incoming motion data to determine vertical and horizontal shift values and uses OpenCV functions (e.g., warpAffine or warpPerspective) to modify and combine the video frames in real-time, producing a stabilized video stream that is published as a new ROS topic. This stabilized stream can then be used by other ROS nodes for navigation, object detection, or transmitted for remote viewing. This open-source integration would make the combination of motion sensors and image processing for video stabilization on robotics platforms universally obvious.

graph LR
    subgraph Imaging Device
        I[Image Sensor] -- Publishes /camera/image_raw --> ROS_T(ROS Topic Bus)
        M[Motion Sensors (IMU)] -- Publishes /imu/data --> ROS_T
    end
    subgraph Stabilization Node (OpenCV)
        SN[ROS Node: Video Stabilizer] --> Sub_IR(Subscribes /camera/image_raw)
        SN --> Sub_IMU(Subscribes /imu/data)
        Sub_IMU -- Motion Data --> SN_P(Process Motion)
        Sub_IR -- Image Frame --> SN_P
        SN_P -- Calculate V/H Shifts --> SN_M(Modify Image w/ OpenCV)
        SN_M --> Pub_SV(Publishes /camera/image_stabilized)
    end
    ROS_T --> SN
    SN --> ROS_T

3. Blockchain-Verified Motion Data and Stabilization Metadata with WebM/VP9 Open Standard

• Scenario Description: The method of Claim 28 and apparatus of Claim 29 are extended to publicly record motion information and stabilization metadata on a public blockchain, specifically for video encoded using the open-source WebM container format with VP9 video codec. The imaging device detects motion, determines shifts, modifies images, and combines them. Before applying WebM/VP9 compression, the processor calculates cryptographic hashes of the raw motion sensor data (or derived transfer functions) and the stabilization parameters (e.g., the applied vertical and horizontal shifts for keyframes). These hashes, along with timestamps, are embedded within the WebM metadata (e.g., using Matroska elements or custom WebM tracks) and simultaneously submitted as transactions to a public blockchain (e.g., Ethereum or a specialized media blockchain). This provides an immutable, auditable record of the stabilization process and source motion data, ensuring transparency and verifiable content integrity for publicly shared videos, while utilizing a widely adopted open video standard.

flowchart TD
    A[Capture Video & Motion Data] --> B[Determine/Apply Shifts];
    B --> C[Stabilized Video Frames];
    C --> D{Hash Motion Data & Stabilization Metadata};
    D --> E[Embed Hashes in WebM Metadata];
    C & E --> F[WebM/VP9 Encoder];
    F --> G[Final WebM Video File];
    D --> H[Submit Hashes to Blockchain];
    G --> I[Store Final Video];