Obviousness

The patent US9860450B2 claims methods and an apparatus for correcting camera shake in digital video. This involves capturing a video, using motion sensors to detect and record device motion during capture, determining vertical and horizontal shift values based on this motion information, modifying the images based on these shifts, combining the modified images into a final video, and storing it. Claims 28 and 29 additionally incorporate video compression techniques.

To analyze the obviousness under 35 U.S.C. § 103, we need to consider if a person having ordinary skill in the art (PHOSITA) would have been motivated to combine prior art references to arrive at the claimed invention, with a reasonable expectation of success. Since I do not have access to a database of prior art specifically cited against US9860450B2 during its prosecution or a general patent prior art search tool, I cannot definitively identify specific prior art combinations.

However, based on the general description of the invention and common knowledge in the field of image processing and digital video stabilization prior to the patent's priority date of March 25, 2004, I can outline potential areas of prior art and likely motivations for combination.

General Areas of Prior Art (pre-March 25, 2004):

• Image Stabilization Techniques:
  • Optical Image Stabilization (OIS): Electro-mechanical devices moving lens elements to compensate for camera movement. The patent itself mentions these as prior art, noting their cost and impact on lens characteristics.
  • Digital Image Stabilization (DIS): Techniques involving analyzing motion within image sequences and computationally shifting or warping frames to counteract shake. This often involves motion estimation algorithms.
• Motion Sensing Technology:
  • Accelerometers and Gyroscopes: Sensors capable of detecting and recording motion (linear and angular acceleration/velocity) were available and increasingly integrated into consumer electronics by the early 2000s.
• Video Processing and Compression:
  • Techniques for capturing, processing, and compressing digital video were well-established, including methods for frame alignment, noise reduction, and various forms of video compression (e.g., MPEG standards).
• Image Deconvolution/Deblurring:
  • Algorithms for deblurring images by estimating a point spread function (PSF) or transfer function (e.g., using blind deconvolution or explicit motion data) were known in academic and specialized imaging fields.

Potential Obviousness Argument (Hypothetical):

A PHOSITA in March 2004, working on improving digital video quality and counteracting camera shake, would likely have been aware of:

1. Digital image stabilization methods that analyze pixel-level motion within a video stream to align frames.
2. Hardware-based motion sensors (accelerometers/gyroscopes) that could provide precise external measurements of camera movement.
3. The benefits of combining information from different sources for more robust results in signal processing.

Hypothetical Combination of Prior Art References:

Let's assume the existence of hypothetical prior art references at the time of the invention:

• Prior Art A (Digital Video Stabilization): Discloses a method and apparatus for stabilizing digital video by analyzing inter-frame motion vectors to computationally shift frames and combine them into a stabilized video. This reference, however, relies solely on image content analysis for motion estimation.
• Prior Art B (Motion Sensor Integration): Discloses the integration of accelerometers or gyroscopes into portable electronic devices (e.g., digital cameras) to detect device orientation or movement for various applications (e.g., screen rotation, rudimentary shake detection for single photos). This reference, however, does not explicitly detail using sensor data for pixel-level video stabilization.
• Prior Art C (Image Deblurring/Deconvolution Principles): Discloses the mathematical principles of using a known or estimated motion transfer function to deconvolve a blurred image and recover a sharper original. This reference might demonstrate this for single still images or in contexts not directly related to real-time video stabilization using motion sensors.

Motivation for Combination and Explanation of Obviousness:

A PHOSITA, seeing the limitations of solely relying on image-based motion estimation (e.g., difficulty with uniform motion, featureless regions, or fast/complex movements) (from Prior Art A), would be motivated to seek more robust and accurate motion information. Prior Art B would teach the availability and integration of motion sensors into imaging devices. It would be an obvious design choice for the PHOSITA to combine the direct motion measurements from the sensors (Prior Art B) with the digital video stabilization techniques (Prior Art A).

The motivation would be to improve the accuracy and robustness of video stabilization. The motion sensor data provides an objective, direct measurement of camera movement, which could be used to:

• Corroborate or refine image-based motion estimation, particularly in challenging scenarios.
• Directly calculate the required shift values (vertical and horizontal) for each frame, as taught by US9860450B2, rather than relying solely on computationally intensive and potentially less accurate image analysis.
• Provide a feed-forward mechanism for stabilization, predicting motion rather than reacting to it solely from pixel changes.

Prior Art C (deblurring principles) would further teach the underlying mathematical basis for using a "transfer function" representing motion to "undo" blur. While US9860450B2 describes deriving vertical and horizontal shift values rather than a full deconvolution filter, the concept of using measured motion to inform image correction for blur reduction is conceptually linked. A PHOSITA would readily understand that the motion information from the sensors could be translated into precise shift values to align video frames, effectively reducing blur caused by camera shake.

Regarding the claims with video compression (Claims 28 and 29), applying video compression to a final, corrected video is a standard practice in digital video processing to reduce file size and facilitate storage/transmission. A PHOSITA would consider it an obvious implementation choice to apply known video compression techniques to the output of any video processing pipeline, including one for stabilization. There would be no inventive step in adding a video compression step to an already stabilized video.

Therefore, the combination of these hypothetical prior art references, driven by the motivation to improve the accuracy and reliability of digital video stabilization, would likely render the claims of US9860450B2 obvious to a PHOSITA.