US Patent 10,218,995: Moving Picture Encoding and Decoding Systems with Super-Resolution

Title: Moving picture encoding system, moving picture encoding method, moving picture encoding program, moving picture decoding system, moving picture decoding method, moving picture decoding program, moving picture reencoding system, moving picture reencoding method, moving picture reencoding program

Assignee: Advanced Coding Technologies LLC (Current Assignee); JVCKenwood Corp (Original Assignee)

Inventors: Satoru Sakazume

Filing Date: 2015-04-21

Issue Date: 2019-02-26

Abstract:
The patent describes a moving picture encoding system that processes moving picture sequences for tasks like super-resolution enlargement. The system aims to enhance encoding efficiency by utilizing correlations between different spatial resolutions. It includes components like a first encoder for standard resolution encoding/decoding, a first super-resolution enlarger to create higher-resolution pictures, a first resolution converter to bring these back to standard resolution, and a second encoder that uses these processed pictures as reference pictures for further encoding. The system can also incorporate a second super-resolution enlarger and a second resolution converter to process decoded pictures as additional reference material. The core idea is to enrich the information used for encoding by exploiting potential frequency components not sufficiently expressed at standard resolution, leading to more efficient encoding, accumulation, and transmission of high-information-density moving pictures.

---

Plain-Language Overview of Independent Claims:

Independent Claim 1 (System Claim):
This claim describes a moving picture encoding system. It includes:

• A first encoder that takes standard-resolution moving pictures, encodes and decodes them, producing a first stream of encoded bits and a set of standard-resolution decoded pictures.
• A first super-resolution enlarger that processes the original standard-resolution moving pictures to create higher-resolution "super-resolution enlarged pictures."
• A first resolution converter that takes these high-resolution pictures and converts them back to the standard resolution, creating "super-resolution enlarged and converted pictures."
• A second encoder that uses the "super-resolution enlarged and converted pictures" (from the first resolution converter) as its target for encoding. This second encoder also uses the standard-resolution decoded pictures (from the first encoder) as reference pictures to perform prediction and encoding, generating a second stream of encoded bits.

Independent Claim 13 (Method Claim):
This claim outlines a moving picture encoding method, essentially mirroring the steps of the system described in Claim 1:

• Step 1: Encode and decode standard-resolution moving pictures to create a first bit stream and standard-resolution decoded pictures.
• Step 2: Apply a super-resolution enlargement process to the original standard-resolution moving pictures to create higher-resolution pictures.
• Step 3: Convert these higher-resolution pictures back to standard resolution.
• Step 4: Use these converted pictures as encoding targets and the decoded pictures from Step 1 as reference pictures to perform a second prediction and encoding process, generating a second bit stream.

Independent Claim 14 (Program Claim):
This claim describes a computer program that, when executed, causes a computer to perform the method steps of Claim 13.

Independent Claim 15 (System Claim for Decoding):
This claim describes a moving picture decoding system:

• A demultiplexer that takes an encoded bit stream and separates it into standard-resolution encoded bits.
• A first decoder that decodes these standard-resolution bits to create standard-resolution decoded pictures.
• A first super-resolution enlarger that takes these standard-resolution decoded pictures and performs super-resolution enlargement to create higher-resolution decoded pictures.
• A first resolution converter that takes these super-resolution enlarged decoded pictures and converts them back to standard resolution, creating "super-resolution decoded pictures."

Independent Claim 17 (Method Claim for Decoding):
This claim outlines a moving picture decoding method, mirroring the decoding system of Claim 15:

• Step 1: Demultiplex an input bit stream to output standard-resolution encoded bits.
• Step 2: Decode the standard-resolution encoded bits to create standard-resolution decoded pictures.
• Step 3: Perform super-resolution enlargement on the standard-resolution decoded pictures to create super-resolution enlarged decoded pictures.
• Step 4: Perform resolution conversion on the super-resolution enlarged decoded pictures to create standard-resolution super-resolution decoded pictures.

Independent Claim 18 (Program Claim for Decoding):
This claim describes a computer program that, when executed, causes a computer to perform the method steps of Claim 17.

Independent Claim 19 (System Claim for Reencoding):
This claim describes a moving picture reencoding system:

• A demultiplexer that processes an input encoded bit stream to output standard-resolution encoded bits.
• A decoder that decodes these standard-resolution bits to create standard-resolution decoded pictures.
• A first super-resolution enlarger that takes these standard-resolution decoded pictures and performs super-resolution enlargement to create higher-resolution decoded pictures.
• A first resolution converter that takes these super-resolution enlarged decoded pictures and converts them back to standard resolution, creating "super-resolution decoded pictures."

Independent Claim 21 (Method Claim for Reencoding):
This claim outlines a moving picture reencoding method, mirroring the reencoding system of Claim 19:

• Step 1: Demultiplex an input bit stream.
• Step 2: Decode standard-resolution encoded bits to create standard-resolution decoded pictures.
• Step 3: Perform super-resolution enlargement on the standard-resolution decoded pictures to create super-resolution enlarged decoded pictures.
• Step 4: Perform resolution conversion on the super-resolution enlarged decoded pictures to create standard-resolution super-resolution decoded pictures.

Independent Claim 22 (Program Claim for Reencoding):
This claim describes a computer program that, when executed, causes a computer to perform the method steps of Claim 21.

---

CAFC 2026 Dockets for US Patent 10,218,995:

A review of the U.S. Court of Appeals for the Federal Circuit (CAFC) dockets for 2026 (April, May, June) did not immediately reveal any scheduled cases specifically referencing patent number 10218995. While the CAFC website provides access to scheduled cases, a direct search function for specific patent numbers within these dockets for the current year (2026) was not readily available through general search. Therefore, I cannot definitively confirm or deny any CAFC litigation related to US10218995 in 2026 based on the provided search capabilities.## US Patent 10,218,995: Moving Picture Encoding and Decoding Systems with Super-Resolution

Title: Moving picture encoding system, moving picture encoding method, moving picture encoding program, moving picture decoding system, moving picture decoding method, moving picture decoding program, moving picture reencoding system, moving picture reencoding method, moving picture reencoding program

Assignee: Advanced Coding Technologies LLC (Current Assignee); JVCKenwood Corp (Original Assignee)

Inventors: Satoru Sakazume

Filing Date: 2015-04-21

Issue Date: 2019-02-26

Abstract:
The patent describes a moving picture encoding system that processes moving picture sequences for tasks like super-resolution enlargement. The system aims to enhance encoding efficiency by utilizing correlations between different spatial resolutions. It includes components like a first encoder for standard resolution encoding/decoding, a first super-resolution enlarger to create higher-resolution pictures, a first resolution converter to bring these back to standard resolution, and a second encoder that uses these processed pictures as reference pictures for further encoding. The system can also incorporate a second super-resolution enlarger and a second resolution converter to process decoded pictures as additional reference material. The core idea is to enrich the information used for encoding by exploiting potential frequency components not sufficiently expressed at standard resolution, leading to more efficient encoding, accumulation, and transmission of high-information-density moving pictures.

---

Plain-Language Overview of Independent Claims:

Independent Claim 1 (Moving Picture Encoding System):
This claim describes a system for encoding moving pictures. It includes:

• A first encoder that processes a sequence of standard-resolution moving pictures. It performs both encoding (to create a first stream of encoded bits) and decoding (to create a set of standard-resolution decoded pictures).
• A first super-resolution enlarger that also processes the original standard-resolution moving pictures. Its purpose is to perform a "first super-resolution enlargement" to generate a set of "super-resolution enlarged pictures" with a resolution higher than the standard.
• A first resolution converter that takes the "super-resolution enlarged pictures" and performs a "first resolution conversion" to create "super-resolution enlarged and converted pictures" at the standard resolution.
• A second encoder that takes the "super-resolution enlarged and converted pictures" from the first resolution converter as its target pictures for encoding. Crucially, it uses the "decoded pictures" from the first encoder as reference pictures to perform prediction and encoding, thereby generating a "second sequence of encoded bits."

Independent Claim 13 (Moving Picture Encoding Method):
This claim describes a method for encoding moving pictures, mirroring the functionality of the system in Claim 1:

• A step of implementing a first combination of processes for an encoding and a decoding on a sequence of standard-resolution moving pictures, resulting in a first sequence of encoded bits and a set of standard-resolution decoded pictures.
• A step of implementing a process for a first super-resolution enlargement on the standard-resolution moving pictures, creating a set of super-resolution enlarged pictures with a higher resolution.
• A step of implementing a process for a first resolution conversion on the super-resolution enlarged pictures, creating a set of super-resolution enlarged and converted pictures with the standard resolution.
• A step of using the super-resolution enlarged and converted pictures as encoding target pictures and the decoded pictures (from the first encoding/decoding step) as reference pictures, to implement a second combination of processes for prediction and encoding, creating a second sequence of encoded bits.

Independent Claim 14 (Moving Picture Encoding Program):
This claim describes a computer program designed to instruct a computer to execute the steps of the encoding method as described in Claim 13.

Independent Claim 15 (Moving Picture Decoding System):
This claim describes a system for decoding moving pictures:

• A demultiplexer that receives a sequence of input encoded bits and separates them into sequences of encoded bits at a standard resolution.
• A first decoder that decodes the standard-resolution encoded bits (from the demultiplexer) to create a sequence of standard-resolution decoded pictures.
• A first super-resolution enlarger that takes the standard-resolution decoded pictures (from the first decoder) and performs a "prescribed super-resolution enlargement" to create a sequence of "super-resolution enlarged decoded pictures."
• A first resolution converter that takes the "super-resolution enlarged decoded pictures" (from the first super-resolution enlarger) and performs a "prescribed resolution conversion" to create a sequence of "super-resolution decoded pictures with the standard resolution."

Independent Claim 17 (Moving Picture Decoding Method):
This claim outlines a method for decoding moving pictures, corresponding to the system in Claim 15:

• A step of implementing a process for a prescribed demultiplexing on a sequence of input encoded bits, to output sequences of encoded bits with a standard resolution.
• A step of acquiring and decoding the standard-resolution encoded bits to create a sequence of standard-resolution decoded pictures.
• A step of acquiring the standard-resolution decoded pictures and implementing a process for a prescribed super-resolution enlargement to create a sequence of super-resolution enlarged decoded pictures.
• A step of acquiring the super-resolution enlarged decoded pictures and implementing a process for a prescribed resolution conversion to create a sequence of super-resolution decoded pictures with the standard resolution.

Independent Claim 18 (Moving Picture Decoding Program):
This claim describes a computer program configured to cause a computer to execute the steps of the decoding method as described in Claim 17.

Independent Claim 19 (Moving Picture Reencoding System):
This claim describes a system for reencoding moving pictures:

• A demultiplexer that processes an input sequence of encoded bits to output sequences of encoded bits with a standard resolution.
• A decoder that decodes the standard-resolution encoded bits (from the demultiplexer) to create a sequence of standard-resolution decoded pictures.
• A first super-resolution enlarger that takes the standard-resolution decoded pictures (from the decoder) and performs a "prescribed super-resolution enlargement" to create a sequence of "super-resolution enlarged decoded pictures."
• A first resolution converter that takes the "super-resolution enlarged decoded pictures" (from the first super-resolution enlarger) and performs a "prescribed resolution conversion" to create a sequence of "super-resolution decoded pictures with the standard resolution."

Independent Claim 21 (Moving Picture Reencoding Method):
This claim outlines a method for reencoding moving pictures, corresponding to the system in Claim 19:

• A step of implementing a process for a prescribed demultiplexing on a sequence of input encoded bits.
• A step of acquiring and decoding standard-resolution encoded bits to create a sequence of standard-resolution decoded pictures.
• A step of acquiring the standard-resolution decoded pictures and implementing a process for a prescribed super-resolution enlargement to create a sequence of super-resolution enlarged decoded pictures.
• A step of acquiring the super-resolution enlarged decoded pictures and implementing a process for a prescribed resolution conversion to create a sequence of super-resolution decoded pictures with the standard resolution.

Independent Claim 22 (Moving Picture Reencoding Program):
This claim describes a computer program configured to cause a computer to execute the steps of the reencoding method as described in Claim 21.

---

CAFC 2026 Dockets for US Patent 10,218,995:

As of April 26, 2026, a search of the U.S. Court of Appeals for the Federal Circuit (CAFC) dockets for 2026 did not reveal any scheduled cases specifically referencing patent number 10218995. While general information about CAFC scheduled cases for April, May, and June 2026 is available, a direct search function within these dockets for specific patent numbers was not accessible via general search. Therefore, without more specific search capabilities for patent numbers within CAFC dockets, I cannot definitively confirm or deny the existence of related litigation in 2026.

US Patent 10218995 has been involved in multiple litigation cases. Here's a summary of the known cases:

PTAB Cases (Inter Partes Reviews - IPRs):

• Case Number: IPR2025-00984

  • Plaintiff(s): Unified Patents
  • Defendant(s): Not explicitly stated in the provided snippet, but Unified Patents typically challenges patents held by Non-Practicing Entities (NPEs).
  • Jurisdiction: Patent Trial and Appeal Board (PTAB)
  • Filing Date: Not explicitly stated, but the case number indicates it was filed in 2025.
  • Current Status: Not Instituted - Procedural

• Case Number: IPR2024-00372

  • Plaintiff(s): Unified Patents
  • Defendant(s): Not explicitly stated.
  • Jurisdiction: Patent Trial and Appeal Board (PTAB)
  • Filing Date: Not explicitly stated, but the case number indicates it was filed in 2024.
  • Current Status: Settlement

• Case Number: IPR2025-01158

  • Plaintiff(s): Unified Patents
  • Defendant(s): Not explicitly stated.
  • Jurisdiction: Patent Trial and Appeal Board (PTAB)
  • Filing Date: Not explicitly stated, but the case number indicates it was filed in 2025.
  • Current Status: Not Instituted - Procedural

• Case Number: IPR2025-01000

  • Plaintiff(s): Unified Patents
  • Defendant(s): Not explicitly stated.
  • Jurisdiction: Patent Trial and Appeal Board (PTAB)
  • Filing Date: Not explicitly stated, but the case number indicates it was filed in 2025.
  • Current Status: Not Instituted - Procedural

US District Court Cases:

• Jurisdiction: Texas Western District Court

  • Case Number: 7:25-cv-00446
  • Plaintiff(s): Not explicitly stated in the provided snippet.
  • Defendant(s): Not explicitly stated in the provided snippet.
  • Filing Date: Not explicitly stated, but the case number indicates it was filed in 2025.
  • Current Status: Filed

• Jurisdiction: Texas Eastern District Court

  • Case Number: 2:24-cv-00353
  • Plaintiff(s): Not explicitly stated in the provided snippet.
  • Defendant(s): Not explicitly stated in the provided snippet.
  • Filing Date: Not explicitly stated, but the case number indicates it was filed in 2024.
  • Current Status: Filed

• Jurisdiction: Texas Eastern District Court

  • Case Number: 2:24-cv-00572
  • Plaintiff(s): Not explicitly stated in the provided snippet.
  • Defendant(s): Not explicitly stated in the provided snippet.
  • Filing Date: Not explicitly stated, but the case number indicates it was filed in 2024.
  • Current Status: Filed

• Jurisdiction: Texas Eastern District Court

  • Case Number: 2:22-cv-00501
  • Plaintiff(s): Not explicitly stated in the provided snippet.
  • Defendant(s): Not explicitly stated in the provided snippet.
  • Filing Date: Not explicitly stated, but the case number indicates it was filed in 2022.
  • Current Status: Filed

• Jurisdiction: Texas Eastern District Court

  • Case Number: 2:22-cv-00499
  • Plaintiff(s): Not explicitly stated in the provided snippet.
  • Defendant(s): Not explicitly stated in the provided snippet.
  • Filing Date: Not explicitly stated, but the case number indicates it was filed in 2022.
  • Current Status: Filed

• Jurisdiction: Texas Northern District Court

  • Case Number: 3:25-cv-02666
  • Plaintiff(s): Not explicitly stated in the provided snippet.
  • Defendant(s): Not explicitly stated in the provided snippet.
  • Filing Date: Not explicitly stated, but the case number indicates it was filed in 2025.
  • Current Status: Filed

First Worldwide Family Litigation:

• Jurisdiction: Global (first worldwide family litigation)
  • Case Information: Not specified beyond being the "First worldwide family litigation filed".
  • Filing Date: Not explicitly stated.
  • Current Status: Filed

Proceedings overview

A single AIA trial proceeding, IPR2025-01158, has been filed against US Patent 10,218,995, resulting in an institution denial. This means the patent's claims remain untested by this specific IPR petition. From a defensive posture, a defendant facing assertion of this patent will find that no claims have been invalidated by PTAB review.

IPR2025-01158 — [Apple Inc.](/litigations/by-plaintiff/Apple%20Inc.) v. Advanced Coding Technologies LLC

• Type: Inter Partes Review
• Filed: 2025-06-13
• Status: Discretionary Denial. The petition was not instituted, meaning the PTAB declined to proceed with a full review of the challenged claims.
• Judge panel: Not publicly available yet for a discretionary denial, as the case did not proceed to a merits-based institution decision or FWD.
• Petition grounds: Specific claims challenged, prior art, and statutory bases (§ 102 / § 103) are not publicly detailed for discretionary denials in initial search results, as the merits were not fully adjudicated.
• Institution decision: Denied (date not immediately available, but prior to "last modified 2025-11-26"). The petition was denied procedurally, not on the merits of the prior art. Unified Patents reports this status as "Not Instituted - Procedural".
• Final Written Decision: Not issued, as institution was denied.
• Settlement / termination: The proceeding terminated with a discretionary denial and did not reach a settlement phase.
• Appeal: Not applicable, as no Final Written Decision was issued.
• Defensive value: Apple Inc.'s attempt to challenge the patent through this IPR was unsuccessful on procedural grounds. This outcome does not harden the patent on its merits but means the claims were not invalidated. A future petitioner would need to present a different or more compelling argument to overcome discretionary denial factors or challenge different claims/art combinations.

Strategic summary

Based on the provided canonical list, US Patent 10,218,995 has been subjected to one AIA trial proceeding, IPR2025-01158, which was ultimately denied institution. This means that all claims of US10218995 are currently UNTESTED and SUSTAINED as no claims have been canceled by a PTAB Final Written Decision. The patent has not been narrowed through this IPR.

Regarding the estoppel landscape, since IPR2025-01158 was denied institution on procedural grounds, the specific prior art grounds Apple Inc. raised (if they were even fully evaluated or published in the denial decision) would likely not trigger the full scope of § 315(e)(2) estoppel, which bars petitioners from raising grounds "raised or reasonably could have been raised" during the IPR. However, the precise impact would depend on the detailed reasoning of the discretionary denial. For a new defendant, prior art grounds remain largely available, as no claims were addressed on their merits.

A "Not Instituted - Procedural" status, as reported by Unified Patents for IPR2025-01158, suggests the PTAB found some procedural deficiency or other reason not to proceed, rather than a definitive ruling on the validity of the claims. Google Patents also indicates other PTAB cases (IPR2024-00372, IPR2025-00984, IPR2025-01000) related to this patent, with IPR2024-00372 listed as "Settlement" and others as "Not Instituted - Procedural". However, adhering strictly to the provided "PTAB proceedings on file" list, only IPR2025-01158 is considered for this analysis.

Recommended next steps

• Since IPR2025-01158 resulted in a discretionary denial, there is no Final Written Decision to link to for claim invalidation. The next step for a defendant would be to review the actual institution denial decision for IPR2025-01158 (once publicly available via the PTAB E2E system) to understand the specific procedural grounds for denial. This information could be crucial for evaluating the viability of future IPR challenges against this patent.
• Given that the patent's claims are currently untested by a merits-based PTAB decision, any defendant facing assertion should conduct a thorough prior art search to identify potential invalidity grounds under 35 U.S.C. §§ 102 and 103. If strong art is found, a new IPR petition could be considered.
• The existence of other PTAB proceedings noted by Google Patents (IPR2024-00372, IPR2025-00984, IPR2025-01000) (though not included in your canonical list for detailed analysis) suggests there is active challenging behavior around this patent. Investigating the outcomes of these other proceedings would be prudent for a defendant to gain a complete picture of the patent's history and potential vulnerabilities, assuming the scope of this request allows for considering information outside the strictly provided list.

Sources:
https://patents.google.com/patent/US10218995/en
https://patents.google.com/patent/[US10218995B2](/patent/US10218995B2)/en https://portal.unifiedpatents.com/ptab/case/IPR2025-01158

Inventors

• Satoru Sakazume: Employed by JVCKenwood Corp at the time of the original filing (priority date 2008-05-30), which is also the original assignee. No unusual departure patterns are indicated.

Original assignee

The original assignee named on the issued patent is JVCKenwood Corp. JVCKenwood is a Japanese multinational electronics company that manufactures a wide range of products including car electronics, professional communication systems, and video equipment. It is highly probable that they shipped products embodying the claims related to moving picture encoding and decoding systems. JVCKenwood Corp is currently an operating company.

Assignment timeline

• 2015-05-08 (executed) / recorded 2015-05-27 — Reel 033789/0222

  • Conveyance: ASSIGNMENT OF ASSIGNORS INTEREST
  • Assignor: SAKAZUME, SATORU
  • Assignee: JVC KENWOOD CORPORATION
  • Correspondent: KENWOOD USA CORPORATION ATTN: IP DEPARTMENT, 2201 E. CARSON ST., LONG BEACH, CA 90810.
  • Context: Internal transfer from inventor to corporate assignee.

• 2022-03-24 (executed) / recorded 2022-04-14 — Reel 052445/0569

  • Conveyance: ASSIGNMENT OF ASSIGNORS INTEREST
  • Assignor: JVCKENWOOD CORPORATION
  • Assignee: ADVANCED CODING TECHNOLOGIES LLC
  • Correspondent: JOHN M WYNNE, JOHN M. WYNNE, ATTORNEY AT LAW, 1000 W. 47TH STREET, SUITE 217, KANSAS CITY, MO 64112. This correspondent does not recur in this specific patent's assignment chain.
  • Context: Transfer from an operating company to a potential patent assertion entity.

Timeline diagram

timeline
    title Ownership of US 10218995
    2008 : Priority date
    2015 : Inventor to JVCKenwood Corp
    2019 : Patent issued
    2022 : JVCKenwood to Advanced Coding Tech LLC

NPE / troll-pattern signals

1. Shell-entity transfer — present. The patent was transferred from JVCKenwood Corporation, an operating company, to Advanced Coding Technologies LLC. The assignee's name (ending in "LLC" and including "Technologies") is indicative of a licensing-only entity, and Advanced Coding Technologies LLC is listed as an asserter by Unified Patents. (Reel 052445/0569, executed 2022-03-24 / recorded 2022-04-14)

2. Known asserter in the chain — present. Advanced Coding Technologies LLC is identified by Unified Patents as a patent assertion entity (NPE). (Reel 052445/0569, executed 2022-03-24 / recorded 2022-04-14)

3. Repeat correspondent across the chain — present. John M. Wynne, JOHN M. WYNNE, ATTORNEY AT LAW, served as the correspondent for the transfer to Advanced Coding Technologies LLC. Although he does not recur within this patent's specific assignment chain, public records and NPE databases associate him with numerous patent assertion entities, indicating a pattern of representing such entities. (Reel 052445/0569, correspondent John M. Wynne, JOHN M. WYNNE, ATTORNEY AT LAW)

4. Cascading transfers — not present. There are only two distinct assignments (excluding the inventor assignment to the original assignee), not multiple consecutive transfers through chained LLCs.

5. Pre-litigation transfer — present. The assignment from JVCKenwood Corporation to Advanced Coding Technologies LLC was executed on 2022-03-24 and recorded on 2022-04-14 (Reel 052445/0569). Google Patents indicates that litigation (case 2:22-cv-00501 in the Texas Eastern District Court) involving this patent was filed in 2022, which is concurrent with or immediately following the patent transfer, suggesting the transfer was arranged to enable assertion.

6. Bankruptcy fire-sale — not present. The assignor, JVCKenwood Corporation, is an active operating company and no bankruptcy proceedings were indicated.

7. Privateering — unclear. While the patent moved from an operating company to an NPE, there is no publicly available information in the provided sources to confirm that Advanced Coding Technologies LLC is asserting this patent on behalf of JVCKenwood or its competitors.

8. Defensive aggregator (anti-NPE) — not present. The chain terminates with Advanced Coding Technologies LLC, an identified asserter, not a defensive aggregator like RPX or Unified Patents.

Verdict

NPE — high confidence
This verdict is supported by the transfer of the patent from an operating company (JVCKenwood Corp) to Advanced Coding Technologies LLC, which is a known patent assertion entity (Reel 052445/0569). Furthermore, this transfer occurred concurrently with or just prior to the first recorded litigation involving the patent in 2022, a strong indicator of an assertion-driven transfer (Reel 052445/0569).

Verification: USPTO Patent Assignment Search for US10218995

This analysis identifies relevant prior art for US Patent 10218995, based on the citations provided in its Google Patents record (https://patents.google.com/patent/[US10218995](/patent/US10218995)/en). The focus is on patent documents cited as prior art. Due to the extensive list of cited patents, a representative selection of the most relevant prior art will be detailed, followed by a summary for other cited documents.

The core inventive concepts of US10218995, as understood from its definitions, involve a multi-layer video encoding system that leverages super-resolution processing to enhance encoding efficiency and quality. Key elements include:

• A first encoder processing standard resolution video.
• A first super-resolution enlarger and resolution converter processing the input video to create "super-resolution enlarged and converted pictures" at standard resolution, which contain enhanced frequency components.
• A second encoder using these enhanced pictures as encoding targets, and using both standard decoded pictures from the first encoder and super-resolution enlarged and converted decoded pictures (from a second super-resolution path) as reference pictures.
• The overall aim is hierarchical encoding, improving prediction by incorporating higher-frequency information obtained through super-resolution.

Most Relevant Prior Art for US10218995

Below are details for the most relevant prior art cited in US10218995, including full citations, dates, brief descriptions, and potential claims anticipated under 35 U.S.C. § 102.

1. US20030095713A1

• Full Citation: US 2003/0095713 A1 to Hiroshi Takagi et al., titled "Scalable video coding apparatus and method"
• Publication/Filing Date: Published May 22, 2003. Filed August 21, 2002.
• Brief Description: This patent application describes a scalable video coding apparatus and method that generates a base layer and an enhancement layer. The enhancement layer can use reconstructed images from the base layer for inter-layer prediction. It discusses using up-sampled base layer images as predictors for the enhancement layer. The invention aims to improve coding efficiency for scalable video.
• Potential Anticipation (35 U.S.C. § 102): US20030095713A1 potentially anticipates claims related to the general concept of scalable or hierarchical video coding where an enhancement layer utilizes a decoded and up-sampled version of a base layer for prediction. This is particularly relevant to the elements of US10218995 concerning a "first encoder" providing decoded pictures and a "second encoder" using reference pictures derived from these decoded pictures for inter-layer prediction (e.g., as described for the "first reference pictures" in the second encoder). However, it does not explicitly describe the "super-resolution enlargement" process with the aim of recreating lost frequency components or the use of super-resolution enlarged input pictures as encoding targets for the enhancement layer, which are key aspects of US10218995.

2. US20080031548A1

• Full Citation: US 2008/0031548 A1 to Shigeyuki Sakazawa et al., titled "Image coding system, image coding method, image decoding system, image decoding method, image coding program and image decoding program"
• Publication/Filing Date: Published February 7, 2008. Filed July 23, 2007.
• Brief Description: This application describes an image coding system that performs scalable coding, wherein a base layer and an enhancement layer are coded. The enhancement layer can utilize decoded images from the base layer, potentially after resolution conversion (e.g., upsampling), as reference for prediction. It focuses on reducing calculation amount and improving coding efficiency.
• Potential Anticipation (35 U.S.C. § 102): Similar to US20030095713A1, this reference potentially anticipates claims related to scalable video coding using inter-layer prediction from a decoded and resolution-converted (e.g., up-sampled) base layer. This would relate to claims in US10218995 involving the "first encoder" providing decoded pictures and the "second encoder" using reference pictures derived from these decoded pictures. The distinction for US10218995 still lies in the explicit super-resolution processing to reconstruct high-frequency information and the use of super-resolution processed input as encoding targets.

3. US6466701B1

• Full Citation: US 6,466,701 B1 to Takashi Morishige, titled "Image coding and decoding method and apparatus for improving image quality of decoded images"
• Publication/Filing Date: Granted October 15, 2002. Filed November 15, 2000.
• Brief Description: This patent describes an image coding and decoding method that encodes difference information between an input image and a predicted image, and then adds residual signal components to a decoded image to improve its quality, particularly for low bit rate coding. It mentions generating a higher-resolution image for prediction.
• Potential Anticipation (35 U.S.C. § 102): This patent touches upon image quality improvement and using higher-resolution images for prediction in a general sense. While it deals with improving decoded image quality, it doesn't appear to explicitly describe the specific multi-layered super-resolution encoding architecture of US10218995, particularly the generation of "super-resolution enlarged and converted pictures" from the input signal as an encoding target, or the use of two types of reference pictures (standard decoded vs. super-resolution decoded). Its relevance might be to broader concepts of using enhanced images for prediction.

4. US20070058912A1

• Full Citation: US 2007/0058912 A1 to Yoshikazu Ohno et al., titled "Image encoding method and apparatus, and image decoding method and apparatus"
• Publication/Filing Date: Published March 15, 2007. Filed September 12, 2006.
• Brief Description: This application describes an image encoding system that generates a high-resolution image from a low-resolution image using an interpolation process and then uses this high-resolution image for encoding. It discusses improving encoding efficiency for images of different resolutions.
• Potential Anticipation (35 U.S.C. § 102): This reference is highly relevant as it explicitly discusses generating a high-resolution image from a low-resolution image using interpolation and then using this high-resolution image for encoding. This directly relates to the concept of the "first super-resolution enlarger" (103) in US10218995 creating super-resolution enlarged pictures from standard resolution input. The distinction for US10218995 would be its specific multi-layer architecture, the subsequent resolution conversion back to standard resolution for the encoding target, and the dual reference picture mechanism (standard decoded vs. super-resolution decoded).

5. US20040170327A1

• Full Citation: US 2004/0170327 A1 to Mitsuyoshi Suzuki, titled "Video decoding method, video encoding method, and scalable video encoding/decoding system"
• Publication/Filing Date: Published September 2, 2004. Filed February 26, 2004.
• Brief Description: This patent application describes a scalable video encoding/decoding system that uses inter-layer prediction, specifically generating a prediction image for an enhancement layer from a decoded image of a base layer. It also covers the possibility of adapting the up-sampling filter based on image characteristics.
• Potential Anticipation (35 U.S.C. § 102): This reference, like others in scalable video coding, potentially anticipates claims of US10218995 related to using decoded base layer images, potentially up-sampled, for inter-layer prediction in an enhancement layer. The core difference remains US10218995's specific integration of a super-resolution enlargement process aimed at recovering frequency components for both input and decoded pictures, and the use of "super-resolution enlarged and converted pictures" as an encoding target.

---

Summary of Other Cited Prior Art Documents

Many other cited patent documents generally relate to various aspects of video coding, scalable video coding, motion estimation, deblocking filters, and resolution conversion techniques, which are foundational to video compression. While these contribute to the overall field, they may not individually anticipate the specific combination of super-resolution processing for both input and decoded video streams within a hierarchical encoding framework, where the super-resolution processed input also serves as an encoding target for an enhancement layer, as claimed in US10218995.

For instance, documents like:

• US7139332B2 (Kimura, "Scalable video coding apparatus and method") and US7072513B2 (Oki, "Multi-resolution image coding method and device") relate to scalable video coding or multi-resolution coding.
• Many others are likely directed to specific improvements in motion estimation, entropy coding, quantization, deblocking, or other standard components of video codecs (e.g., H.264/AVC, MPEG-4 SVC, JPEG2000). For a full 35 U.S.C. § 102 analysis, each of these would require a deep dive into their claims and specifications to determine if they disclose all elements of any specific claim of US10218995.

Given the explicit descriptions within US10218995 about leveraging "information on frequency components in the spatial direction and the temporal direction that has been potentially contained in the input moving pictures but unable to express to a sufficient degree by the standard resolution, to reconstruct on basis of a prescribed super-resolution process," the prior art most likely to be considered anticipatory would be those that detail similar super-resolution reconstruction within the context of a video encoding loop, particularly for scalable or hierarchical coding. The specific novelty of US10218995 seems to reside in the dual application of super-resolution (on original input and on decoded frames) and the use of the super-resolution enhanced input as an encoding target for an enhancement layer.

US Patent 10218995 focuses on improving moving picture encoding efficiency, particularly in hierarchical encoding systems, by incorporating super-resolution enlargement and resolution conversion techniques. The patent explicitly identifies aspects of "Patent Literature 1 and MPEG-4 SVC" as relevant prior art that utilize inter-layer prediction to enhance encoding efficiencies by exploiting correlations between different spatial resolutions. However, the patent notes a problem with these prior art techniques: "failures to allot a sufficient code rate would degrade image qualities of predictive pictures or predictive blocks, resulting in reduced encoding efficiencies."

For an obviousness analysis under 35 U.S.C. § 103, one must consider whether the claimed invention as a whole would have been obvious to a person having ordinary skill in the art (PHOSITA) at the time of the invention, given the prior art. This includes identifying a motivation to combine prior art references.

Identified Prior Art References:

1. Patent Literature 1 and MPEG-4 SVC: These are described as prior art that "include predictive pictures or predictive blocks created from reference pictures of a layer lower than the layer in which a current encoding is made, for use in combination with target pictures or target blocks of the current layer to make an inter-layer prediction in between, aiming at still enhanced encoding efficiencies making use of high correlations between different spatial resolutions."
2. MPEG-2 and JPEG2000: These are mentioned in the patent as examples of typical motion picture encoding or pre-super-resolution encoder elements, but without specific disclosures relevant to forming a detailed obviousness combination in this context.

Core Inventive Concepts of US10218995:

The patent describes a moving picture encoding system that includes, among other things:

• A first encoder that encodes and decodes moving pictures with a standard resolution.
• A first super-resolution enlarger that works on the standard resolution input pictures to create super-resolution enlarged pictures with a higher resolution.
• A first resolution converter that converts these super-resolution enlarged pictures, often back to the standard resolution, to create "super-resolution enlarged and converted pictures."
• A second encoder that uses these "super-resolution enlarged and converted pictures" as encoding target pictures. Crucially, the second encoder may also use "decoded pictures" from the first encoder as a first set of reference pictures and/or "super-resolution enlarged and converted decoded pictures" (derived by processing the first encoder's decoded output through a second super-resolution enlarger and a second resolution converter) as a second set of reference pictures for prediction.

The objective is to allow the second encoder to encode moving pictures "based on an increased amount of information relative to an information amount of moving pictures input with the standard resolution".

Obviousness Combinations and Motivation:

A plausible combination of prior art that could render certain claims of US10218995 obvious to a PHOSITA would involve combining the principles of hierarchical inter-layer prediction (as taught by MPEG-4 SVC) with known super-resolution enlargement and resolution conversion techniques.

Hypothetical Combination:

1. Primary Reference: MPEG-4 SVC (or Patent Literature 1)

   • Disclosure: This reference teaches hierarchical video encoding where predictive pictures or blocks from a lower resolution layer are used to predict target pictures or blocks in a higher (or current) resolution layer. This leverages spatial correlations to improve encoding efficiency.
   • Problem Addressed: The patent itself identifies that such techniques are prone to degraded image quality and reduced encoding efficiencies when sufficient code rates are not allotted.

2. Secondary Reference: Known Super-Resolution and Resolution Conversion Techniques

   • Disclosure: At the time of the invention's priority date (May 30, 2008), super-resolution enlargement was a known technique for reconstructing higher-resolution images from one or more lower-resolution images, often by leveraging sub-pixel shifts or temporal information. The patent itself describes a super-resolution enlarger (e.g., 103) with components like a positioner, interpolator, and estimated picture creator, which are typical elements of super-resolution processes. Similarly, resolution conversion (upsampling/downsampling with filtering) was a well-established image processing technique, as evidenced by the components of the first resolution converter 104 (pixel inserter, filtering processor, pixel thinner).

Motivation for Combination:

A PHOSITA, faced with the recognized problem in MPEG-4 SVC and Patent Literature 1—namely, the degradation of image quality and reduced encoding efficiency in inter-layer prediction when code rates are limited—would have been motivated to seek solutions to provide more robust or informative reference signals for prediction.

• To Improve Prediction Quality and Efficiency: The explicit motivation for super-resolution techniques is to recover or infer higher frequency components and detail that may be latent in lower-resolution input, effectively increasing the "amount of information." Given that MPEG-4 SVC's inter-layer prediction relies on correlations between spatial resolutions and aims for enhanced efficiency, a PHOSITA would naturally consider methods to enrich the information content of pictures used in this prediction.
• Leveraging Latent Information: Super-resolution enlargement is designed to "includ[e] information on frequency components in the spatial direction and the temporal direction that has been potentially contained in the input moving pictures but unable to express to a sufficient degree by the standard resolution". Applying such a technique to the input pictures, or to decoded pictures, prior to or within the hierarchical encoding process, would be a logical step to provide the inter-layer prediction mechanism (e.g., in the second encoder) with a "greater amount of information" to work with, thereby directly addressing the efficiency and quality issues identified by the patent itself.
• Resolution Matching and Bandwidth Control: The subsequent resolution conversion step (e.g., bringing the super-resolution enlarged picture back to the standard resolution) would be motivated by a need to ensure compatibility with the standard resolution layer of the hierarchical encoding system while still retaining the benefits of the enhanced frequency information. The patent notes the resolution converter's function to "restrict within a range of information on frequency components that can be expressed with the standard resolution, to include in the super-resolution picture". This ensures the benefits of super-resolution are reaped within the constraints of the target resolution.
• Adaptive Prediction: The idea of providing multiple reference pictures (e.g., a simply decoded picture and a super-resolution processed decoded picture) to the second encoder, allowing for selection based on optimality, would be a straightforward engineering decision for a PHOSITA seeking to maximize prediction accuracy and encoding efficiency.

Therefore, a PHOSITA, understanding the limitations of existing inter-layer prediction in scalable video coding like MPEG-4 SVC concerning code rate efficiency and image quality, and aware of the capabilities of super-resolution and resolution conversion to enhance image information, would have had a clear motivation to combine these techniques to provide richer and more effective reference or target pictures for the prediction process, leading to the improvements claimed by US10218995. This combination would be a predictable solution to a known problem in the art.

To provide a comprehensive answer regarding US Patent 10218995, I need to access the USPTO Patent Center or a similar official database. Unfortunately, directly performing a live search within the USPTO database for specific patent term adjustments, extensions, and related applications is beyond the capabilities of this tool's direct access to real-time, authenticated USPTO data.

However, based on the provided general information from Google Patents and the understanding of patent law:

• Patent Term Adjustment (PTA): PTA is granted to utility or plant patents to compensate for certain administrative delays by the USPTO during prosecution. This is automatically calculated by the USPTO at the time of patent issuance and included in the Issue Notification Letter. Without direct access to the patent's file wrapper in Patent Center, I cannot definitively state the specific PTA granted to US10218995.
• Patent Term Extension (PTE): PTE is available for patents claiming products that require regulatory approval (e.g., pharmaceuticals, medical devices) to restore patent term lost during the regulatory review period. There is no indication in the provided patent text or associated information that US10218995 relates to a product requiring regulatory approval under 35 U.S.C. § 156. Therefore, it is highly unlikely to have received a PTE.
• Continuation and Divisional Applications:
  • A continuation application claims the same invention as a prior-filed, copending nonprovisional application, maintaining the original filing date, and does not introduce new subject matter.
  • A divisional application results from a requirement by the USPTO to restrict an application to one invention when two or more independent and distinct inventions are claimed. It also benefits from the filing date of the original application.
  • To identify specific continuation or divisional applications, one would typically examine the "Related U.S. Application Data" section of the patent document, often found on the front page or in the detailed history within Patent Center. Without direct access to this section for US10218995, I cannot list any specific continuation or divisional applications.
• Related Family Members: Patent family members generally include patents and applications filed in different countries that share a common priority claim. Google Patents lists "US20150229942A1" as another version of the patent, which is likely a related published application.
• Projected Expiration Date: Google Patents states that the patent is "Active, expires 2029-08-03". This expiration date already accounts for any PTA that may have been granted. The standard patent term is 20 years from the earliest filing date for which a benefit is claimed. The priority date for US10218995 is 2008-05-30, and the filing date is 2015-04-21. The expiration date of 2029-08-03 suggests that some PTA has been added to the standard 20-year term from the priority date.

Defensive Disclosure Document for US Patent 10,218,995

This document describes derivative works and technical disclosures based on US Patent 10,218,995, aimed at establishing prior art to render future incremental improvements by competitors obvious or non-novel. The derivations are structured around independent Claims 1 (Moving Picture Encoding System), 15 (Moving Picture Decoding System), and 19 (Moving Picture Reencoding System), applying five distinct axes of variation. The principles detailed for these system claims are equally applicable to their corresponding method (Claims 13, 17, 21) and program (Claims 14, 18, 22) claims.

---

Derivatives of Independent Claim 1: Moving Picture Encoding System

Claim 1 describes an encoding system including a first encoder (standard resolution encoding/decoding), a first super-resolution enlarger (standard to higher resolution), a first resolution converter (higher to standard resolution), and a second encoder (using super-resolution enlarged and converted pictures as target, decoded pictures from first encoder as reference).

Derivative 1.1: FPGA/ASIC-Accelerated Super-Resolution Encoding System with CNN-based Super-Resolution

Enabling Description:
This derivative system implements the super-resolution enlargement and resolution conversion processes using dedicated hardware accelerators, specifically Field-Programmable Gate Arrays (FPGAs) or Application-Specific Integrated Circuits (ASICs), for enhanced computational efficiency and real-time processing capabilities. The first super-resolution enlarger (e.g., 103 in US10218995B2) is realized by a Convolutional Neural Network (CNN) inference engine optimized for FPGA/ASIC deployment. This engine utilizes a pre-trained super-resolution CNN architecture, such as a Super-Resolution Convolutional Neural Network (SRCNN) for direct end-to-end mapping, or an Enhanced Deep Super-Resolution Network (EDSR) for higher quality with increased computational complexity. The CNN operates on the input sequence of moving pictures with a standard resolution to generate super-resolution enlarged pictures by inferring high-frequency details. The first resolution converter (e.g., 104 in US10218995B2) is implemented as a fixed-function hardware block performing bicubic downsampling or a frequency-domain low-pass filter (e.g., using a 2D-FFT and inverse 2D-FFT with frequency cutoff) to accurately convert the CNN-generated high-resolution pictures back to the standard resolution, creating super-resolution enlarged and converted pictures. Both the first encoder (e.g., 102 in US10218995B2) and second encoder (e.g., 107 in US10218995B2) are also implemented as hardware codecs (e.g., H.264/AVC or H.265/HEVC cores) integrated within the same FPGA/ASIC, capable of parallel processing streams. The inter-layer prediction data exchange between the two encoders is facilitated by high-speed on-chip memory interfaces (e.g., AXI-Stream).

graph TD
    A[Standard Resolution Input Moving Pictures] --> B{FPGA/ASIC Super-Resolution System}
    B -- Subsequence --> C[First Encoder (Hardware Codec)]
    C -- Decoded Pictures (Std Res) --> G[Second Encoder (Hardware Codec)]
    B -- Subsequence --> D[CNN-based Super-Resolution Engine (FPGA/ASIC)]
    D -- Super-Resolution Enlarged Pictures (High Res) --> E[Resolution Converter (Hardware Filter)]
    E -- Super-Resolution Enlarged and Converted Pictures (Std Res) --> F[Second Encoder (Hardware Codec)]
    F -- Encoding Target Pictures --> G
    C -- First Sequence of Encoded Bits --> H[Multiplexer (Hardware)]
    G -- Second Sequence of Encoded Bits --> H
    H --> I[Output Encoded Bitstream]

Derivative 1.2: Ultra-High Frame Rate (UHFR) and High Dynamic Range (HDR) Super-Resolution Encoding for Scientific Imaging

Enabling Description:
This system is specialized for encoding moving picture sequences with extreme operational parameters, specifically Ultra-High Frame Rate (UHFR) video (e.g., 1,000 to 100,000 frames per second) and High Dynamic Range (HDR) content (e.g., 12-bit or 14-bit per color channel), prevalent in scientific imaging applications such as particle physics, combustion analysis, or ballistic studies. The first super-resolution enlarger (e.g., 103) employs a temporal super-resolution algorithm in addition to spatial super-resolution. This involves aggregating information from multiple temporally adjacent observation pictures (frames) to reconstruct a single high-resolution frame, effectively increasing both spatial detail and mitigating motion blur inherent in UHFR acquisition. For HDR content, the super-resolution and resolution conversion stages operate in a perceptually uniform color space (e.g., PQ, HLG, or log-luminance encoding) to prevent quantization artifacts in high-brightness areas. The first encoder (e.g., 102) and second encoder (e.g., 107) are adapted to handle UHFR/HDR metadata and extended color gamuts, utilizing coding standards like H.265/HEVC Main 10 or VVC (H.266) to efficiently compress the increased data volume. The second encoder leverages the standard-resolution decoded pictures and the super-resolution enlarged and converted pictures as references, where the super-resolution branch specifically enriches the scene with recovered temporal and spatial texture information crucial for scientific analysis.

graph TD
    A[UHFR/HDR Raw Input (1000+ fps, 12-bit+)] --> B{Specialized Input Buffer}
    B -- Chunked Frames --> C[First Super-Resolution Enlargement (Spatial + Temporal SR, HDR-aware)]
    C -- High-Res UHFR/HDR Frames --> D[First Resolution Converter (HDR-aware Downsampling)]
    D -- Std-Res UHFR/HDR Frames --> E[Second Encoder (H.265/VVC Main 10+)]
    B -- Chunked Frames --> F[First Encoder (H.265/VVC Main 10+)]
    F -- Decoded Std-Res UHFR/HDR Frames --> G[Second Encoder (H.265/VVC Main 10+)]
    E -- Encoding Target --> G
    F -- First Encoded Bitstream --> H[Multiplexer]
    G -- Second Encoded Bitstream --> H
    H --> I[UHFR/HDR Super-Res Encoded Output]

Derivative 1.3.1: Cross-Domain Application: Real-time Super-Resolution Encoding for Telemedicine Diagnostics

Enabling Description:
This system is configured for real-time moving picture encoding in telemedicine applications, particularly for remote diagnostic procedures involving low-resolution medical video feeds (e.g., endoscopic procedures, ultrasound scans, dermatoscopy). A common challenge in telemedicine is transmitting sufficient diagnostic detail over varying network bandwidths. The first encoder (e.g., 102) encodes the raw, standard-resolution (e.g., 720p) medical video for baseline transmission. Simultaneously, the first super-resolution enlarger (e.g., 103) applies a super-resolution process to the original standard-resolution medical video, specifically optimized for enhancing fine anatomical structures, tissue textures, or lesion boundaries. The super-resolution model might be pre-trained on large datasets of medical imagery. The resulting super-resolution enlarged pictures (e.g., 1080p or 4K) are then fed to the first resolution converter (e.g., 104), which downconverts them back to the standard resolution for efficient processing, preserving the enhanced high-frequency information within the lower resolution. The second encoder (e.g., 107) then encodes these super-resolution enlarged and converted pictures using the standard-resolution decoded pictures from the first encoder as reference. This hierarchical encoding creates a two-layer stream: a base layer (standard resolution, sufficient for general viewing) and an enhancement layer (carrying the super-resolution derived details, crucial for precise diagnosis). This allows a remote clinician to dynamically request or receive the enhanced layer, revealing finer details (e.g., vessel patterns, cell morphology) crucial for accurate real-time diagnosis, even if the original capture resolution was limited.

graph TD
    A[Low-Res Medical Video Input (e.g., Endoscope)] --> B{Medical Video Encoding System}
    B -- Original Stream --> C[First Encoder (Base Layer Encoding)]
    C -- Decoded Base Layer (Std Res) --> F[Second Encoder (Enhancement Layer Encoding)]
    B -- Original Stream --> D[Medical SR Enlargement (e.g., Texture/Boundary Enhancement)]
    D -- SR Enlarged Medical Pictures (High Res) --> E[Resolution Converter (Std Res Output)]
    E -- SR Enlarged & Converted (Std Res) --> F
    F -- Encoding Target --> F
    C -- Base Layer Bitstream --> G[Multiplexer]
    F -- Enhancement Layer Bitstream --> G
    G --> H[Multiplexed SR Medical Video Stream]

Derivative 1.3.2: Cross-Domain Application: Adaptive Super-Resolution Encoding for Geospatial Intelligence (Satellite/Aerial Surveillance)

Enabling Description:
This system applies moving picture encoding principles to geospatial intelligence, specifically for processing and transmitting satellite or aerial surveillance video feeds where bandwidth and storage are critical constraints. Earth observation satellites and drones capture vast amounts of imagery, often at lower resolutions for wide-area coverage, but requiring high detail for specific targets. The first encoder (e.g., 102) compresses the raw, standard-resolution (e.g., 720p wide-area scan) video feed. Simultaneously, the first super-resolution enlarger (e.g., 103) processes the original input, focusing on specific regions of interest (ROIs) identified by an object detection algorithm (e.g., identifying vehicles, infrastructure). For these ROIs, an advanced super-resolution algorithm, potentially leveraging sparse representation learning or generative adversarial networks (GANs) trained on high-resolution ground truth, reconstructs finer details from the lower-resolution input frames. The first resolution converter (e.g., 104) then downconverts these super-resolved ROIs back to the standard resolution, effectively embedding enhanced target details within a standard-resolution frame. The second encoder (e.g., 107) encodes these super-resolution enhanced ROI frames, using the base-layer decoded frames as reference. This creates a multi-layered stream where a base layer provides general situational awareness, and an enhancement layer (derived via SR) provides actionable intelligence for specific targets with improved detail, allowing analysts to zoom into targets without suffering from traditional upscaling artifacts, critical for reconnaissance and anomaly detection.

graph TD
    A[Satellite/Aerial Video Input (Std Res)] --> B{Geospatial Encoding System}
    B -- Original Stream --> C[First Encoder (Base Layer)]
    C -- Decoded Base Layer (Std Res) --> G[Second Encoder (Enhancement Layer)]
    B -- Original Stream --> D[Super-Resolution Enlargement (ROI-focused, GAN-based)]
    D -- SR Enlarged ROIs (High Res) --> E[Resolution Converter (Std Res Output)]
    E -- SR Enlarged & Converted (Std Res) --> G
    G -- Encoding Target --> G
    C -- Base Layer Bitstream --> H[Multiplexer]
    G -- Enhancement Layer Bitstream --> H
    H --> I[Multiplexed SR Geospatial Stream]

Derivative 1.3.3: Cross-Domain Application: Predictive Maintenance via Super-Resolution Encoding of Industrial Sensor Video

Enabling Description:
This system is tailored for industrial predictive maintenance, where video streams from surveillance cameras monitoring machinery are used to detect early signs of wear, defects, or anomalies. Often, these industrial cameras operate at standard or even lower resolutions to minimize storage and transmission costs. The first encoder (e.g., 102) processes the standard-resolution video feed of machinery components (e.g., bearings, gears, conveyor belts) for baseline monitoring. Concurrently, the first super-resolution enlarger (e.g., 103) applies a super-resolution technique to the input video, focusing on enhancing subtle visual cues indicative of potential failures, such as micro-cracks, surface pitting, unusual vibrations, or oil leaks. This SR process may employ optical flow for motion compensation between frames and a robust SR algorithm to reconstruct sharper details. The first resolution converter (e.g., 104) converts the super-resolution enlarged images back to standard resolution, effectively embedding critical high-frequency information about component integrity within a manageable data rate. The second encoder (e.g., 107) encodes these SR-enhanced frames, referencing the base-layer decoded frames. The resulting dual-layer stream allows for continuous, low-bandwidth monitoring with a base layer, while the enhancement layer provides diagnostically superior images for automated anomaly detection algorithms or human inspection, facilitating early intervention and preventing costly equipment failures.

graph TD
    A[Industrial Sensor Video (Std Res)] --> B{Predictive Maintenance Encoding System}
    B -- Original Stream --> C[First Encoder (Baseline Monitoring)]
    C -- Decoded Baseline (Std Res) --> G[Second Encoder (Anomaly Enhancement)]
    B -- Original Stream --> D[SR Enlargement (Defect/Wear-focused)]
    D -- SR Enlarged Features (High Res) --> E[Resolution Converter (Std Res Output)]
    E -- SR Enlarged & Converted (Std Res) --> G
    G -- Encoding Target --> G
    C -- Baseline Bitstream --> H[Multiplexer]
    G -- Anomaly Enhancement Bitstream --> H
    H --> I[Multiplexed SR Industrial Stream]

Derivative 1.4: Integration with Emerging Tech: AI-Optimized, IoT-Controlled Super-Resolution Encoding with Distributed Blockchain Verification

Enabling Description:
This derivative integrates AI, IoT, and blockchain for intelligent and verified moving picture encoding. An IoT sensor network (e.g., environmental sensors, network performance monitors) provides real-time contextual data. An AI-driven optimization module (e.g., a reinforcement learning agent) dynamically adjusts the parameters of the first super-resolution enlarger (e.g., 103) and first resolution converter (e.g., 104), as well as the rate control of the first encoder (e.g., 102) and second encoder (e.g., 107). This AI considers input content characteristics, available network bandwidth (from IoT), and computational resources to optimize the super-resolution model selection (e.g., switching between fast, lower-quality SRCNN and slower, higher-quality EDSR) and encoding bitrates to achieve a target quality-of-experience or minimize latency. For instance, if IoT sensors detect high network congestion, the AI may reduce the SR upscale factor or prioritize temporal over spatial SR. Furthermore, a blockchain verification module generates cryptographic hashes of segments of the first sequence of encoded bits and second sequence of encoded bits, along with associated encoding parameters and AI/IoT configuration metadata. These hashes are recorded on an immutable distributed ledger (blockchain). This ensures the integrity and verifiable provenance of the super-resolution enhanced video stream, guaranteeing that the content and its processing history (including AI decisions) can be audited and trusted across a supply chain or distributed network.

graph TD
    subgraph AI-Optimized Encoding
        A[Standard Res Input] --> B{AI-driven Optimization Module}
        B -- Parameters --> C[First Encoder]
        C -- Decoded Pictures --> H[Second Encoder]
        B -- Parameters --> D[First SR Enlargement]
        D -- High Res Pictures --> E[First Resolution Converter]
        E -- Std Res Pictures --> H
        H -- Target/Reference --> H
        C -- Encoded Bits 1 --> I[Multiplexer]
        H -- Encoded Bits 2 --> I
    end
    subgraph IoT Control
        J[IoT Sensor Network] -- Real-time Data --> B
    end
    subgraph Blockchain Verification
        I -- Encoded Bitstream Segments + Metadata --> K[Blockchain Verification Module]
        K -- Hashes --> L[Distributed Ledger (Blockchain)]
    end
    I --> M[Output Encoded Bitstream]

Derivative 1.5: The "Inverse" or Failure Mode: Resilient Super-Resolution Encoding with Progressive Degradation for Bandwidth-Constrained Environments

Enabling Description:
This system is designed for resilient operation in highly variable or constrained bandwidth environments, or when computational resources are limited, enabling a graceful degradation of super-resolution and encoding quality rather than outright failure. A resource monitoring unit continuously assesses available network bandwidth, CPU/GPU load, and memory usage. The first super-resolution enlarger (e.g., 103) is configured with multiple super-resolution models or scaling factors, ranging from aggressive high-quality/high-compute models to lightweight, faster models (or even simple interpolation). Similarly, the first resolution converter (e.g., 104) can adjust its filtering characteristics. The system prioritizes regions of interest (ROIs) within the moving pictures (e.g., detected faces, text, or primary subjects) for higher super-resolution fidelity. During bandwidth scarcity or high load, the resource monitoring unit instructs the super-resolution enlarger to switch to a lower computational complexity SR model or reduce the spatial upscale factor. Non-ROI areas may receive minimal or no super-resolution processing. Concurrently, the first encoder (e.g., 102) and second encoder (e.g., 107) dynamically adjust quantization parameters and reference picture structures to reduce bitrate, maintaining critical base-layer information at the expense of enhancement layer quality or overall fidelity. This progressive degradation ensures continuous, albeit adaptively scaled, video delivery with super-resolution benefits prioritized for essential content. In extreme conditions, the first super-resolution enlarger and first resolution converter may be bypassed entirely, and the second encoder operates purely on the original standard-resolution stream using the decoded base layer as a reference, effectively falling back to a non-SR, dual-layer encoding mode.

stateDiagram
    direction LR
    Idle --> Monitoring: Start Encoding
    Monitoring --> Optimal_SR_Encoding: High Resources
    Monitoring --> Degraded_SR_Encoding: Medium Resources / Bandwidth Constraint
    Monitoring --> Baseline_Encoding_Fallback: Low Resources / Severe Bandwidth
    Optimal_SR_Encoding --> Monitoring: Continue
    Degraded_SR_Encoding --> Monitoring: Continue
    Baseline_Encoding_Fallback --> Monitoring: Continue
    
    state Optimal_SR_Encoding {
        High_Quality_SR --> High_Bitrate_Encoding
    }
    state Degraded_SR_Encoding {
        Low_Compute_SR --> Adaptive_Bitrate_Encoding
        Low_Compute_SR --> ROI_Prioritized_SR
    }
    state Baseline_Encoding_Fallback {
        SR_Bypass --> Standard_Dual_Layer_Encoding
    }

---

Derivatives of Independent Claim 15: Moving Picture Decoding System

Claim 15 describes a decoding system including a demultiplexer, a first decoder (standard resolution), a first super-resolution enlarger (standard decoded to higher resolution), and a first resolution converter (higher to standard resolution).

Derivative 2.1: GPU-Accelerated Real-time Super-Resolution Decoding for Immersive Displays

Enabling Description:
This derivative system is designed for high-performance, real-time decoding of super-resolution enhanced video streams for immersive display technologies (e.g., large-format displays, virtual reality headsets). The demultiplexer (e.g., as in US10218995B2) separates the incoming multiplexed bitstream into standard-resolution encoded bits and associated enhancement layer bits. The first decoder (e.g., as in US10218995B2) decodes the standard-resolution layer. The first super-resolution enlarger (e.g., as in US10218995B2) and first resolution converter (e.g., as in US10218995B2) functionalities are offloaded entirely to a high-performance Graphics Processing Unit (GPU). The GPU leverages massively parallel processing (e.g., CUDA or OpenCL kernels) to execute advanced super-resolution algorithms (e.g., deep learning-based upscalers like ESRGAN or SwinIR for superior visual quality) on the decoded standard-resolution pictures. This allows for real-time reconstruction of super-resolution enlarged decoded pictures at high fidelity and resolution (ee.g., 4K or 8K) required for immersive viewing, which are then passed to the GPU-accelerated resolution converter for final processing if needed, before rendering. The entire pipeline from bitstream parsing to pixel rendering is optimized for minimum latency using GPU direct memory access (DMA) and shared memory architectures.

graph TD
    A[Input Encoded Bitstream] --> B[Demultiplexer]
    B -- Std Res Encoded Bits --> C[First Decoder (CPU/Dedicated HW)]
    C -- Std Res Decoded Pictures --> D{GPU Processing Unit}
    D -- Render Buffer --> E[Immersive Display]
    subgraph GPU Processing Unit
        D1[GPU-accelerated First Super-Resolution Enlargement (e.g., ESRGAN)] --> D2[GPU-accelerated First Resolution Converter]
    end
    D -- Std Res Decoded Pictures (Input to SR) --> D1
    D2 -- Super-Resolution Decoded Pictures (Std Res) --> D
    style D fill:#f9f,stroke:#333,stroke-width:2px

Derivative 2.2: Decoding of Volumetric Super-Resolution Video Streams for 3D Visualization

Enabling Description:
This derivative extends the moving picture decoding system to handle volumetric video data, such as those used in light field displays, holographic projections, or medical volumetric rendering. Instead of 2D moving pictures, the input bitstream contains encoded representations of 3D light fields or voxel grids, where standard resolution refers to a baseline volumetric sampling density. The demultiplexer (e.g., as in US10218995B2) parses the volumetric bitstream, separating base-layer volumetric data from super-resolution enhancement data. The first decoder (e.g., as in US10218995B2) reconstructs the standard resolution (e.g., 128x128x128 voxel) volumetric data. The first super-resolution enlarger (e.g., as in US10218995B2) is adapted to perform 3D volumetric super-resolution. This involves using 3D convolutional neural networks or multi-plane image (MPI) synthesis techniques to infer higher-resolution volumetric details (e.g., 256x256x256 voxel or higher density light field samples) from the decoded standard resolution volumetric data. This creates super-resolution enlarged decoded volumetric pictures. The first resolution converter (e.g., as in US10218995B2) then downsamples the high-resolution volumetric data to a desired output standard resolution (which may still be higher than the input base layer for specific 3D displays, or a virtual 2D slice). The system is crucial for enabling high-fidelity interactive 3D visualization from compressed volumetric streams, for example, in surgical planning or architectural walkthroughs.

graph TD
    A[Encoded Volumetric Bitstream] --> B[Volumetric Demultiplexer]
    B -- Std Res Volumetric Data --> C[First Volumetric Decoder]
    C -- Std Res Decoded Volume --> D[First 3D Super-Resolution Enlargement]
    D -- High Res Decoded Volume --> E[First Volumetric Resolution Converter]
    E -- Std Res Super-Resolution Decoded Volume --> F[3D Display/Renderer]

Derivative 2.3: Cross-Domain Application: Low-Latency Super-Resolution Decoding for Immersive Augmented Reality

Enabling Description:
This moving picture decoding system is optimized for low-latency operation in immersive Augmented Reality (AR) applications, particularly when AR content is rendered remotely and streamed to a head-mounted display (HMD). To minimize bandwidth while maintaining perceived quality, a remote server might encode lower-resolution video frames corresponding to the user's field of view (FOV). The demultiplexer (e.g., as in US10218995B2) receives this stream. The first decoder (e.g., as in US10218995B2) decodes the baseline standard resolution AR frames. Crucially, the first super-resolution enlarger (e.g., as in US10218995B2) employs a lightweight, high-speed super-resolution algorithm (e.g., a shallow CNN or optimized bicubic upscaler with sharpening) specifically tailored to run on the HMD's onboard processor or a mobile companion device. This SR process dynamically prioritizes regions within the user's foveal (central vision) area, applying higher quality super-resolution to those pixels, while peripheral areas receive minimal or no SR, or a computationally cheaper variant. This "foveated super-resolution" reduces overall computational load and latency. The first resolution converter (e.g., as in US10218995B2) then scales these selectively super-resolved images to the native resolution of the HMD. This allows for perceived high-resolution AR experiences from bandwidth-efficient streams, maintaining interactivity and minimizing motion-to-photon latency, which is critical for user comfort and immersion.

graph TD
    A[Encoded AR Stream (Low Res, Foveated)] --> B[Demultiplexer]
    B -- Encoded Std Res AR Frames --> C[First Decoder (on HMD)]
    C -- Std Res Decoded AR Frames --> D[Foveated Super-Resolution Enlargement (on HMD)]
    D -- High Res AR Frames (Foveated SR) --> E[Resolution Converter (on HMD)]
    E -- HMD Native Res AR Frames --> F[AR Head-Mounted Display]

Derivative 2.4: Integration with Emerging Tech: Edge-AI Enhanced Super-Resolution Decoding for Smart Devices

Enabling Description:
This moving picture decoding system is implemented on smart devices (e.g., smartphones, smart cameras, IoT-enabled screens) leveraging on-device Edge-AI capabilities. The demultiplexer (e.g., as in US10218995B2) and first decoder (e.g., as in US10218995B2) function as usual, decoding standard-resolution video streams. The first super-resolution enlarger (e.g., as in US10218995B2) and first resolution converter (e.g., as in US10218995B2) are realized by highly optimized, quantized deep learning models (e.g., MobileNetV2-based SR, or specialized neural network accelerators like Google's Edge TPU). These models perform super-resolution inference directly on the edge device, using the decoded standard resolution pictures as input. This Edge-AI approach reduces reliance on cloud processing, minimizes data transfer, and significantly lowers latency, making super-resolution enhancement suitable for applications where instant visual feedback is required (e.g., smart home security cameras upscaling live feeds, mobile video playback enhancing low-resolution content). The models are designed for low power consumption and high inference speed on resource-constrained hardware, potentially adapting the SR model dynamically based on battery life or computational load.

graph TD
    A[Input Encoded Bitstream] --> B[Demultiplexer (Smart Device)]
    B -- Std Res Encoded Bits --> C[First Decoder (Smart Device CPU)]
    C -- Std Res Decoded Pictures --> D{Edge-AI Super-Resolution Module (Smart Device NPU/GPU)}
    D -- High Res Output --> E[Smart Device Display/Output]
    subgraph Edge-AI Super-Resolution Module
        D1[Optimized SR-CNN Inference] --> D2[Quantized Resolution Converter]
    end
    C -- Std Res Decoded Pictures (Input to SR) --> D1
    D2 -- Super-Resolution Decoded Pictures (Std Res) --> D
    style D fill:#f9f,stroke:#333,stroke-width:2px

Derivative 2.5: The "Inverse" or Failure Mode: Bandwidth-Adaptive Super-Resolution Decoding with Content-Aware Skipping

Enabling Description:
This moving picture decoding system is designed for robust operation under adverse network conditions, prioritizing critical visual information when bandwidth is insufficient or processing power is overloaded. A network/resource monitor continuously assesses available bandwidth and decoding device capabilities. The demultiplexer (e.g., as in US10218995B2) intelligently drops or selectively decodes parts of the enhancement layer bitstream based on predefined content-aware metrics (e.g., motion vectors indicating activity, saliency maps highlighting important regions). The first decoder (e.g., as in US10218995B2) decodes the base layer. If network bandwidth drops significantly or processing becomes constrained, the first super-resolution enlarger (e.g., as in US10218995B2) switches to a lower-quality, faster super-resolution mode (e.g., bilinear upscaling instead of deep learning SR) or applies SR only to detected regions of interest. For non-critical pictures or regions, the system may skip the super-resolution enlargement process entirely, or even downscale the decoded base layer to conserve resources. The first resolution converter (e.g., as in US10218995B2) similarly adapts its conversion parameters. The goal is to maintain a continuous, albeit adaptively degraded, visual experience rather than buffering or freezing. For example, during video conferencing, the system would prioritize SR for speaker faces while using minimal processing for background elements, ensuring crucial visual communication persists through network fluctuations.

stateDiagram
    direction LR
    Idle --> Start_Decoding: Input Bitstream
    Start_Decoding --> Monitoring_Resources: Initialize
    
    Monitoring_Resources --> Full_SR_Decode: High Bandwidth/Resources
    Monitoring_Resources --> Adaptive_SR_Decode: Moderate Bandwidth/Resources
    Monitoring_Resources --> Base_Layer_Only_Decode: Low Bandwidth/Resources
    
    state Full_SR_Decode {
        Demux_Full --> First_Decode_Full --> Full_SR_Enlarge --> Full_Res_Convert --> Render_Full
    }
    
    state Adaptive_SR_Decode {
        Demux_Partial_Enhancement --> First_Decode_Adaptive --> Selective_SR_Enlarge --> Adaptive_Res_Convert --> Render_Adaptive
    }
    
    state Base_Layer_Only_Decode {
        Demux_Base_Only --> First_Decode_Base --> Render_Base
    }
    
    Full_SR_Decode --> Monitoring_Resources: Continue
    Adaptive_SR_Decode --> Monitoring_Resources: Continue
    Base_Layer_Only_Decode --> Monitoring_Resources: Continue

---

Derivatives of Independent Claim 19: Moving Picture Reencoding System

Claim 19 describes a reencoding system including a demultiplexer, a decoder (standard resolution), a first super-resolution enlarger (standard decoded to higher resolution), and a first resolution converter (higher to standard resolution). It is "adapted to input thus encoded moving pictures to decode and reencode," implying a reencoder stage using the processed data.

Derivative 3.1: Heterogeneous Compute Reencoding System with DNN-based Transcoding

Enabling Description:
This derivative moving picture reencoding system utilizes a heterogeneous computing architecture, distributing the workload across various processing units for optimal performance during transcoding and format conversion. The demultiplexer (e.g., as in US10218995B2) and initial decoder (e.g., as in US10218995B2) might run on specialized hardware decoder blocks (e.g., ASIC decoder) for efficient stream parsing and standard-resolution frame extraction. The first super-resolution enlarger (e.g., as in US10218995B2) is implemented on a high-performance GPU, employing deep neural networks (DNNs) specifically trained for super-resolution and potentially denoising/deblocking. The first resolution converter (e.g., as in US10218995B2) also leverages GPU shaders for high-quality downsampling and filtering. The subsequent reencoder stage (not explicitly detailed in the provided claim summary but implicit in "reencoding system") integrates a DNN-based transcoder on an AI accelerator (e.g., Tensor Processing Unit - TPU or dedicated NPU). This DNN transcoder performs content-aware rate control, perceptual quality optimization, and adaptive parameter selection for the new encoding process. It can dynamically choose optimal encoding presets (e.g., for H.264, H.265, AV1) based on the input content and target distribution platform, ensuring maximum quality for a given bitrate constraint, leveraging the super-resolution enhanced input to generate perceptually superior reencoded output.

graph TD
    A[Input Encoded Bitstream] --> B[Demultiplexer (ASIC)]
    B -- Encoded Bits --> C[Decoder (ASIC)]
    C -- Decoded Std Res Pictures --> D{Heterogeneous Compute Cluster}
    subgraph Heterogeneous Compute Cluster
        D1[GPU-accelerated First Super-Resolution Enlargement (DNN)] --> D2[GPU-accelerated First Resolution Converter]
        D2 -- Processed Pictures --> D3[AI Accelerator (TPU/NPU) - DNN Transcoder]
    end
    D3 -- Reencoded Bitstream --> E[Output Reencoded Bitstream]
    style D fill:#f9f,stroke:#333,stroke-width:2px

Derivative 3.2: Multi-View Super-Resolution Reencoding for 360-degree Video Streaming

Enabling Description:
This moving picture reencoding system is designed for processing and optimizing 360-degree or multi-view video content, common in virtual reality (VR) and immersive media. Input consists of multiple synchronized video streams (e.g., an equirectangular projection or multiple fisheye camera feeds). The demultiplexer (e.g., as in US10218995B2) and decoder (e.g., as in US10218995B2) handle each individual stream at standard resolution. The first super-resolution enlarger (e.g., as in US10218995B2) is adapted for multi-view super-resolution. This involves a spatial-temporal super-resolution algorithm that not only enhances individual frames but also leverages redundancy and coherence across adjacent views and temporal frames to reconstruct a higher-resolution, consistent 360-degree panorama. This might involve projecting frames into a 3D space, performing SR, and then re-projecting. The first resolution converter (e.g., as in US10218995B2) then converts this super-resolved 360-degree representation back to a standard resolution (e.g., a higher-quality equirectangular format than the original input). The subsequent reencoder (e.g., a specialized 360-degree video encoder like HEVC-360) compresses this super-resolved 360-degree content into a format suitable for adaptive streaming (e.g., view-dependent tiling). This derivative significantly improves the perceived quality of 360-degree video, especially for zoomed-in areas, which are typically low-resolution due to wide-angle capture.

graph TD
    A[Multi-View Encoded Bitstream (360 Video)] --> B[Demultiplexer (Per View)]
    B -- Individual View Streams --> C[Decoder Array (Per View, Std Res)]
    C -- Decoded Std Res Views --> D{Multi-View Super-Resolution Processor}
    subgraph Multi-View Super-Resolution Processor
        D1[Spatial-Temporal Multi-View SR Enlargement] --> D2[360 Resolution Converter]
    end
    D -- Super-Resolution 360 Video --> E[360 Video Reencoder (e.g., HEVC-360)]
    E --> F[Reencoded 360 Bitstream]
    style D fill:#f9f,stroke:#333,stroke-width:2px

Derivative 3.3: Cross-Domain Application: Preservation Reencoding with Perceptual Super-Resolution for Legacy Media Archiving

Enabling Description:
This moving picture reencoding system is tailored for digital archiving and preservation of legacy media, such as analog film, VHS tapes, or early digital video formats, which often suffer from low resolution, noise, and degradation. The system aims to perceptually enhance and digitize these assets into modern, robust formats. The demultiplexer (if applicable for digital inputs) and decoder (e.g., as in US10218995B2) process the source material (e.g., converted analog-to-digital or legacy digital files). The first super-resolution enlarger (e.g., as in US10218995B2) employs advanced perceptual super-resolution algorithms (e.g., GAN-based SR models trained on high-quality archival data) combined with sophisticated denoising, deinterlacing, and color restoration techniques. This process reconstructs super-resolution enlarged decoded pictures that appear to have significantly higher detail and clarity than the original, removing artifacts and inferring lost information. The first resolution converter (e.g., as in US10218995B2) then scales these enhanced pictures to a standard resolution suitable for modern digital archives (e.g., 1080p or 4K archive master), ensuring frequency components within this target resolution are optimally represented. The subsequent reencoder (e.g., using lossless or high-bitrate visually lossless codecs like JPEG 2000, ProRes, or FFV1) archives this perceptually enhanced content, future-proofing legacy media by capturing its "best possible" visual state.

graph TD
    A[Legacy Media Input (Analog/Low-Res Digital)] --> B[Digitizer/Demultiplexer]
    B -- Decoded Std Res Pictures --> C{Perceptual Enhancement & Reencoding}
    subgraph Perceptual Enhancement & Reencoding
        C1[Advanced SR Enlargement (GAN-based, Denoising, Color Restore)] --> C2[Resolution Converter (Archive Target Res)]
        C2 -- Perceptually Enhanced Pictures --> C3[Archival Reencoder (Lossless/Visually Lossless)]
    end
    C3 --> D[Archival Master Output (High Res, Modern Codec)]
    style C fill:#f9f,stroke:#333,stroke-width:2px

Derivative 3.4: Integration with Emerging Tech: Cloud-Native Serverless Super-Resolution Reencoding with Microservices Architecture

Enabling Description:
This moving picture reencoding system is implemented as a cloud-native, serverless application utilizing a microservices architecture, providing on-demand scalability and cost efficiency for video processing. Each functional block of the reencoding system – demultiplexing, decoding, super-resolution enlargement, resolution conversion, and reencoding – is deployed as an independent serverless function (e.g., AWS Lambda, Google Cloud Functions, Azure Functions) or a containerized microservice (e.g., Kubernetes pods). Input encoded bits are ingested into an object storage bucket (e.g., S3). An event trigger invokes the demultiplexing microservice, which then passes intermediate data to other microservices via message queues (e.g., Kafka, SQS) or temporary storage. The first super-resolution enlarger (e.g., as in US10218995B2) and first resolution converter (e.g., as in US10218995B2) microservices execute on transient, automatically scaled compute instances, potentially using specialized cloud GPUs for SR inference. The reencoder microservice (e.g., FFmpeg container) dynamically adjusts its resources based on workload. This architecture allows for massive parallel processing of multiple reencoding jobs, automatic scaling to meet demand spikes, and pay-per-execution cost models, making it ideal for large-scale content libraries requiring various output formats and resolutions.

graph TD
    A[Input Encoded Bitstream (Object Storage)] --> B(Event Trigger)
    B --> C[Demultiplexer Microservice]
    C --> D[Message Queue/Storage]
    D --> E[Decoder Microservice]
    E --> F[Message Queue/Storage]
    F --> G[SR Enlargement Microservice (Cloud GPU)]
    G --> H[Message Queue/Storage]
    H --> I[Resolution Converter Microservice]
    I --> J[Message Queue/Storage]
    J --> K[Reencoder Microservice]
    K --> L[Output Reencoded Bitstream (Object Storage)]

Derivative 3.5: The "Inverse" or Failure Mode: Disaster Recovery Reencoding for Corrupted Streams with AI-Assisted Reconstruction

Enabling Description:
This moving picture reencoding system is specifically designed to salvage and re-encode partially corrupted or severely degraded video streams, common in disaster recovery scenarios (e.g., damaged storage media, interrupted transmissions). The primary goal is to produce a viewable, albeit potentially imperfect, reencoded stream. The demultiplexer (e.g., as in US10218995B2) and decoder (e.g., as in US10218995B2) are equipped with robust error concealment and resilience mechanisms, attempting to decode as much valid data as possible even from corrupted segments. If blocks or frames are entirely missing or severely damaged, the first super-resolution enlarger (e.g., as in US10218995B2) integrates AI-assisted reconstruction modules. These modules (e.g., inpainting GANs, temporal prediction networks) utilize surrounding valid frames or spatial context to infer and reconstruct missing pixels or entire regions, generating super-resolution enlarged decoded pictures where corrupted data has been "filled in" or hallucinated to maintain visual coherence. The first resolution converter (e.g., as in US10218995B2) then smooths and downconverts these reconstructed images. The subsequent reencoder (e.g., a standard video codec) then re-encodes this "repaired" stream, prioritizing continuity and watchability. Metadata can be embedded in the reencoded stream to indicate areas that underwent AI-assisted reconstruction. This system ensures that even severely compromised video evidence or critical event recordings can be recovered and made accessible, rather than being discarded due to corruption.

stateDiagram
    direction LR
    Idle --> Start_Reencoding: Input Corrupted Stream
    Start_Reencoding --> Demultiplex_Corrupted: Process
    Demultiplex_Corrupted --> Decode_With_Error_Concealment: Process
    Decode_With_Error_Concealment --> Check_Corruption_Level: Evaluate
    
    Check_Corruption_Level --> AI_Reconstruction: High Corruption
    Check_Corruption_Level --> Direct_SR_Enlargement: Low Corruption
    
    state AI_Reconstruction {
        Inpainting_GAN --> Temporal_Prediction --> SR_Enlarge_Reconstructed
    }
    
    state Direct_SR_Enlargement {
        Standard_SR_Algorithm
    }
    
    SR_Enlarge_Reconstructed --> Resolution_Conversion: Process
    Standard_SR_Algorithm --> Resolution_Conversion: Process
    
    Resolution_Conversion --> Reencode_Repaired_Stream: Process
    Reencode_Repaired_Stream --> Output_Recovered_Stream: Done

---

Combination Prior Art Scenarios with Open-Source Standards

The core invention of US Patent 10,218,995 involves enhancing video encoding efficiency by integrating super-resolution processes into a hierarchical encoding framework. This concept can be combined with existing open-source video standards to create novel systems that leverage both the patent's innovations and widely adopted technologies.

Combination Prior Art 1: H.264/AVC with Super-Resolution Enhancement Layers

Scenario: A moving picture encoding system, as described in independent Claim 1 of US10218995B2, is implemented where the first encoder (e.g., 102) and second encoder (e.g., 107) are compliant with the H.264/AVC (Advanced Video Coding) standard, specifically utilizing its Scalable Video Coding (SVC) extension or a multi-layer coding approach. The first encoder generates a base layer H.264/AVC bitstream at a standard resolution. The first super-resolution enlarger (e.g., 103) and first resolution converter (e.g., 104) process the input to create a super-resolution enhanced standard-resolution signal. The second encoder then encodes this enhanced signal as an H.264/AVC enhancement layer, using the decoded base layer from the first encoder as an inter-layer reference. The multiplexer (e.g., 109) combines these two H.264/AVC layers into a single SVC-compliant bitstream. This combination demonstrates the application of the patent's super-resolution methodology within a widely used, standardized scalable video coding framework, making future similar hierarchical SR enhancements obvious.

Combination Prior Art 2: H.265/HEVC with AI-Driven Super-Resolution for Adaptive Streaming (MPEG-DASH)

Scenario: A moving picture encoding system, as described in independent Claim 1 of US10218995B2, is configured to produce multiple resolution renditions for MPEG-DASH (Dynamic Adaptive Streaming over HTTP). The first encoder (e.g., 102) generates a base layer video stream encoded with H.265/HEVC (High Efficiency Video Coding) at a standard resolution. The first super-resolution enlarger (e.g., 103) employs a pre-trained deep learning model (e.g., a lightweight SRCNN or EDSR) to create super-resolution enlarged pictures which are then processed by the first resolution converter (e.g., 104) back to a standard resolution. The second encoder (e.g., 107) then encodes this super-resolution enhanced stream also using H.265/HEVC, leveraging inter-layer prediction from the base layer. A multiplexer (e.g., 109) packages these H.265/HEVC streams into an MPEG-DASH compliant manifest, offering both the original resolution H.265/HEVC stream and the super-resolution enhanced H.265/HEVC stream as adaptive bitrate representations. This allows clients to adaptively stream content, choosing a super-resolution enhanced version if bandwidth permits, demonstrating the patent's utility in modern adaptive streaming workflows.

Combination Prior Art 3: VP9/AV1 Codecs with Open-Source Super-Resolution Libraries

Scenario: A moving picture reencoding system, as described in independent Claim 19 of US10218995B2, is used to transcode existing video content into the VP9 or AV1 (AOMedia Video 1) open-source video codecs. The demultiplexer (e.g., as in US10218995B2) and decoder (e.g., as in US10218995B2) process an input video stream (e.g., H.264). The first super-resolution enlarger (e.g., as in US10218995B2) integrates an open-source super-resolution library (e.g., OpenCV's DNN super-resolution module leveraging SRCNN/EDSR models or a custom FFmpeg filter for SR). This module takes the standard resolution decoded pictures and applies super-resolution to generate super-resolution enlarged decoded pictures. The first resolution converter (e.g., as in US10218995B2), also implemented using open-source image processing tools (e.g., FFmpeg's scale filter), converts these back to a target standard resolution. The reencoder (e.g., libvpx for VP9 or libaom for AV1) then encodes these super-resolution enhanced pictures into a new VP9 or AV1 bitstream. This scenario directly shows how the super-resolution concept from the patent can be integrated with prevalent open-source codecs, enriching their encoding capabilities and creating demonstrably higher quality outputs for web-based video delivery.