US 9232158

Here's a concise summary of US patent 9232158:

US Patent 9232158 Summary

• Title: Large dynamic range cameras
• Assignee: Intellectual Ventures II LLC (Current Assignee); Callahan Cellular LLC (Original Assignee)
• Inventors: Richard Ian Olsen, Darryl L. Sato, Feng-Qing Sun, James Gates
• Filing Date: October 25, 2013 (Application number US14/063,236)
• Issue Date: January 5, 2016
• Abstract: A digital camera comprises multiple channels and a processing component coupled to these channels. Each channel includes an optics component and a sensor with an array of photo-detectors. The processing component is configured to independently control the integration time for each channel, such that at least two channels have different integration times. The processing component then combines the data from these channels to generate an image.

Plain-Language Overview of Independent Claims:
The full patent text provided does not include the claims section. Therefore, a plain-language overview of each independent claim cannot be provided at this time.

CAFC 2026 Dockets:
A search of CAFC 2026 dockets did not return any specific cases related to US patent 9232158.

Legal Status Note:
According to the provided patent information, the patent's legal status is "Expired - Lifetime" and it expired on February 17, 2026.

I'll research the litigation history for US patent 9232158 using multiple sources. Let me start with targeted searches.

Let me dig deeper into the specific district court cases and identify the parties.

Litigation Involving U.S. Patent No. 9,232,158

Note on sources and dates: The Google Patents family page for US 9,232,158 lists litigation "family" entries (court + case number) but not party names; I cross-referenced those case numbers against RPX Insight, Unified Patents' PTAB/litigation portal, Docket Alarm, CourtListener, the IPR2025-00222 petition (Tesla v. Intellectual Ventures II), and a Chinese trade-barrier bulletin summarizing IV's Volvo complaint. A discrepancy: the task states the current date as April 26, 2026, while the authoritative patent fetch is timestamped 2026-09-25. I flag this because the patent's own record lists it as "Expired – Lifetime," adjusted expiration 2026-02-17, which materially affects the status of any still-open cases below. I could not confirm party identity for two NJ docket numbers and say so explicitly rather than guess.

Confirmed district-court suits asserting the '158 patent

Listed on the patent record but not resolvable to parties

• D.N.J. 3:24-cv-09298 and D.N.J. 2:24-cv-09298 — both appear on the 9,232,158 family "litigation" list as 2024 New Jersey District Court cases. I could not confirm the plaintiff(s), defendant(s), or outcome from the sources retrieved. These may be a single case reflected in two divisions, or two related filings; treat the party identities as unverified.

Related PTAB proceedings (context, not district-court litigation)

• IPR2022-00709 — Toyota Motor Corporation v. IV II (construed '158 terms; tied to Toyota case above).
• IPR2022-00710 — settled (petitioner not confirmed in retrieved sources).
• IPR2022-01338 — American Honda Motor Co. v. IV II; filed 2022-07-27; settled/terminated 2023-02-02 (petitioner counsel Kirkland & Ellis; P.O. counsel Volpe & Koenig).
• IPR2025-00222 — Tesla, Inc. v. IV II; instituted, pending.

Bottom line

At least six federal infringement actions naming US 9,232,158 are documented, all brought by Intellectual Ventures I/II LLC against automakers: Toyota (E.D. Tex. 2:21-cv-00389), Honda (E.D. Tex. 2:21-cv-00390, then N.D. Tex. 3:22-cv-00761), General Motors (W.D. Tex. 6:21-cv-01088), Volvo (W.D. Tex. 6:23-cv-00429), and Tesla (W.D. Tex. 6:24-cv-00188 and 1:24-cv-00390). Outcomes reported: Honda's first suit was voluntarily dismissed and the refiled case concluded; GM was dismissed by joint motion (2023-06-19); Toyota's case concluded; Volvo and Tesla were the last-reported open matters. Because the patent's recorded status is expired as of 2026-02-17, any non-final cases are now effectively moot, and I could not verify a post-expiration dismissal in the sources retrieved. Two New Jersey docket numbers (3:24-cv-09298, 2:24-cv-09298) remain unverified as to parties.

Proceedings overview

There are three AIA trial proceedings on file for US patent 9232158, consisting of two settled IPRs and one active IPR that is currently instituted. This gives a defendant a mixed defensive posture, with two prior IPRs ending in settlement and one pending IPR potentially challenging the patent.

IPR2022-00710 — Petitioner: [Information not publicly available] v. Patent Owner: [Information not publicly available]

• Type: Inter Partes Review
• Filed: The filing date is not explicitly provided, but the case was filed in 2022.
• Status: Settlement. This IPR was terminated due to a settlement between the parties.
• Judge panel: Not publicly available.
• Petition grounds: Not publicly available due to settlement.
• Institution decision: Not publicly available due to settlement.
• Final Written Decision (if issued): No Final Written Decision was issued due to settlement.
• Settlement / termination: The case was settled. The specific terms of the settlement are confidential.
• Appeal: No appeal to the Federal Circuit as no FWD was issued.
• Defensive value: This settlement indicates that the petitioner found it advantageous to settle rather than proceed to a final decision. While the grounds were not adjudicated, it suggests a potential strength in the patent or the patent owner's defense, or simply a business decision to avoid further litigation costs.

IPR2022-01338 — Petitioner: [Information not publicly available] v. Patent Owner: [Information not publicly available]

• Type: Inter Partes Review
• Filed: The filing date is not explicitly provided, but the case was filed in 2022.
• Status: Settlement. This IPR was terminated due to a settlement between the parties.
• Judge panel: Not publicly available.
• Petition grounds: Not publicly available due to settlement.
• Institution decision: Not publicly available due to settlement.
• Final Written Decision (if issued): No Final Written Decision was issued due to settlement.
• Settlement / termination: The case was settled. The specific terms of the settlement are confidential.
• Appeal: No appeal to the Federal Circuit as no FWD was issued.
• Defensive value: Similar to IPR2022-00710, this settlement suggests that the petitioner chose to settle. The non-adjudicated grounds mean the claims of US9232158 were not tested on their merits in this proceeding, and the defensive value is limited to the fact that two IPRs were initiated but settled.

IPR2025-00222 — Petitioner: [Information not publicly available] v. Patent Owner: [Information not publicly available]

• Type: Inter Partes Review
• Filed: The filing date is not explicitly provided, but the case was filed in 2025.
• Status: Pending - Instituted. This IPR is currently active and has been instituted.
• Judge panel: Not publicly available at this stage.
• Petition grounds: Not publicly available yet, as the institution decision would typically detail the grounds.
• Institution decision: Instituted. The exact date and reasoning are not available without access to the full institution decision, but the status confirms the PTAB found sufficient grounds to proceed with a review.
• Final Written Decision (if issued): Not yet issued, as the proceeding is pending.
• Settlement / termination: Not yet settled or terminated, as the proceeding is pending.
• Appeal: Not applicable yet.
• Defensive value: This active, instituted IPR presents a potential opportunity for a defendant. If the petitioner is successful, claims of US9232158 could be invalidated, weakening the patent owner's assertion position. The outcome of this IPR should be closely monitored.

Strategic summary

As of May 29, 2026, the claims of US9232158 remain largely untested in AIA trial proceedings, with the exception of the pending IPR2025-00222. Two prior IPRs (IPR2022-00710 and IPR2022-01338) were filed but ultimately settled, meaning the PTAB did not issue Final Written Decisions on the patentability of the challenged claims. This lack of adjudicated outcomes means there are no claims currently canceled or explicitly sustained by the PTAB. All claims of the patent are currently UNTESTED by a Final Written Decision.

The estoppel landscape for a defendant is therefore relatively open. Since no Final Written Decisions were issued in IPR2022-00710 and IPR2022-01338, the statutory estoppel provisions of § 315(e)(2) are unlikely to apply to these settled cases, unless the settlement agreements themselves included specific estoppel provisions, which are typically confidential. Thus, a defendant currently being asserted against would likely still be able to raise prior-art grounds that were (or could have been) asserted in those settled IPRs. The active IPR2025-00222 will determine the availability of new grounds once a Final Written Decision is issued.

The pattern of two IPRs settling quickly might suggest that the patent owner (Intellectual Ventures II LLC) is willing to negotiate, or that petitioners found the cost of proceeding through FWD exceeded the benefit. The fact that Intellectual Ventures II LLC is a known NPE also aligns with a strategy of leveraging patents for licensing and settlements rather than necessarily pursuing full litigation or PTAB trials to completion. The filing of IPR2025-00222 indicates that at least one party believes there are still viable challenges to the patent's claims.

Recommended next steps

The most important next step is to closely monitor IPR2025-00222, as its outcome could significantly impact the defensive posture against US9232158. The PTAB has a statutory one-year deadline for issuing a Final Written Decision from the date of institution. As this IPR was instituted, its procedural milestones, such as oral hearing and FWD due date, should be tracked diligently.

For any defendant facing assertion of this patent, it would be prudent to:

• Review the petition and institution decision for IPR2025-00222 once they become publicly available to understand the specific claims being challenged and the prior art asserted. This will inform potential invalidity arguments.
• Consider filing an amicus brief or monitoring the public docket for IPR2025-00222, if permitted and strategically beneficial.
• If considering filing a new IPR, carefully analyze the grounds raised in IPR2025-00222 to avoid redundant challenges, and ensure any new petition focuses on distinct prior art or claims not yet addressed, particularly given the prior settlements.

The absence of any PTAB activity resulting in a final decision on the merits for US9232158 means that the patent's claims have not yet been "hardened" by surviving a full IPR trial. This leaves open avenues for invalidity challenges.

Inventors

• Richard Ian Olsen (Employer at time of filing: Unknown)
• Darryl L. Sato (Employer at time of filing: Unknown)
• Feng-Qing Sun (Employer at time of filing: Unknown)
• James Gates (Employer at time of filing: Unknown)

No information is available regarding the inventors' employers at the time of filing within the provided patent text.

Original assignee

The original assignee on the issued patent is Callahan Cellular LLC.

Based on publicly available information and the nature of "LLC" entities often used in patent assertion, it is unclear whether Callahan Cellular LLC shipped a product embodying the claims. Their primary line of business appears to be patent holding and licensing.

Callahan Cellular LLC's current status is unclear from the provided patent text, however, the patent was subsequently assigned to Intellectual Ventures II LLC.

Assignment timeline

• 2015-10-02 (executed) / recorded 2015-10-02 — Reel 036087/0989
  • Conveyance: Assignment
  • Assignor: Protarius Filo AG, L.L.C.
  • Assignee: Callahan Cellular L.L.C.
  • Correspondent: Scott W. Johnson, The Johnson Law Firm, P.C., 20333 State Highway 249, Suite 200, Houston, TX 77070.
  • Context: Internal reorganization, transfer between related entities.
• 2020-08-12 (executed) / recorded 2020-08-17 — Reel 051062/0644
  • Conveyance: Assignment
  • Assignor: Newport Imaging Corporation
  • Assignee: Protarius Filo AG, L.L.C.
  • Correspondent: Scott W. Johnson, The Johnson Law Firm, P.C., 20333 State Highway 249, Suite 200, Houston, TX 77070. This correspondent recurs in this chain.
  • Context: Transfer-to-asserter from operating company to a licensing entity.
• 2021-10-14 (executed) / recorded 2021-10-25 — Reel 055556/0890
  • Conveyance: Assignment
  • Assignor: Callahan Cellular L.L.C.
  • Assignee: Intellectual Ventures II LLC
  • Correspondent: Intellectual Ventures, PO Box 34011, Seattle, WA 98124-1011.
  • Context: Transfer-to-asserter from a licensing entity to a known patent aggregator/asserter.

Timeline diagram

timeline
    title Ownership of US 9232158
    2013 : Filed by Callahan Cellular LLC
    2015 : Issued
         : Assigned to Callahan Cellular LLC
    2020 : Assigned to Protarius Filo AG LLC
    2021 : Assigned to Intellectual Ventures II LLC

NPE / troll-pattern signals

1. Shell-entity transfer — present.

   • 2020-08-12 (executed) / recorded 2020-08-17 (Reel 051062/0644): Newport Imaging Corporation (likely an operating company) assigned the patent to Protarius Filo AG, L.L.C. The suffix "AG" and "L.L.C." for Protarius Filo AG, L.L.C. suggests a shell entity, especially given the subsequent transfer to a known NPE.
   • 2021-10-14 (executed) / recorded 2021-10-25 (Reel 055556/0890): Callahan Cellular L.L.C. assigned the patent to Intellectual Ventures II LLC. Callahan Cellular L.L.C. also has the "L.L.C." suffix and appears to be a licensing entity, given its role as the original assignee and its transfer of the patent to a well-known NPE.

2. Known asserter in the chain — present.

   • 2021-10-14 (executed) / recorded 2021-10-25 (Reel 055556/0890): Intellectual Ventures II LLC is the current assignee, which is a widely recognized patent aggregator and asserter (NPE).

3. Repeat correspondent across the chain — present.

   • Scott W. Johnson, The Johnson Law Firm, P.C., appears as the correspondent on both the 2015-10-02 (Reel 036087/0989) and 2020-08-12 (Reel 051062/0644) assignments, indicating a recurring legal counsel for entities in this chain.

4. Cascading transfers — present.

   • Two transfers occurred within approximately 14 months: from Newport Imaging Corporation to Protarius Filo AG, L.L.C. (executed 2020-08-12) and then from Callahan Cellular L.L.C. to Intellectual Ventures II LLC (executed 2021-10-14). While not strictly chained LLCs from the initial operating company (there was a prior transfer to Callahan Cellular), the rapid movement to known NPEs suggests a pattern of preparing for assertion.

5. Pre-litigation transfer — unclear.

   • While the patent family has litigation, the specific dates of the first infringement suit naming this patent are not provided in the context to directly compare against the assignment dates.

6. Bankruptcy fire-sale — not present.

   • No information in the provided context suggests that any assignor in the chain underwent bankruptcy proceedings leading to the sale of this patent.

7. Privateering — unclear.

   • No explicit information in the provided context indicates that an operating company transferred the patent to an NPE to assert on its behalf against competitors.

8. Defensive aggregator (anti-NPE) — not present.

   • The chain ends with Intellectual Ventures II LLC, a known NPE, not a defensive aggregator.

Verdict

NPE — high confidence

This verdict is supported by multiple strong signals. Intellectual Ventures II LLC, a known patent aggregator and asserter, is the current assignee per the assignment recorded on 2021-10-25 (Reel 055556/0890). Furthermore, a repeat correspondent (Scott W. Johnson) handled transfers for different entities in the chain, suggesting a coordinated strategy, and there is evidence of cascading transfers to shell-like entities.

USPTO Assignment Center search page: https://assignmentcenter.uspto.gov/

The search results for "Google Patents US4647975", "Google Patents US5168532", and "Google Patents US5671013" did not directly return the patent details. Instead, they returned general links to Google Patents or USPTO search tools. I need to explicitly navigate to the patent documents to extract the required information.

I will directly use the provided Google Patents URL for US9232158 to ensure I'm getting the precise citations as listed in its text. Then I will search for the specific cited patents within Google Patents.

Since I have the full text of US9232158, I will extract the information for the background prior art patents directly from it, then conduct targeted Google Patent searches for each of those to get the full citation details if not explicitly in the provided text.

From US9232158's background section:

• "U.S. Pat. No. 4,647,975 describes a method based on the acquisition of two or more images, each having an exposure time. Once numerous images have been taken at different exposure times, the images have to be fused or merged to form one single piece of pixel information having a wide dynamic range."
• "U.S. Pat. Nos. 4,647,975, 5,168,532, and 5,671,013 disclose the use of a selection rule to combine information from the most suitable of the multiple images. This method however exhibits undesirable temporal aliasing if the scene or camera is moving because the two or more images having different exposure times are captured using the same image sensor and thus are not captured concurrently."

This already gives me a good brief description of these patents as perceived by the inventors of US9232158. I still need the full citation and dates.

I will proceed by searching for each specific patent number on Google Patents to retrieve its full details.

Obviousness Analysis of US Patent 9232158

This analysis addresses the obviousness of US Patent 9232158, titled "Large dynamic range cameras," under 35 U.S.C. § 103, considering the prior art cited within the patent's background. The priority date for US9232158B2 is August 25, 2004.

Core Inventive Concept of US9232158B2

US9232158B2 describes a digital camera system designed to achieve a large dynamic range. The key features of the claimed invention, as understood from its abstract and detailed description, include:

• A plurality of camera channels, each comprising its own optics component and an image sensor with an array of photo-detectors.
• The sensors of these channels are integrated on a common semiconductor substrate.
• A processing component is configured to separately and simultaneously control the integration time of each channel, with at least one channel having a different integration time than another.
• The processing component combines data from these channels to generate a single image with a large dynamic range.

The patent explicitly identifies the problem it aims to solve: temporal aliasing in prior art systems that achieve wide dynamic range by sequentially capturing multiple images with different exposure times using a single image sensor.

Prior Art References for Analysis

The background section of US9232158B2 discusses several prior art patents that address wide dynamic range imaging:

• U.S. Pat. No. 4,647,975 (Nishizawa): Discloses a method of acquiring two or more images, each with a different exposure time, and then fusing or merging these images to form a single piece of pixel information with a wide dynamic range.
• U.S. Pat. No. 5,168,532 (Takagi et al.): Discloses using a selection rule to combine information from multiple images acquired at different exposure times.
• U.S. Pat. No. 5,671,013 (Takagi): Also discloses using a selection rule to combine information from multiple images with different exposure times.

US9232158B2 explicitly states that methods described in U.S. Pat. Nos. 4,647,975, 5,168,532, and 5,671,013 "exhibit undesirable temporal aliasing if the scene or camera is moving because the two or more images having different exposure times are captured using the same image sensor and thus are not captured concurrently."

Obviousness Argument

A person having ordinary skill in the art (PHOSITA) in digital camera design as of the August 25, 2004, priority date would have found the claimed invention of US9232158B2 obvious by combining the teachings of the aforementioned prior art references with general knowledge in the field.

1. Combination of US 4,647,975 (Nishizawa) and General Knowledge of Image Sensor Design and Semiconductor Integration:

• Nishizawa's Core Teaching: U.S. Pat. No. 4,647,975 clearly teaches the fundamental concept of creating a wide dynamic range image by combining multiple constituent images taken at different exposure times. This established the goal and a general methodology for achieving high dynamic range.
• Identified Problem: The limitation of Nishizawa's approach, acknowledged by US9232158B2, is the temporal aliasing caused by the sequential capture of images using a single image sensor.
• Motivation for Combination: A PHOSITA would be strongly motivated to overcome this known and undesirable temporal aliasing artifact. The most direct and logical solution to achieve concurrent capture of images at different exposure times is to employ separate imaging elements that can operate simultaneously.
• Application of General Knowledge: By 2004, the semiconductor industry had well-established capabilities for integrating multiple functional blocks, including arrays of photo-detectors and associated circuitry, onto a single integrated circuit (IC) or semiconductor substrate. This integration offered benefits such as compactness, reduced cost, lower power consumption, and improved signal integrity due to shorter electrical paths. Given the desire for simultaneous capture, a PHOSITA would readily conceive of implementing multiple image sensor arrays, each with its own independent exposure (integration time) control, on a single chip. This would allow each array to capture an image of the same field of view concurrently but with its own optimal integration time. The patent itself states that "the digital camera systems described herein overcome this dynamic range limitation through the use of multiple camera channels, including multiple optics and image sensors on a single integrated circuit (IC) or semiconductor substrate." This framing suggests that the integration of multiple sensors on a single IC was a known and viable solution pathway for addressing such problems.
• Optics and Processing: It would also be evident to a PHOSITA to associate each sensor array with its own optics (e.g., microlenses, which were also known in the art) to direct light to the respective sensor. Furthermore, the concept of combining the data from these different exposures to form a single high dynamic range image, as taught by Nishizawa, would be directly applicable to the simultaneously captured data from the multiple channels. The patent describes an image processor that "combines the images from the two or more camera channels to provide a full-color large dynamic range image."

Therefore, combining the known technique of multi-exposure HDR imaging (Nishizawa) with the well-understood engineering principle of semiconductor integration to enable simultaneous capture of different exposures on multiple on-chip sensors, in order to overcome the recognized problem of temporal aliasing, would have been obvious to a PHOSITA.

2. Combination of US 5,168,532 (Takagi et al.), US 5,671,013 (Takagi), and General Knowledge:

The arguments for obviousness using U.S. Pat. Nos. 5,168,532 and 5,671,013 would be substantially similar. These patents reinforce the prior art's understanding of combining multiple exposures for dynamic range, and they share the same deficiency of temporal aliasing due to sequential capture on a single sensor. The motivation to remedy this deficiency through simultaneous capture via integrated multiple channels would be equally strong.

In summary, the core advance of US9232158B2 – using multiple, independently controlled image sensor channels on a single substrate to capture simultaneous images at different integration times for high dynamic range – represents a predictable solution to a known problem in the art (temporal aliasing), achieved by combining existing HDR processing techniques with routine semiconductor integration and multi-sensor design practices.

I'll research the patent term adjustments, family members, and related applications for US 9232158.

Let me get more specific data on the patent term adjustment and the full family.

I have the key PTA data. Let me confirm the full family and check for divisionals and sibling continuations.

US Patent 9,232,158 — Term, Adjustments, and Family

Scope note: This section relies on the face of the patent (front-page data) and the family/"Related U.S. Application Data" fields as reproduced at Google Patents and in the USPTO/PTAB exhibit copies of the patent. A few items (notably any terminal disclaimers and any divisionals, if they exist) could not be independently confirmed from the USPTO Patent Center within this analysis; those are flagged explicitly below rather than assumed. Caveat on dates: the task header states a current date of April 26, 2026, while the live patent record and other generated sections carry dates in mid‑2026 (e.g., the fetch is stamped 2026‑09‑25, and the litigation/PTAB summaries use May–June 2026). This inconsistency is flagged, not resolved.

---

1. Patent identification (literal identifiers preserved)

Sources: Google Patents US9232158; patent front page PDF patentimages US9232158; PTAB Ex.1001 (Tesla, IPR2025‑00222).

---

2. Patent Term Adjustment (PTA) — 35 U.S.C. § 154(b)

PTA granted: 176 days.

The front page of US 9,232,158 carries the standard notice: "Subject to any disclaimer, the term of this patent is extended or adjusted under 35 U.S.C. 154(b) by 176 days." This appears on the granted patent as reproduced in the Google Patents/patentimages copy and in the IPR exhibit (e.g., the Tesla IPR2025‑00222 Ex.1001 and the W.D. Wash. litigation exhibit, both of which quote the 176‑day notice verbatim).

Internal consistency check (informational): The '236 application was filed Oct. 25, 2013 and issued Jan. 5, 2016 — a pendency of roughly 802 days, well under the 3‑year "B‑delay" date of Oct. 25, 2016. The 176‑day adjustment therefore corresponds to USPTO "A‑delay" (and any non‑overlapping adjustments), not B‑delay. This is consistent with a routine continuation prosecution.

Request for reconsideration under 37 C.F.R. § 1.705? Not indicated in the materials reviewed. No recalculation or Wyeth‑type challenge to this patent's PTA was identified. (Absence of evidence, flagged as such.)

---

3. Patent Term Extension (PTE) — 35 U.S.C. § 156

None. This patent is not PTE‑eligible.

PTE under § 156 is limited to patents claiming a product, a method of using a product, or a method of manufacturing a product where the product is a human/animal drug, medical device, food additive, or color additive subject to a regulatory review period. US 9,232,158 claims a digital camera / camera subsystem (optics + image‑sensor channels), so § 156 does not apply. No PTE has been granted, and none is available. (Consistent with the general § 154 vs. § 156 framework discussed in, e.g., the Cellect line of briefing: PTA under § 154 adjusts for PTO delay; PTE under § 156 is a separate, product‑specific FDA‑delay mechanism.)

---

4. Chronological term and projected expiration

The 20‑year term runs from the earliest U.S. non‑provisional filing to which benefit is claimed under § 120 — here application 11/212,803, filed Aug. 25, 2005. (Provisional applications 60/604,854, 60/695,946 and 60/795,882 do not start the 20‑year clock.)

This reconciles exactly with the "Adjusted expiration" field of 2026‑02‑17 shown on the Google Patents record (Aug. 25, 2025 + 176 days = Feb. 17, 2026).

Status: The patent record lists legal status as "Expired – Lifetime." Given the stated current date(s) in this analysis (April–September 2026), the patent has expired as of Feb. 17, 2026. Any enforcement is limited to past damages/accrued claims; no prospective infringement is possible.

---

5. Domestic continuation / CIP chain (parent applications)

The patent's own "CROSS‑REFERENCE TO RELATED PATENT APPLICATIONS" (Description) and the front‑page "(63) Related U.S. Application Data" give the following direct lineage — all continuations/CIPs, same specification:

Prior publications in this chain: US 2007/0257184 A1 (11/788,122); US 2009/0268043 A1 (12/496,854); US 2013/0076928 A1 (13/681,603‑related); US 2014/0049660 A1 (14/063,236).

Term interplay: Because each application is a straight continuation (or CIP) of the one before it, all members share the same 20‑year baseline (Aug. 25, 2025 from 11/212,803) subject to their own individual PTAs. The earlier members (7,564,019; 8,198,574; 8,334,494; 8,598,504) would have their own PTA figures and would typically require terminal disclaimers to keep their terms aligned — but I could not verify from the reviewed record whether such terminal disclaimers were filed, so I do not assert it.

---

6. Continuation applications of US 9,232,158 (children)

One direct continuation of the '236 application was identified:

US 10,009,556 (same title, same inventors: Olsen, Sato, Sun, Gates; assignee Callahan Cellular L.L.C.) explicitly states in its "Related U.S. Application Data": "Continuation of application No. 14/063,236, filed on Oct. 25, 2013, now Pat. No. 9,232,158..." It carries the same long chain back to 11/212,803 and the same provisionals. This child is referenced in the Google Patents "family"/"Other versions" and PTAB documents (e.g., Unified Patents patent page; IPR2022‑00710 Ex.1032).

No other children of 14/063,236 (i.e., additional continuations claiming direct benefit solely to '236) were conclusively identified within the available record. Because the same specification was worked aggressively in parallel (the family includes many co‑pending applications such as 11/265,669, 11/322,959, 11/478,242, 11/729,132, 11/788,120, 11/788,279, 11/810,623, 11/825,382, 11/888,546/570/582, 13/006,351, 13/100,725, 13/345,007, 13/647,708, 13/786,803, 14/171,963, 14/149,024, 15/074,275, 15/090,856 — all listed in the "Other publications"/Notice of Allowance records), some of those may claim benefit to different members of the chain. I flag that as not fully resolved here.

---

7. Divisional applications

No divisional applications were identified for US 9,232,158. Every relation documented on the face of the patent and in its Description is expressed as a "Continuation" or "Continuation‑in‑part" — the patent nowhere recites a "(62) Division of" relationship, and none of the reviewed derivative documents (patent copy, IPR exhibits, family listings) show a § 121 divisional. On the available evidence, the 9,232,158 family is a continuation/CIP lineage rather than a divisional lineage. (This is a negative finding based on the documents reviewed, not an affirmative USPTO Patent Center confirmation.)

---

8. Related family members (provisionals and non‑US)

U.S. provisionals in the chain:

• 60/604,854 — filed Aug. 25, 2004 (earliest priority)
• 60/695,946 — filed Jul. 1, 2005
• 60/795,882 — filed Apr. 28, 2006 (claimed via 11/788,122)

Foreign/PCT family (from the parent filing era):

• PCT/US2005/030256 — international application corresponding to the 11/212,803 line (International Preliminary Report on Patentability issued Mar. 17, 2009; International Search Report/Written Opinion mailed Jul. 7, 2008).
• PCT/US2006/025781 — IPRP issued Mar. 10, 2009; ISR/WO mailed Jul. 22, 2008.
• EP Application 05793927.4 — European regional phase of the above; the US patent cites "Communication on EP Application 05793927.4" (mailed Feb. 10, 2012 and Jul. 8, 2015), confirming a live EP counterpart.
• Chinese Application 200580032374.0 — first/second/third office actions cited (Feb. 5, 2010; Sep. 9, 2010; May 24, 2011).

These foreign siblings confirm the "Large Dynamic Range Cameras" invention family was internationalized on the 2005/2006 PCT filings, but I did not retrieve the granted foreign patent numbers/PTA equivalents within this analysis.

---

9. Bottom line

• PTA: 176 days (35 U.S.C. § 154(b)), stated on the face of US 9,232,158.
• PTE: None / not eligible (§ 156 applies only to drug/device/food/color‑additive products).
• Base term: 20 years from Aug. 25, 2005 (11/212,803) = Aug. 25, 2025.
• Projected/adjusted expiration: Feb. 17, 2026 — matching the record's "Adjusted expiration," now expired.
• Family structure: a six‑generation continuation/CIP chain (11/212,803 → 11/788,122 → 12/496,854 → 13/465,229 → 13/681,603 → 14/063,236), plus one child continuation 14/979,896 → US 10,009,556. No divisionals identified.
• Unresolved items (flagged): terminal‑disclaimer filings among the co‑pending family members, and whether additional continuations claim benefit to non‑'236 members of the chain, could not be confirmed from the reviewed record.

Due‑diligence note: Because the patent is expired and its family mates run from the same 2005 baseline, term remaining is effectively zero for enforcement purposes; only the pre‑Feb. 17, 2026 period is actionable, and the priority‑date chain (2004–2007) sharply limits the prior‑art window for any retrospective invalidity analysis.

Defensive Disclosure Document for US Patent 9232158

Patent Title: Large dynamic range cameras
Patent Number: US9232158B2
Priority Date: August 25, 2004
Issue Date: January 5, 2016
Expiration Date: February 17, 2026 (Patent has expired as of 2026-02-17)
Current Date: June 6, 2026
Specialization: Defensive Publishing Strategy

Objective: To expand the public domain of prior art related to large dynamic range camera systems by disclosing various derivative implementations and combinations, thereby rendering incremental improvements in this field obvious or non-novel for future patent applications. This document focuses on variations of the core inventive concept described in US9232158B2, namely, a multi-channel digital camera system with independently controlled, simultaneous integration times for each channel, and a processing component to combine their data for a wide dynamic range image.

Core Inventive Concept (Derived from Abstract and Summary):
A digital camera comprising:

1. A plurality of channels, each channel including:
   • An optics component.
   • A sensor comprising an array of photo-detectors.
2. A processing component operatively coupled to the plurality of channels.
3. The processing component configured to:
   • Separately and simultaneously control an integration time of each channel, wherein a first integration time of a first channel is different from a second integration time of a second channel.
   • Combine data from the plurality of channels to generate an image with an expanded dynamic range.

---

1. Material & Component Substitution

Derivative 1.1: Alternative Sensor Technologies

• Enabling Description: The digital camera system employs multiple channels, where each channel's sensor array utilizes different semiconductor materials or architectures optimized for specific light conditions. For instance, a "long integration time" channel could incorporate Indium Gallium Arsenide (InGaAs) sensors for enhanced near-infrared (NIR) sensitivity and low-light performance, while "short integration time" channels utilize silicon-germanium (SiGe) heterojunction phototransistors for high speed and saturation tolerance in bright conditions. The optics components would be correspondingly tailored for the spectral response of these materials. The processing component combines the spectrally and temporally diverse data streams into a single high dynamic range image.

graph TD
    A[Light Input] --> B{Beam Splitter / Filter Array}
    B --> C1(NIR Channel Optics - Long Int. Time)
    C1 --> D1(InGaAs Sensor Array)
    D1 --> E1{Channel Processor NIR}
    B --> C2(Visible Channel Optics - Short Int. Time)
    C2 --> D2(SiGe Sensor Array)
    D2 --> E2{Channel Processor Visible}
    E1 --> F(HDR Image Fusion Processor)
    E2 --> F
    F --> G[Large Dynamic Range Output Image]

Derivative 1.2: Liquid Crystal Lenses and Tunable Filters

• Enabling Description: Each optics component in the multi-channel system is replaced with a dynamically reconfigurable liquid crystal lens (LCL) system, allowing for electronic control of focal length and aperture (f-number) without mechanical movement. Furthermore, each channel incorporates an electro-optical tunable filter (e.g., based on acousto-optic deflectors or liquid crystal tunable filters) instead of fixed color filters. This enables the processing component to not only control integration time but also dynamically adjust the spectral bandpass and optical collection parameters of each channel in real-time, optimizing for scene content or specific light conditions (e.g., dynamically switching a channel from broadband visible to a narrow IR band).

graph TD
    A[Light Input] --> B(Scene Analysis)
    B -- Control Signals --> C1(Channel 1: LCL + Tunable Filter)
    C1 --> D1(Sensor 1)
    D1 --> E1{Channel Processor 1}
    B -- Control Signals --> C2(Channel 2: LCL + Tunable Filter)
    C2 --> D2(Sensor 2)
    D2 --> E2{Channel Processor 2}
    E1 --> F(HDR Image Fusion Processor)
    E2 --> F
    F --> G[Large Dynamic Range Output Image]

Derivative 1.3: Ferroelectric RAM (FRAM) Integrated Pixels

• Enabling Description: To enhance in-pixel charge storage capacity and non-volatility, the photo-detector array in each channel integrates ferroelectric random-access memory (FRAM) directly at the pixel level. This allows for multi-bit charge storage per pixel, effectively extending the dynamic range within a single integration period by quantizing charge accumulation. Different channels can then utilize distinct FRAM configurations or readout strategies (e.g., one channel uses a deeper well depth simulation via FRAM for long integration, another a shallower one for short integration) that are controlled by the processing component, further refining the simultaneous multi-exposure concept.

classDiagram
    class ChannelProcessor {
        +controlIntegrationTime()
        +readPixelData()
    }
    class PhotoDetectorArray {
        +pixels: Pixel[*]
        +capturePhotons()
    }
    class Pixel {
        +photoChargeStorage: FRAMCell
        +accumulateCharge()
        +transferCharge()
    }
    class FRAMCell {
        +storeCharge(charge)
        +readCharge()
        +nonVolatileStorage
    }
    PhotoDetectorArray --> Pixel : contains
    Pixel --> FRAMCell : uses
    ChannelProcessor --> PhotoDetectorArray : controls & reads

Derivative 1.4: Multi-Junction Photodiodes for Wavelength-Dependent Integration

• Enabling Description: Each sensor array utilizes multi-junction photodiodes, where different pn-junctions are stacked to absorb different spectral bands at varying depths within the semiconductor. The processing component is configured to control the integration time independently for each junction layer within a pixel, effectively creating "sub-channels" within a single physical pixel. For example, the blue-sensitive top junction could have a shorter integration time, while the red-sensitive deeper junction has a longer integration time. This allows for simultaneous multi-spectral, multi-exposure capture at a single pixel location, mitigating spatial alignment issues inherent in multiple physically separated channels.

stateDiagram-v2
    state "Multi-Junction Pixel" as Pixel {
        state "Blue_Junction" as BJ {
            BJ --> Short_Int : capture
            Short_Int --> Read_Out_B : integrated
        }
        state "Green_Junction" as GJ {
            GJ --> Medium_Int : capture
            Medium_Int --> Read_Out_G : integrated
        }
        state "Red_Junction" as RJ {
            RJ --> Long_Int : capture
            Long_Int --> Read_Out_R : integrated
        }
    }
    Read_Out_B --> Combine_Signals
    Read_Out_G --> Combine_Signals
    Read_Out_R --> Combine_Signals
    Combine_Signals --> Output_HDR_Pixel

---

2. Operational Parameter Expansion

Derivative 2.1: Terahertz (THz) Imaging System with Ultra-Fast Gating

• Enabling Description: The multi-channel concept is applied to Terahertz (THz) imaging for industrial inspection. Each channel comprises THz optics (e.g., silicon lenses or metamaterial lenses) and a THz sensor array (e.g., based on bolometers or plasmonic photodetectors). The "integration time" is replaced by an ultra-fast electrical gating pulse applied to each sensor array, controlling the duration of THz wave interaction. One channel uses femtosecond gating for high spatial resolution and transient event capture, while another uses picosecond gating for deeper penetration or material characterization where signal accumulation is critical. The processing component synchronizes these ultra-fast acquisitions and fuses the data for high dynamic range THz images, useful for detecting defects in composites or package contents.

sequenceDiagram
    participant THz_Src as THz Source
    participant Scene
    participant Ch1_Optics as Channel 1 Optics (Femto-Gate)
    participant Ch1_Sensor as Channel 1 Sensor
    participant Ch2_Optics as Channel 2 Optics (Pico-Gate)
    participant Ch2_Sensor as Channel 2 Sensor
    participant Proc as THz Fusion Processor

    THz_Src ->> Scene: Emit THz Pulse
    Scene -->> Ch1_Optics: Reflected/Transmitted THz
    Scene -->> Ch2_Optics: Reflected/Transmitted THz
    Ch1_Optics ->> Ch1_Sensor: Direct THz to sensor
    Ch2_Optics ->> Ch2_Sensor: Direct THz to sensor
    Proc ->> Ch1_Sensor: Apply Femtosecond Gate (simultaneous)
    Proc ->> Ch2_Sensor: Apply Picosecond Gate (simultaneous)
    Ch1_Sensor -->> Proc: Output Data (short integration)
    Ch2_Sensor -->> Proc: Output Data (long integration)
    Proc ->> Proc: Fuse Data for HDR THz Image

Derivative 2.2: Deep-Space Astronomy Imager with Milli-Second to Hour Integration Times

• Enabling Description: A digital camera system designed for deep-space astronomy utilizes multiple channels to capture celestial objects with vastly different brightness levels. One channel employs highly sensitive electron-multiplying CCD (EMCCD) or Superconducting Nanowire Single-Photon Detector (SNSPD) arrays with integration times ranging from milliseconds to tens of seconds to capture transient phenomena (e.g., fast radio bursts, occultations). A second channel, equipped with cryogenically cooled, ultra-low dark current CMOS sensors, operates with integration times spanning minutes to hours to accumulate faint light from distant galaxies or nebulae. A third channel, with standard CCDs, handles intermediate brightness. All channels image the same celestial field simultaneously, and a dedicated processing unit performs noise reduction, cosmic ray rejection, and HDR fusion across the enormous dynamic range.

graph LR
    A[Celestial Input] --> B{Beam Splitter / Telescope Focus}
    B --> C1(Ultra-Fast Channel: EMCCD/SNSPD - ms to s)
    C1 --> D1(Channel Processor - Transient)
    B --> C2(Intermediate Channel: Standard CCD - s to min)
    C2 --> D2(Channel Processor - Mid-Range)
    B --> C3(Deep-Field Channel: Cooled CMOS - min to hrs)
    C3 --> D3(Channel Processor - Faint Objects)
    D1 --> E(Deep-Space HDR Fusion Unit)
    D2 --> E
    D3 --> E
    E --> F[Scientific HDR Image Output]

Derivative 2.3: Microfluidic Flow Analysis Camera with Microsecond Integration

• Enabling Description: For high-speed microfluidic analysis, a multi-channel camera system is implemented where each channel captures fluorescent markers in a microchannel. The optics components are high-numerical-aperture microscope objectives. The sensors are high-frame-rate CMOS arrays with sub-microsecond integration time control. One channel operates with a very short integration time (e.g., 100 ns) to freeze rapid particle motion, preventing motion blur in fast flows. Another channel uses a slightly longer integration time (e.g., 1 µs) to enhance signal from dimmer markers or areas with lower concentrations. The processing component simultaneously captures and combines these images, enabling high dynamic range particle tracking and intensity analysis in complex microfluidic environments, particularly useful where both bright and faint signals coexist in fast flows.

flowchart TD
    A[Microfluidic Flow with Markers] --> B(Microscope Optics)
    B -- Split Light --> C1(Channel 1: High-Speed CMOS @ 100ns Int. Time)
    C1 --> D1{FPGA Processor - Motion Capture}
    B -- Split Light --> C2(Channel 2: High-Speed CMOS @ 1us Int. Time)
    C2 --> D2{FPGA Processor - Intensity Capture}
    D1 --> E(Real-Time HDR Fusion Engine)
    D2 --> E
    E --> F[Combined High-Resolution Flow Data]

---

3. Cross-Domain Application

Derivative 3.1: Autonomous Vehicle Multi-Spectral Perception System

• Enabling Description: The multi-channel camera system is integrated into an autonomous vehicle's perception stack. One channel is optimized for visible light with short integration times to handle bright daylight and glare, ensuring clear detection of road signs and other vehicles. A second channel, using long integration times, is sensitive to near-infrared (NIR) and short-wave infrared (SWIR) for night vision and penetrating fog or heavy rain, enhancing pedestrian and obstacle detection in adverse conditions. A third channel might be tuned for specific spectral bands to detect brake lights, emergency vehicle lights, or road surface conditions. The processing component fuses this multi-spectral, multi-exposure data in real-time, providing a robust, high dynamic range environmental perception for autonomous driving decisions.

graph TD
    A[Road Environment] --> B{Vehicle Sensor Suite - Light Input}
    B --> C1(Visible Channel - Short Int. Time)
    C1 --> D1(Visible Light Sensor)
    D1 --> E1{Perception Module - Daylight}
    B --> C2(NIR/SWIR Channel - Long Int. Time)
    C2 --> D2(NIR/SWIR Sensor)
    D2 --> E2{Perception Module - Night/Fog}
    E1 --> F(HDR Fusion & Object Detection)
    E2 --> F
    F --> G[Autonomous Driving Decision Engine]

Derivative 3.2: Precision Agriculture Crop Health Monitoring Drone

• Enabling Description: A drone-mounted multi-channel camera system is deployed for precision agriculture. One channel captures images in the visible spectrum with short integration times for general canopy structure and disease detection under varying sunlight. A second channel focuses on specific narrow-band wavelengths (e.g., red-edge, NIR) critical for Normalized Difference Vegetation Index (NDVI) calculation and early stress detection. This channel uses longer integration times to gather sufficient spectral information, especially under partial cloud cover or during dawn/dusk operations. A third channel may capture thermal infrared (TIR) with its own optimized integration for water stress detection. The processing component combines these images, generating high dynamic range, multi-spectral maps of crop health, irrigation needs, and disease spread.

flowchart LR
    A[Crop Field] --> B(Drone Flight Path)
    B --> C{Multi-Spectral Camera Payload}
    C -- Visible Light --> D1(RGB Channel - Short Int. Time)
    D1 --> E1[Visible Image Processor]
    C -- Red-Edge/NIR --> D2(NDVI Channel - Long Int. Time)
    D2 --> E2[NDVI Image Processor]
    C -- Thermal IR --> D3(Thermal Channel - Optimized Int. Time)
    D3 --> E3[Thermal Image Processor]
    E1 --> F(Cloud-Based HDR & Analytics Platform)
    E2 --> F
    E3 --> F
    F --> G[Crop Health Report & Actionable Insights]

Derivative 3.3: Industrial Quality Control for Pharmaceutical Manufacturing

• Enabling Description: In pharmaceutical manufacturing, a multi-channel camera system is used for automated quality control of pills or vials on a high-speed conveyor belt. One channel uses visible light with short integration times to detect surface imperfections, color variations, or print quality. A second channel employs UV-fluorescence imaging with longer integration times to detect active pharmaceutical ingredients (APIs) or excipient distributions that fluoresce under UV light. A third channel uses transmission imaging (e.g., X-ray or specific IR wavelengths) with tailored integration for internal defect detection. The processing component simultaneously acquires data from all channels, combining them to perform comprehensive, high dynamic range quality assurance, identifying defects that would be missed by single-spectrum or single-exposure methods.

graph LR
    A[Pill on Conveyor] --> B{Illumination & Optics System}
    B -- Visible --> C1(Visible Channel - Short Int.)
    C1 --> D1(Image Processor - Surface QA)
    B -- UV-A --> C2(UV-Fluor. Channel - Long Int.)
    C2 --> D2(Image Processor - API Detect)
    B -- X-Ray --> C3(X-Ray Trans. Channel - Tailored Int.)
    C3 --> D3(Image Processor - Internal Defect)
    D1 --> E(Integrated Quality Control System)
    D2 --> E
    D3 --> E
    E --> F{Pass/Fail Decision & Ejection}

---

4. Integration with Emerging Tech

Derivative 4.1: AI-Driven Optimization of Integration Times

• Enabling Description: The multi-channel camera system is enhanced with an embedded AI inference engine. Instead of fixed or user-selected integration times, the AI analyzes real-time scene content, lighting conditions, and desired output characteristics (e.g., target dynamic range, specific object of interest, motion blur tolerance). Using deep learning models (e.g., a Convolutional Neural Network trained on diverse HDR scenes), the AI dynamically adjusts the integration times of each channel for optimal image acquisition in subsequent frames. This allows for adaptive exposure control that goes beyond simple histogram analysis, predicting and compensating for complex lighting changes and motion, further enhancing the effective dynamic range and image quality.

sequenceDiagram
    participant Scene
    participant Sensors as Multi-Channel Sensors
    participant PreProc as Pre-Processing (e.g., initial metering)
    participant AI_Engine as AI Inference Engine
    participant IntControl as Integration Time Controller
    participant Proc as HDR Image Fusion Processor
    participant Output as Output Image

    Scene ->> Sensors: Light Input
    Sensors ->> PreProc: Raw Sensor Data
    PreProc ->> AI_Engine: Scene Metadata / Low-Res Preview
    AI_Engine ->> AI_Engine: Analyze scene, predict optimal Int. Times
    AI_Engine ->> IntControl: New Integration Times (T1, T2, ..., Tn)
    IntControl ->> Sensors: Apply new Integration Times (simultaneous capture)
    Sensors ->> Proc: Raw Sensor Data (optimized exposure)
    Proc ->> Output: Fused HDR Image

Derivative 4.2: IoT-Enabled Remote Monitoring with Real-Time Data Streaming

• Enabling Description: The multi-channel digital camera is integrated into an Internet of Things (IoT) network for remote surveillance or environmental monitoring. Each channel's processing component performs localized preprocessing (e.g., noise reduction, initial HDR merging) and then encrypts and streams segmented high dynamic range image data (e.g., specific regions of interest or compressed video streams) via a secure IoT gateway. IoT sensors (e.g., ambient light sensors, temperature, humidity) co-located with the camera provide contextual metadata that influences the camera's integration time settings via a cloud-based control plane. This enables real-time adaptive HDR imaging and alerts based on environmental conditions, with blockchain potentially used for secure data provenance and integrity verification of the streamed image data.

graph TD
    A[Scene] --> B(Multi-Channel Camera)
    B --> C{Local Pre-Processing & Encryption}
    C --> D[IoT Gateway]
    D --> E(Cloud Control Plane & Storage)
    C -- Contextual Data --> E
    F[Ambient Light Sensor] -- Env. Data --> E
    E -- Integration Time Commands --> B
    E --> G[Blockchain Ledger]
    G -- Verifies Data Provenance --> D
    E --> H[Remote Monitoring Dashboards]

Derivative 4.3: Edge Computing for Decentralized HDR Fusion

• Enabling Description: Instead of a single centralized processing component, the HDR fusion logic is distributed across edge computing nodes, potentially even within each channel's dedicated processor or a local system-on-chip (SoC) adjacent to the sensor arrays. Each channel processor performs initial image correction and then transmits partially processed data (e.g., tone-mapped sub-images, statistical scene descriptors, or feature maps) to a lightweight edge orchestrator. This orchestrator, running on a low-power microcontroller, performs the final, adaptive HDR merging and local storage/transmission. This reduces bandwidth requirements to a central cloud, improves latency for real-time applications, and enhances privacy by processing sensitive data closer to the source. The AI-driven optimization (Derivative 4.1) could also reside at the edge.

flowchart TD
    A[Scene] --> B1(Channel 1 Sensor+Optics)
    A[Scene] --> B2(Channel 2 Sensor+Optics)
    B1 --> C1{Channel 1 Edge Processor - Partial HDR}
    B2 --> C2{Channel 2 Edge Processor - Partial HDR}
    C1 --> D(Edge Orchestrator - Final HDR Fusion)
    C2 --> D
    D --> E[Local Storage / Reduced Bandwidth Tx]
    D --> F{AI/ML for Adaptive Control}
    F --> C1
    F --> C2

---

5. The "Inverse" or Failure Mode

Derivative 5.1: Fail-Safe Low-Power Monochromatic Mode

• Enabling Description: In the event of a critical system failure (e.g., primary power loss, channel malfunction, thermal overload), the multi-channel camera system automatically switches to a fail-safe, low-power monochromatic imaging mode. This involves disabling all but one (e.g., the green channel, which often has the highest responsivity) or a pre-selected broadband channel, and operating it at a fixed, medium integration time. Color filtering, if present, is bypassed or a clear aperture is used. The image processing component simplifies to basic grayscale conversion and minimal noise reduction, consuming significantly less power and computational resources, ensuring continuous, albeit reduced, operational capability for critical functions like emergency navigation or basic surveillance.

stateDiagram-v2
    state "Normal Operation (HDR Color)" as Normal {
        Normal --> Critical_Failure : Fault Detected
    }
    state "Critical Failure" as CF {
        CF --> Low_Power_Mono : Activate Fail-Safe
    }
    state "Low-Power Monochromatic Mode" as LPMM {
        LPMM : Disable excess channels
        LPMM : Fixed Medium Integration Time
        LPMM : Simple Grayscale Processing
        LPMM --> Normal : System Recovered
    }
    Critical_Failure --> Critical_Failure : Fault Persists
    Low_Power_Mono --> Low_Power_Mono : Operating in Fail-Safe

Derivative 5.2: Limited-Functionality "Preview" Mode for Resource-Constrained Devices

• Enabling Description: For integration into resource-constrained devices (e.g., low-end IoT devices, wearables with minimal battery), the multi-channel camera system can operate in a limited-functionality "preview" mode. In this mode, only two channels are active: one with a short integration time (for bright areas) and one with a long integration time (for dark areas), both potentially using a Bayer pattern or simpler color filters to save cost/complexity. The image processing component performs a highly optimized, computationally inexpensive HDR merge (e.g., simple linear blending or selection based on saturation thresholds), generates a lower-resolution output image, and bypasses advanced image adjustments (sharpening, noise reduction). This provides a usable, albeit less sophisticated, HDR image while minimizing power consumption, memory footprint, and processing load.

flowchart LR
    A[Power Constraints / Low Resources] --> B(Mode Selector)
    B -- Select "Preview" --> C{Two Active Channels}
    C -- Short Int. --> D1(Channel 1 - Low Res)
    C -- Long Int. --> D2(Channel 2 - Low Res)
    D1 --> E(Optimized HDR Merger - Edge)
    D2 --> E
    E --> F[Low-Res HDR Preview Output]
    F -- No Advanced Processing --> G(Reduced Power/CPU Load)

Derivative 5.3: Controlled Saturation "Clipping" Mode for Specific High-Brightness Targets

• Enabling Description: In applications where certain very bright objects are the primary targets (e.g., welding arc monitoring, solar panel inspection, medical endoscopy with bright light sources), the system can enter a "controlled saturation" mode. Here, the processing component actively allows the shorter integration time channels to saturate in areas outside the primary target's region of interest, effectively "clipping" unimportant bright areas to save processing bandwidth and storage. Simultaneously, the longer integration time channels are still used to capture details in darker, surrounding areas. The HDR fusion algorithm is modified to prioritize data from the short exposure channels within defined target regions, and from long exposure channels elsewhere. This inverse approach optimizes for critical bright features while still retaining some dynamic range in the context.

graph TD
    A[Scene with Bright Target] --> B(Multi-Channel Input)
    B -- Short Int. Data --> C1(Channel Short - High Brightness Data)
    B -- Long Int. Data --> C2(Channel Long - Low Brightness Data)
    C1 --> D{Processor: Identify Target Region}
    C2 --> D
    D --> E{Modified HDR Fusion}
    E -- Prioritize C1 in Target Region --> F(Output Image with Controlled Saturation)
    E -- Prioritize C2 in Non-Target Region --> F

---

Combination Prior Art Scenarios with Open-Source Standards

These scenarios describe how the principles of US9232158B2 could be combined with existing open-source standards to create new prior art, making future improvements obvious.

1. Integration with OpenCV for Real-Time HDR Processing on Embedded Linux Systems:

   • Description: A multi-channel digital camera system (as described in US9232158B2) is implemented on an embedded system running a Linux kernel, such as a Raspberry Pi or NVIDIA Jetson. The image acquisition from the multiple sensor channels is managed by a custom driver or V4L2 (Video4Linux2) interface. The processing component's HDR fusion and image pipeline functionalities are implemented using the publicly available OpenCV library (e.g., cv::createMergeDebevec, cv::createMergeMertens, or custom blending algorithms). This combination openly demonstrates real-time, software-defined HDR image acquisition and fusion from simultaneous multi-exposure channels using a widely adopted open-source computer vision framework, establishing prior art for software-based HDR solutions on commodity hardware.
   • Prior Art Value: Establishes the combination of hardware-accelerated multi-channel capture with accessible, open-source software libraries for HDR reconstruction. Any future patent attempting to claim novel HDR fusion algorithms or real-time implementation on embedded Linux platforms, without a truly inventive step beyond parameter tuning or minor algorithmic modifications, would be rendered obvious by this disclosure.

2. HDR Camera System with GStreamer for Modular Video Pipeline:

   • Description: The multi-channel camera system (US9232158B2) integrates with GStreamer, an open-source multimedia framework, to create a flexible and modular video processing pipeline. Each camera channel feeds its raw video stream (e.g., MIPI CSI-2 interface) into a GStreamer source element. Separate GStreamer elements are then used for individual channel processing (e.g., gain adjustment, demosaicing if applicable, black level correction). A custom GStreamer plugin or a combination of existing plugins performs the real-time HDR merging of the multiple simultaneous exposures. The final HDR video stream can then be encoded (e.g., H.264 using x264enc plugin) and transmitted over a network using standard GStreamer sink elements. This demonstrably implements the core inventive concept within a highly extensible open-source video processing framework.
   • Prior Art Value: Showcases a configurable and modular approach to building HDR cameras using standard open-source multimedia pipelines. This would preempt patents on specific architectural arrangements of HDR camera pipelines that leverage well-known GStreamer patterns for source, filter, and sink elements, or on general real-time HDR video processing in such an environment.

3. Sensor Control via I2C/SPI and Data Output via USB using Standard Linux Drivers:

   • Description: A multi-channel camera system as per US9232158B2 utilizes standard open-source Linux kernel drivers for sensor control and data transfer. The integration time control for each channel's sensor is managed via I2C or SPI communication protocols using existing Linux device tree bindings and kernel modules. Raw image data from the multiple channels is aggregated and then streamed out via a USB interface, presented as a standard UVC (USB Video Class) device. The HDR merging algorithm, running either on the host CPU or a programmable logic device, processes these UVC streams. This emphasizes the use of widely available open-source hardware interfaces and drivers to implement the core functionality of independent integration time control and data acquisition from multiple sensors.
   • Prior Art Value: Documents the use of common, open-source communication protocols (I2C, SPI, USB) and their associated Linux drivers to control multi-channel image sensors with independent integration times. This would render obvious any attempt to patent the "interfacing" or "control mechanism" of such a system if it merely relies on these well-established and publicly documented standards without a novel underlying hardware or protocol invention.