US 9709404

I'll search for authoritative information on US 9709404, including any 2026 CAFC docket activity.

Let me confirm whether any litigation, PTAB, or CAFC activity names this specific patent.

US Patent 9,709,404 B2 — Analyst Summary

Important caveat up front: I searched for USPTO records, litigation, PTAB, and CAFC 2026 docket activity specifically naming patent 9,709,404. I found no CAFC 2026 docket entry, no IPR/PTAB proceeding, and no district-court complaint naming this patent. Absence of a search hit is not proof of absence — public docket aggregators lag and are incomplete — so I flag this as "no evidence found," not "confirmed none exists." I also note that numeric near-matches appeared in searches (e.g., a Brazilian application "BR 9709404"; a Northern District of California "‑940 patent" in Interum Group v. Zoom; Ohio N.D. case citing U.S. Patent 10,334,940). None of these are U.S. Patent 9,709,404 and none should be conflated with it.

Bibliographic data (from the patent front page / Google Patents record)

Abstract

"A vision-aided inertial navigation system (VINS) is described in which a filter-based sliding-window estimator implements an Iterative Kalman Smoother (IKS) to track the 3D motion of a VINS system, such as a mobile device, in real-time using visual and inertial measurements."

Plain-language overview of the independent claims

The patent has three independent claims: 1 (system), 10 (method), and 17 (non-transitory computer-readable storage device). Claim 17's full body text was not returned in my searches (I only retrieved its preamble: "A non-transitory computer-readable storage device comprising program code to cause a processor to perform the operations of:"). I therefore describe 17 as mirrored to claim 10 with reasonable—but not authoritative—confidence, and flag that uncertainty.

Claim 1 — Vision-aided inertial navigation system (apparatus)

A VINS comprises: (a) at least one camera/image source producing image data containing features observed from multiple poses along a trajectory; (b) an IMU producing motion data; and (c) a hardware-based processing unit with an estimator that computes a sliding window of state estimates (at least position and orientation per pose) plus a covariance/uncertainty for each estimate. The estimator is defined by a specific three-step computation:

1. Classify observed visual features into at least a first set and a second set, based on where in the sliding window (i.e., which pose) each feature was observed — the second set corresponding to older poses than the first set.
2. Apply an extended Kalman filter to update, within the sliding window, the state estimates for both the VINS and the features, using IMU data and image data associated with both sets of features.
3. Update the covariances using the IMU data and image data associated with the second (older) set only, without using the image data of the first (newer) set.

In short: the state gets corrected using all observations, but the covariance/consistency accounting is driven only by older "mature" features. This is the core "two-step" decoupling that lets measurements be re-processed without double-counting.

Claim 10 — Method

Mirrors claim 1 in method format: receiving image data along a VINS trajectory; receiving IMU data; computing, with a processing unit having an estimator, a sliding window of state estimates for position and orientation across poses. The computing step recites the same three operations — classify features by pose position in the window (older second set vs. newer first set); apply an EKF to update state estimates for the VINS and the features using IMU and image data from both sets; and update each covariance using the older (second) set without the newer (first) set.

Claim 17 — Non-transitory computer-readable storage device

A computer-readable storage device containing program code causing a processor to perform the operations of the method claims. Based on the specification ("the invention is directed to a computer-readable storage medium containing instructions that cause a programmable processor to implement the methods described herein") and on dependent claim practice, this claim almost certainly tracks claim 10's steps. I did not retrieve the literal text of claim 17 and cannot verify its exact scope.

Notable dependent-claim context (for claim-charting orientation)

• Claims 2–3: refine the classification to mature vs. immature features; compute a set of prior constraints linearly relating immature-pose state estimates to mature-pose state estimates; state updates use all IMU + image data plus the prior constraints, while covariance updates use only mature features and tail-end IMU data plus the prior constraints.
• Claim 4: the EKF update uses the covariance and prior constraints without computing a Hessian matrix — this is the express numerical-stability distinction over inverse-filter (INVF) approaches.
• Claim 7: recites six feature categories (SR, APF, IPF, OSF, OF, NSF).
• Claim 8: a resource-aware estimation policy deciding which measurements to process given available computation.
• Claims 5, 6, 9: map-building; integration into tablet/laptop/phone/robot/vehicle; and VINS embodied as robot, vehicle, mobile device, or wearable.

Technical gist / why it matters

The patent's stated contribution is a sliding-window Iterative Kalman Smoother that keeps an EKF-style prior (a covariance matrix + a set of linearized constraints) rather than a Hessian, enabling re-linearization and re-processing of inertial and visual measurements across overlapping windows. The specification reports 32-bit (single-precision, ARM NEON) viability, ~4× speed-up, real-time wearable operation, and improved convergence of IMU velocity and IMU-camera clock drift versus conventional EKF.

Uncertainty flags

1. No CAFC 2026 docket, PTAB proceeding, or litigation found for this exact patent. Treat as "no evidence located," not a verified negative.
2. Claim 17's literal text is unverified in this analysis.
3. Assignee history: the Google Patents record shows two 2017–2018 reassignments to "Regents of the University of Minnesota," consistent with inventor-specific assignments; the current listed assignee is the University of Minnesota System / Regents of the University of Minnesota. There is also a 2009-era NSF confirmatory-license event shown in search results, but that appeared in connection with a different patent's legal-event list (U.S. 2009/0248304), so I do not attribute an NSF license to 9,709,404 without further verification.
4. Any statement about current litigation status should be re-verified against PACER, the PTAB's API, and the CAFC docket directly, since aggregator coverage of 2026 matters may be incomplete.

I'll search for litigation involving US patent 9709404 across patent litigation resources.

Let me search more targeted litigation databases.

Let me check the Unified Patents litigation portal and other case-level sources directly.

Litigation Search Results for U.S. Patent No. 9,709,404

Bottom line: Based on my searches, I found no litigation — district court, ITC, or Federal Circuit — involving U.S. Patent No. 9,709,404. I could not identify any plaintiff, defendant, jurisdiction, case number, filing date, or outcome for a case asserting this patent.

What I verified about the patent itself (anchor for the search)

• Patent: US 9,709,404 B2, "Iterative Kalman Smoother for robust 3D localization for vision-aided inertial navigation" (Google Patents)
• Inventors: Stergios I. Roumeliotis; Dimitrios G. Kottas
• Assignee: Regents of the University of Minnesota
• Application: US 15/130,736; filed 2016-04-15; priority 2015-04-17; granted 2017-07-18; anticipated expiration 2036-04-15
• Provisional: U.S. Provisional App. No. 62/149,194 (filed Apr. 17, 2015)

What the searches returned (and why none is a hit for the '404 patent)

The results surfaced University of Minnesota patent litigation, but against different patents — not the '404:

None of these Minnesota cases, IPRs, or appeals involves U.S. 9,709,404 (a 2017-issued VINS patent). Other hits (e.g., Daignean Technologies v. Ericsson/Nokia/T-Mobile, various IPRs by Unified Patents) concerned unrelated patents and are not relevant to the '404.

Important limitations and caveats

• I was unable to directly query the Unified Patents litigation portal, PACER, or Docket Navigator with a case list for this specific patent (my direct calls to those sources were cut off). My conclusion rests on general web searches, which returned no case naming 9,709,404.
• Small university-licensed patents like this one are frequently asserted quietly (e.g., against camera/AR/robotics products), and complaints may not be well indexed by general search engines. A negative general-web result is not proof of no litigation.
• Modern UMN VINS patent families (including this one) may be asserted alongside related patents; the '404 did not appear in any UMN assertion I located.

Recommended next steps for a definitive answer

To confirm the absence of litigation with high confidence, run these targeted queries:

1. Unified Patents Portal — search the litigation database and "District Court Analytics" for patent number 9709404 (portal.unifiedpatents.com/litigation).
2. PACER / Docket Navigator — search all district courts for "9,709,404" or "9709404."
3. USPTO/PTAB — search for IPR/PGR petitions naming US 9,709,404 (none appeared in my searches).
4. ITC EDIS — check for any Section 337 investigation citing the patent.

Based on what I can currently access, the honest answer is: no litigation involving US Patent 9,709,404 is known.

I'll start by verifying the structured data's "no PTAB activity" default against public sources, then trace any litigation/appeal trail.

Let me search for litigation and any PTAB dockets referencing this patent.

No hits so far tie this patent to any PTAB case. Let me probe more specifically, including the patent's family/continuations and any assertion activity.

Proceedings overview

Zero AIA trial proceedings are on file for US 9,709,404 — the USPTO ODP structured block returns no IPR, PGR, or CBM, and my open-web sweep (PTAB filings, Docket Alarm PTAB records, Federal Circuit / CourtListener dockets) surfaced no petition, institution decision, FWD, or appeal naming this patent; breakdown is therefore 0 active / 0 claims invalidated / 0 claims sustained / 0 settled / 0 institution denied, which means the patent has never been tested at the PTAB and is, defensively, an unknown rather than a hardened or broken asset — you cannot lean on an existing invalidation, but you also face no § 315(e) estoppel and no preliminary-construction baggage from a prior panel.

No proceeding to report

The "For each proceeding" template is intentionally empty here. Rather than invent a docket, here is the audit trail of what was checked and what it showed:

Caveat, stated plainly: my search budget was capped and PTAB E2E full-text search was not directly queried. The absence of results is strong but not cryptographic. Before relying on this, run a direct post-grant search on PTAB E2E by patent number (https://ptacts.uspto.gov) and the Patent Center "Post-Grant" tab for US 15/130,736.

Strategic summary

Claim status: the entire issued claim set of the '404 is UNTESTED. No claim has been canceled, confirmed, or construed by any PTAB panel; no Final Written Decision exists to quote. The patent's only public adjudicative events are its prosecution history (filed 2016-04-15, priority 2015-04-17, granted 2017-07-18, expiration 2036-04-15), an inventor-assignment chain to the Regents of the University of Minnesota, and a small-entity-then-large-entity maintenance-fee trail. For a defendant, that cuts both ways: there is no "claims 1–5 are canceled, the demand letter is baseless" argument available, and equally no adverse PTAB record that the plaintiff will have to explain away.

Estoppel landscape: essentially dark. Statutory estoppel under § 315(e)(2) arises only from an IPR that reaches a final written decision, and § 325(e)(2) likewise for PGR — neither is triggered because nothing was instituted. So every prior-art ground remains open to you, including: § 102/§ 103 art the patent owner's own specification cites or incorporates (the KLT tracking paper by Lucas & Kanade, the Kneip et al. BMVC 2011 2-point RANSAC paper, and the incorporated applications 12/383,371; 14/271,971; 14/733,468), machine-vision/VINS literature published before 2015-04-17, and — critically — system prior art (§ 102(a)(1) public use / on-sale) and non-patent literature that IPR petitioners cannot use at the PTAB at all. That asymmetry is your best lever: a § 282 district-court invalidity case has a materially wider evidentiary menu than any IPR would have. There is also no § 315(b) one-year clock running against you right now, and filing your own IPR would not create estoppel against other defendants.

Pattern signals. The patent owner, Regents of the University of Minnesota, is a seasoned and aggressive PTAB participant — but on unrelated technologies, which is useful only as calibration of its litigation temperament, not as precedent about the '404. UMN litigated state sovereign immunity all the way up in Regents of the Univ. of Minn. v. LSI Corp., 926 F.3d 1327 (Fed. Cir. 2019) (affirming that state sovereign immunity does not bar IPR; appeals docket 18-1559, opinion at https://cases.justia.com/federal/appellate-courts/cafc/18-1559/18-1559-2019-06-14.pdf), arising from Ericsson's IPRs (IPR2017-01186, -01197, -01200, -01213, -01214) and LSI's IPR2017-01068. In UMN's 4G LTE campaign (Regents of Univ. of Minn. v. AT&T Mobility LLC, D. Minn. No. 0:14-cv-04666, consolidated with -04669/-04671/-04672), the court stayed the case pending IPR, then lifted the stay after the Board declined to institute — the case went on to claim construction, summary-judgment practice, and a jury trial, and the Verizon branch ended in a negotiated dismissal (plaintiff's claims with prejudice; defendants' counterclaims without prejudice). Separately, UMN lost an IPR badly in Gilead (IPR2017-01712, U.S. 8,815,830), where all asserted claims were invalidated and the Federal Circuit affirmed. No defensive aggregator (e.g., Unified Patents, RPX) appears anywhere in the '404 chain. The net read: UMN is willing to litigate to judgment, is not frightened of the PTAB, and has both won and lost there — nothing about the '404 suggests it would roll over if challenged.

Recommended next steps

1. Do not assert PTAB history that does not exist. Any defendant brief or opinion letter stating the '404 was "upheld in IPR" or "had claims canceled" would be a fabrication. The accurate statement is: no AIA trial has been filed or instituted.
2. Verify the negative directly. Pull the patent's Post-Grant tab in Patent Center (app. 15/130,736) and search PTAB E2E by patent number: https://ptacts.uspto.gov. Also confirm no ex parte reissue/reissue-related challenge has been filed (the '404 is a UMN portfolio patent from the Roumeliotis/Kottas VINS family, and UMN has previously used reissue, e.g., RE45,230).
3. Because nothing is instituted, no trial-stage milestones exist — there is no institution-decision deadline, no § 316(a)(11) one-year FWD clock, and no oral hearing date to track. If you want those dates, they begin on the day you file.
4. Build the invalidity case on the widest record. Since § 315(e)(2) is inapplicable to this patent, you can pair an IPR (limited to patents/printed publications) with parallel district-court grounds that IPR cannot reach — public use, on-sale, and system art predating 2015-04-17 — without estoppel risk to your district-court case so long as your IPR grounds and your district-court grounds stay distinct in scope.
5. Watch for a companion filing. These UMN VINS patents have a live continuation family (public records show later UMN VINS filings citing the '404, e.g., U.S. 12,529,563). If the patent owner is monetizing this family, an IPR against the '404 or a sibling is the point at which a proceeding would appear here.
6. Link discipline: if a proceeding later issues, cite the FWD via PTAB E2E and any appeal via CourtListener (https://www.courtlistener.com) or the CAFC docket. Nothing to link today, because nothing exists.

Confidence statement: I am highly confident there is no PTAB proceeding on the '404 as of 2026-09-29 based on the ODP block plus two independent search sweeps. I could not run an exhaustive PTAB E2E full-text search within this session, so treat this as a strong negative subject to that one direct verification step.

I'll research the assignment/ownership record for US 9,709,404, focusing on USPTO Assignment Center records and any cross-references.

Let me dig into the specific assignment records and the inventors' employment context.

Let me try to retrieve the actual recorded assignment reel/frame data and the inventors' employment history.

Assignment & Ownership Analysis — US Patent 9,709,404 B2

Data-integrity caveat (read first): The Google Patents record and the patent front page give me the identity and dates of the two recorded ownership events, but my searches did not return the reel/frame numbers, the execution dates, or the correspondent of record for those recordings. I will not invent them. Where a value is unknown I say so explicitly. The Assignment Center entry point for verification is: https://assignmentcenter.uspto.gov/ (legacy index: https://assignment.uspto.gov/patent/index.html — search by patent number 9709404).

Inventors

Pattern check: No unusual departure pattern. Both named inventors were UMN MARS Lab personnel at filing. Roumeliotis remained a UMN professor through and after issuance (he is the named inventor on numerous UMN-owned VINS patents in this family). Kottas was a student who later graduated; the technology track (the MARS Lab's VINS work) was carried into Google's Project Tango via a ~$1.35M Google research grant to the University — a licensing/grant relationship, not an inventor defection or portfolio fire-sale. Corroborating public reporting: the MARS Lab VINS tech "provides the foundation for ATAP's Project Tango," and Roumeliotis/Kottas demoed the app on a Samsung Galaxy S4 (besttechie.com profile of the MARS Lab).

Original assignee

Regents of the University of Minnesota (Minneapolis, MN) — the issued patent names the University as applicant/assignee (front page "(71) Applicant: Regents of the University of Minnesota"; "(73) Assignee" likewise).

• Primary line of business: public research university (higher education + sponsored research + technology transfer). It is an operating institution, but not a commercial product company — it does not itself ship a product embodying the claims.
• Did it ship a product embodying the claims? No. The patented VINS technology was licensed/grant-funded into Google's Project Tango rather than commercialized by UMN directly. UMN operates through its Office of Technology Commercialization (tech transfer), not through product sales.
• Current status: operating and solvent. No bankruptcy, dissolution, or acquisition of the assignee is implicated.

Assignment timeline

The record shows two assignments, both from an individual inventor to the same university employer — i.e., the original inventor-to-employer chain (the university was already the applicant at filing). Both are labeled "reassignment" by Google Patents; both assignees are identical, so this is not a downstream transfer chain.

• Execution date: not retrieved / recorded 2017-06-13 — Reel/frame: not retrieved

  • Conveyance: Assignment
  • Assignor: KOTTAS, Dimitrios G.
  • Assignee: Regents of the University of Minnesota
  • Correspondent: not retrieved (patent-level attorney/agent of record is Shumaker & Sieffert, P.A., but I cannot confirm this firm handled the assignment recording — do not assume)
  • Context: Original inventor-to-employer assignment (standard university obligation-to-assign), recorded ~5 weeks before issuance.

• Execution date: not retrieved / recorded 2018-08-21 — Reel/frame: not retrieved

  • Conveyance: Assignment
  • Assignor: ROUMELIOTIS, Stergios I.
  • Assignee: Regents of the University of Minnesota
  • Correspondent: not retrieved
  • Context: Original inventor-to-employer assignment, recorded ~13 months after issuance.

Interpretation: These two recordings are the same conceptual event — each inventor separately assigning his rights to the university employer. There is no post-issuance transfer to any third party, LLC, or aggregator in the record. If the Assignment Center returns additional entries (e.g., a confirmatory assignment or an NSF confirmatory-license), that is the one gap I could not close and it should be checked directly. Note: the earlier "NSF confirmatory license" flag in the prior summary was traced to a different patent's record (U.S. 2009/0248304), so it is not attributed here.

Because the record exists but I could not pull reel/frame or correspondent, I continue below rather than stopping — but treat the reel/frame cells as unverified.

Timeline diagram

timeline
    title Ownership of US 9709404
    2016 : App filed by Univ of Minnesota
         : Inventors Roumeliotis and Kottas
    2017 : Patent issued July 18
         : Kottas rights assigned to Univ
    2018 : Roumeliotis rights assigned to Univ
    2036 : Anticipated expiration

NPE / troll-pattern signals

Verdict

Insufficient data (only the original assignment) — selected because the record contains only the original inventor-to-employer assignments (recorded 2017-06-13 for Kottas and 2018-08-21 for Roumeliotis, both to Regents of the University of Minnesota), with no reel/frame or correspondent retrievable to fully chart the chain.

Justification: This is best read as a clean, university-owned chain, not an NPE chain. Both recorded events move rights from the inventors to their employer (UMN), the original assignee; there is no downstream transfer to any LLC, aggregator, or asserter, and neither assignee matches any public NPE list. Zero of the eight NPE signals are present on the available record — the only two calls I cannot make are the correspondent-related ones (signals 3 and the reel/frame fields), which are missing data, not adverse findings. Anyone relying on this should confirm the reel/frame and correspondent columns at https://assignmentcenter.uspto.gov/ before treating the chain as complete.

Verification link: https://assignmentcenter.uspto.gov/ — search by patent number 9709404 (or application 15/130,736); or https://assignment.uspto.gov/patent/index.html.

Open items to close (not fabrications — explicitly unknown):

1. Reel/frame numbers for both recordings.
2. Execution dates (distinct from the recording dates shown).
3. Correspondent of record for each recording (the key repeat-player tell).
4. Any confirmatory license or additional recording not surfaced in Google Patents legal events.

I'll search for the patent and its cited prior art references.

Let me pull the full front-page "References Cited" list from the patent record.

Scope and source note before the tables

I ran multiple searches against the Google Patents record, the patent's own front-page PDF, and third-party reprints (Justia, RPX Insight) for US 9,709,404 B2. I was able to retrieve the backward citation list ("References Cited") only in fragments — the PDF front page and Google Patents' "Patent Citations" table — not as one clean authoritative list. I hit the tool step limit before I could pull the complete OCR of column 1 of the printed patent. Everything below is reconstructed from those fragments; entries marked “⚠” are ones I could not fully verify. This is not a substitute for the certificate-of-correction / printed front page.

Do not conflate 9709404 with unrelated IDs the same searches return: Brazilian application BR 9709404 A (a 1997 Procter & Gamble detergent filing), UK patent application 9709404.9, NSF award SBR‑9709404, and retail/transaction codes "9709404." None is US 9,709,404. Per your strict rule, I am reporting these as-found and not merging them.

1. Target patent (verified anchor)

Because the effective filing date is 2015‑04‑17, AIA §102(a)(1) / (a)(2) governs. That matters for the date analysis in §6.

2. An important framing point about "each patent citation"

Two things routinely get mislabeled on aggregator sites, and I want to be explicit:

• "References Cited" (backward citations) = the IDS/examiner list on the '404's own front page. These are the candidate prior art. This is what you asked for.
• "Cited By" (forward citations) = later patents that cite the '404 (e.g., the 2016–2024 UMN family, Fyusion, Qualcomm, Oxford). These are NOT prior art to the '404. Google Patents mixes both in one rendering, and the first search hit I got was largely forward citations. I separate them below.

Also: a reference appearing on the front page is not proof it anticipates. Most were considered and the patent still issued. On the face of this record, none of the cited references discloses the full three‑step claim‑1/10/17 limitation set (classify by window position → EKF state update using both feature sets → covariance update using only the older set). The real §102 question is therefore about individual limitations, and the honest answer is that the cited art is background/cumulative, with the closest items being the inventors' own earlier work.

3. Table A — U.S. patents cited on the face of the '404

Citations as retrieved (⚠ = date/title not independently verified; OCR of the PDF front page is imperfect). "Anticipates?" = my analysis against claims 1 / 10 / 17 (and dependent 2–9, 11–16, 18–19).

4. Table B — U.S. patent-application publications cited

5. Table C — Key non-patent literature cited (selection, highest relevance first)

From the front-page "Other Publications" and the Justia reprint (Justia 12529563 listing):

⚠ Several entries in this literature list (e.g., Bloesch 2017, Burri EuRoC 2016) post-date the '404 priority and are not prior art at all — they appear in later family members' citation lists, not necessarily in the '404's own IDS. Do not treat the Justia list as co-extensive with the '404's front page.

6. §102 date analysis (why most of this can't even be §102 art)

Because the '404 is an AIA patent (priority 2015‑04‑17):

• §102(a)(1) (public use/on-sale/printed publication): only material public before 2015‑04‑17 qualifies. → A1–A4, B1–B11 and the pre‑2015 NPL qualify; the Kottas ISRR tech report qualifies (subject to the §102(b)(1)(A) inventor grace period).
• §102(a)(2) (earlier-filed US patent/app that later publishes): A7 (US 9,243,916, eff. 2013), B12–B15 (WNM family) qualify on effective filing date. However, they are commonly owned by the Regents of the University of Minnesota with the '404 → the §102(b)(2)(C) common-ownership exception very likely disqualifies them as art.** The same reasoning applies to B17 (2016/0327395) and B16, though B16 is the '404's own publication and is never art to itself.
• Post‑priority filings (B14, B15, B16, B17, and any 2016+ NPL) are not prior art to the '404 unless they meet §102(a)(2)'s effective-filing test, and even then the common-ownership exception likely applies.

Net: the legally usable prior art is essentially A1–A4, B1–B11, and pre‑April‑2015 NPL (with the Kottas tech report subject to the grace-period exception).

7. Bottom line on anticipation

No cited reference — individually or as a charted primary reference — appears to anticipate claims 1, 10, or 17. The specific novel core of the '404 is the three-step sequence: (i) classify features by their position/pose within the sliding window into an older and newer set; (ii) apply an EKF to update state using all features (both sets); then (iii) update covariance using only the older/mature set. None of the retrieved references recites this decoupling:

• US 2009/0248304 A1 (Roumeliotis) and US 9,243,916 B2 (Roumeliotis) — closest on the vision-aided sliding-window EKF, but silent on the window-position classification and the separate covariance-update step; also commonly owned (§102(b)(2)(C)).
• US 2008/0167814 A1 (Samarasekera) — closest third-party reference; silent on the same limitations.
• Kottas ISRR tech report — the most dangerous as a §102(a)(1) reference on its face, but (a) it is the inventors' own disclosure (grace-period exception) and (b) needs a full-text element-by-element comparison I could not perform here.

If you need a definitive anticipation opinion, the two references to chart first are US 9,709,404's own Kottas ISRR tech report and US 9,243,916 B2 / US 2009/0248304 A1, comparing each element of claim 1 against the respective disclosures.

8. Uncertainty flags (read before relying on this)

1. The front-page citation list was only partially retrieved. Some entries (⚠) may be incomplete, mis-dated, or OCR-garbled; the "661" class/art-unit and a few titles could not be confirmed. Get the certified printed front page or the Doc 900/reexam-certified copy before treating Table A/B as exhaustive.
2. A3/A1 titles and the Evolution Robotics family mapping (US 7,015,831 / US 7,162,338 / US 2004/0167667) are inferred and should be reconciled against the actual PDF.
3. Google Patents' "Patent Citations" rendering mixes backward and forward citations. Entries B14–B17 and the 2016+ NPL are largely forward/family citations, not necessarily on the '404's own IDS.
4. Claim 17's literal text remains unverified (carried forward). Its scope statements are therefore provisional.
5. No litigation/PTAB/CAFC activity was found for this patent in the earlier section; that conclusion was "no evidence located," and the same caveat applies here.
6. Do not auto-correct: the Alphanumeric IDs BR 9709404, UK 9709404.9, NSF SBR‑9709404, and the various "9709404" retail/transaction codes are distinct from US 9,709,404 and are reported as found.

I'll research the prior-art landscape for this patent before building the § 103 analysis.

Let me retrieve the "Prior Art" / cited-references sections and the key cited-by documents for this patent.

Let me gather the "Cited By" list, the NPL references, and the closely-related sibling prior art (the INVF patent).

Obviousness Analysis — U.S. Patent No. 9,709,404 B2 under 35 U.S.C. § 103

Scope note: This builds on the prior sections. I do not repeat the bibliographic record, claim summary, or the litigation/PTAB findings (none located for the '404). What follows analyzes § 103 on the face of the patent, its own "Prior Art"/"References Cited" content (US9709404B2 Google Patents; front page reproduced at the granted-PDF image), and the art it explicitly names.

1. Legal framework and the "Prior Art section" I am working from

Effective filing date / AIA status. The '404 was filed 2016-04-15 with benefit of provisional 62/149,194 filed 2015-04-17. It is an AIA patent. Prior art therefore must predate the effective filing date (2015-04-17, if the provisional supports the claims) either as a § 102(a)(1) public disclosure, or as a § 102(a)(2) U.S. patent/publication/PCT-designating-US application with an earlier effective filing date. Note the inventor-origin and common-ownership exceptions of § 102(b)(1)(A)/(b)(2)(A),(C) can remove several of the self-cited Roumeliotis documents from the prior art — this materially matters below.

POSITA. A person of ordinary skill here would hold an M.S./Ph.D. in EE, CS, or robotics (or equivalent) with ~2+ years in estimation theory (Kalman filtering), computer vision, and inertial navigation — consistent with the field of the named references.

What the patent's own Prior Art content gives us (confirmed from the front page).

Additional NPL (Bloesch ROVIO/IJRR; Chiu sliding-window factor graphs; Diel thesis; Li/Mourikis; MSCKF) appears in the related-family record — see the Justia reference list returned for UMN VINS family members. Caveat: I retrieved only part of the NPL list; some items below are drawn from the family record rather than the '404's own front page and are flagged as such.

Critical date traps.

• Bloesch, "Robust Visual Inertial Odometry Using a Direct EKF-Based Approach" (IROS 2015-09-28) and Bloesch, "Iterated Extended Kalman Filter Based VIO…" (IJRR 2017) are after the 2015-04-17 priority date. They are not § 102 prior art to the '404 — useful only as evidence of the state of the art if the priority date is successfully challenged.
• US 9,658,070 ("Inverse sliding-window filters for vision-aided inertial navigation systems," UMN, filed 2014-07-11) is § 102(a)(2) art by its earlier effective filing date (assuming no § 102(b)(2) exception applies), despite issuing 2017-05-23.
• The many Roumeliotis self-citations are prior art only if the § 102(b) exceptions don't apply — an issue the record does not resolve.

2. Independent claims 1/10/17 — element map

Claim 1 (system) / claim 10 (method) / claim 17 (CRM; preamble confirmed: "A non-transitory computer-readable storage device comprising program code to cause a processor to perform the operations of:" — RPX claim text).

Reading of the invention. The '404's own admission frames the problem and the solution: conventional EKF "do[es] not allow re-processing the visual and/or inertial measurements within the optimization window," whereas INVF "allows re-processing all inertial and visual measurements" but "typically require[s] high (e.g., 64-bit or more) precision." The claim's two-step split (state update = all data; covariance update = older/mature data only) is exactly the proposed reconciliation. A Petitioner would argue this split is the predictable result of combining (i) the re-processing accuracy benefit known from INVF/fixed-lag smoothing with (ii) the covariance-based EKF prior that preserves numerical stability and constant-time operation.

3. Combinations rendering the claims obvious

Combination A (primary): MSCKF + Sibley SWF + INVF + iterated-EKF update

• US 2009/0248304 A1 / Mourikis & Roumeliotis ICRA 2007 — sliding-window EKF with pose states, covariances, and multi-state feature constraints; each feature absorbed once, at loss.
• Sibley et al. 2010 (Wiley) — sliding window with delayed-state marginalization; explicitly notes that with a one-step window the solution "matches the extended Kalman filter," and that window size trades accuracy against constant-time operation.
• US 9,658,070 / US 2015/0369609 A1 (INVF) — teaches re-processing all visual/inertial measurements across overlapping epochs of a sliding window (the "re-linearize/re-process" capability claim 1's state update relies on), using a Hessian-based prior.
• Iterated EKF update as Gauss-Newton (Bell & Cathey, IEEE TAC 1993 — cited within Sibley's own reference list) — supplies the re-linearization mechanism for the recursive update.

Motivation / reason to combine (KSR + the '404's own admissions):

1. The '404 itself concedes EKF's delayed feature processing is "a major limitation" and that re-processing "can significantly improve accuracy and robustness." That admission is an explicit motivation.
2. INVF is known to achieve the re-processing benefit; its known drawback (Hessian condition number ~10⁹ → 64-bit requirement) motivates retaining the EKF covariance representation — i.e., the claimed two-step split.
3. Consistency ("each feature absorbed/marginalized exactly once") is a recognized requirement in sliding-window filters (MSCKF; Li & Mourikis 2013; Huang et al. 2011) — supplying the reason the covariance update uses only the about-to-be-marginalized (older) features.
4. Real-time, resource-constrained deployment (Project Tango; cell phones/wearables) motivates the constant-time/bounded-budget framing.

Combination B (alternative): MSCKF 2.0 + sliding-window factor graphs + OCSWF

• Li & Mourikis, IJRR 2013 (SAGE) — EKF-based VIO with a window of poses, consistency/observability analysis.
• Chiu, Williams, Dellaert, Samarasekera & Kumar, ICRA 2013, "Robust vision-aided navigation using sliding-window factor graphs" — non-linear re-optimization over a sliding window (re-processing).
• Huang, Mourikis & Roumeliotis, IROS 2011, "Observability-Constrained Sliding Window Filter for SLAM" — correct uncertainty handling during window marginalization.

Motivation: brings together the EKF covariance machinery (A) with the smoothing/re-optimization benefit (B), addressed to the same problem (bounded-time VINS accuracy). This is the classic "combine a filter with a smoother" rationale.

Combination C: for the dependent claims (see § 4)

Inference caution: No single one of these references, as I can verify from the retrieved record, expressly recites the claimed decoupling of the state update from the covariance update. Combination A is the strongest § 103 theory precisely because it juxtaposes the EKF art (which knows the covariance update) with the INVF art (which knows re-processing), and the '404's own text supplies the motivation. The residual gap is a teaching to separate the two updates — which is where a Petitioner would lean hardest on "predictable result"/"design choice," and is also where the patent owner has the best rebuttal (see § 6).

4. Dependent claims

• Claims 2–3 (mature/immature sets; prior constraints linearly relating immature-pose states to mature-pose states; covariance updated only from mature features + tail IMU + those constraints). These largely restate the specification's implementation of independent claim 1. C-KLAM (US 2014/0333741 A1) discloses feature classification and generating constraints between poses in a sliding window; MSCKF generates multi-state constraints among window poses. Obvious as the natural implementation of the claim-1 split.
• Claim 4 (EKF update using covariance + prior constraints without computing a Hessian matrix). This is the express numerical-stability distinction over INVF. But it is simply the inherent property of choosing the EKF covariance form over the information/Hessian form; a POSA seeking 32-bit efficiency (as the '404 itself motivates) would predictably avoid the Hessian. Strong § 103 candidate.
• Claim 7 (six feature classes: SR, APF, IPF, OSF, OF, NSF). A classification enumeration over the window-position/maturity dimensions. Given MSCKF's track-length handling and C-KLAM's feature classification, the specific six-way partition is an obvious sub-categorization (also mirrors the '404's own Z_P/Z_M/Z_I discussion). Reasonable § 103 rejection with MSCKF + C-KLAM + routine partitioning.
• Claim 8 (resource-aware estimation policy allocating CPU budget across feature classes). Weakest § 103 candidate on the record I retrieved. I did not locate a reference in the '404's cited art expressly teaching CPU-budget-driven measurement selection by feature class. This would require additional art (measurement-scheduling/anytime-estimation literature) or a "design choice for real-time operation" argument tied to Project Tango/mobile constraints. Flag as the least-supported obviousness ground and the most likely point of genuine novelty.
• Claims 5, 6, 9 (map building; laptop/tablet/phone/robot/vehicle; robot/vehicle/mobile/wearable embodiments). Routine application; US 2012/0194517 (Izadi, KinectFusion-style mapping) and US 2008/0167814 (Samarasekera) support mapping/embodiment breadth.

5. Why a POSITA would have combined these (consolidated motivation)

1. Same field, same problem (bounded-time visual-inertial pose estimation in GPS-denied environments) — KSR factor.
2. Known deficiencies with known fixes: EKF delay in processing immature tracks; INVF's 64-bit/Hessian instability; both explicitly discussed in the '404's own background and in Li & Mourikis 2013 / Sibley 2010.
3. Predictable results: re-linearization is known to improve nonlinear estimation; marginalizing a feature once preserves consistency. Recombining known elements (EKF covariance prior + window re-processing) is a classic obvious combination under KSR.
4. Market/implementation pull: real-time mobile/wearable VINS (Project Tango) supplied a concrete reason to seek the efficiency/stability trade-off the claims embody.

6. Counterarguments and where the § 103 case is weak

• No express decoupling teaching. The two-step split (state update from all measurements; covariance update from older-set only) is the heart of claim 1. MSCKF and INVF both update state and covariance jointly; a Petitioner must bridge the gap with reasoning rather than an express disclosure. This is the strongest non-obviousness position.
• Teaching away? Li & Mourikis, IJRR 2013 reports MSCKF 2.0 achieving accuracy/consistency "higher than even an iterative, sliding-window fixed-lag smoother." A patent owner could cite this as a reason to avoid smoothing-style re-processing — a genuine teaching-away argument (though framed as a comparative result, not a discouragement of all re-processing).
• Secondary considerations. The spec asserts 32-bit (ARM NEON, Cortex-A15) viability and ~4× speed-ups and "first proof-of-concept real-time 3D indoor localization on a commercial-grade wearable computer." If supported with evidence and nexus to the claims, these could rebut obviousness — but such evidence is asserted, not yet proven in any proceeding (and, per the earlier sections, no litigation or IPR exists to have tested it).
• Inventor's-own-art problem. Much of the closest art is the inventors' own (US 2014/0316698; US 2014/0333741; US 2015/0369609; US 9,658,070; US 9,243,916). § 102(b)(1)(A)/(b)(2)(A),(C) exceptions may remove these as prior art entirely — which would weaken a Petitioner that builds primarily on the UMN family.

7. Evidence gaps / recommended verification before relying on any of this

1. Obtain the complete "Prior Art" tab for US9709404B2 (Patent Citations, Non-Patent Citations, Cited By, Similar Documents) — my retrieval of the NPL list was partial. The "Cited By" items (e.g., US 10,203,209; US 10,907,971; US 11,940,277; WO2018026544; Qualcomm US 10,267,924; Trifo US 10,395,117) are later documents and not prior art — do not conflate them with § 102 art.
2. Confirm each reference's publication/effective-filing date against the 2015-04-17 date (especially Bloesch IROS 2015, Bloesch IJRR 2017, US 2015/0369609, US 2016/0005164, US 2016/0327395 — several may be date-barred or § 102(b)-excepted).
3. Verify content and exact teaching of the non-Roumeliotis citations I have only by number (5,847,755; 7,015,831; 7,162,338; 8,577,539; 8,996,311; 2008/0265097; 2008/0279421; 2010/0110187; 2010/0220176; 2012/0121161). I did not verify their disclosures and will not assert their content.
4. Pull the '404's file wrapper to see whether an examiner-applied § 103 rejection exists (and on which combination), and whether the applicant made any arguments/admissions that could be used or disavowed.
5. Re-check for IPR/PGR under the '404 (none found earlier) — a Petitioner would likely target claim 1 via Combination A, and claim 8 via added scheduling art.

Bottom line: On the retrieved record, independent claims 1/10/17 are vulnerable to a § 103 challenge built on MSCKF (US 2009/0248304) + Sibley's sliding-window marginalization + the INVF art (US 9,658,070 / US 2015/0369609), with the '404's own background admissions supplying the motivation; dependent claims 2–7 and 9 fall with the same rationale, and claim 4 (no Hessian) is essentially inherent to the EKF-covariance choice. Claim 8's CPU-budget estimation policy is the least-supported ground and the most plausible locus of non-obvious subject matter. All of this is provisional absent the full Prior Art tab, the file wrapper, and date verification, and none of it has been tested in any litigation or PTAB proceeding (none located).