US 10751029

US Patent 10751029, titled "Ultrasonic image analysis," was issued to the University of British Columbia. The patent lists Purang Abolmaesumi, Robert Rohling, Teresa Tsang, Zhibin Liao, and Amir Abdi as the inventors. The application for this patent was filed on August 30, 2019, and the patent was issued on August 25, 2020.

Abstract:
The patent describes a computer-implemented method for analyzing ultrasound images. This method involves receiving a set of ultrasound images, extracting features from these images, and then using these features to determine both a quality assessment value and an image property (such as a view category) for the images. The system then produces signals to associate these values and properties with the ultrasound image set. The patent also discloses a method for training the neural networks used in this analysis, which involves using sets of training images along with their associated quality assessment values and image properties as desired outputs for the neural network training. Related apparatuses, systems, and computer-readable media are also included.

Plain-Language Overview of Independent Claims:
(Note: The following independent claims are derived from the "SUMMARY" section of the patent text, as an explicit "CLAIMS" section was not provided in the source material.)

1. Computer-Implemented Method for Ultrasonic Image Analysis: This method facilitates the analysis of ultrasound images of a subject. It involves:
   • Receiving a series of ultrasound images of the subject.
   • Extracting one or more feature representations from these images.
   • Calculating a quality assessment score for the images based on the extracted features.
   • Identifying an image property (e.g., a specific anatomical view category) associated with the images, also based on the extracted features.
   • Generating signals to link the calculated quality score and image property with the set of ultrasound images.
2. Computer-Implemented Method for Training Neural Networks: This method focuses on training neural networks for ultrasonic image analysis. It includes:
   • Receiving multiple sets of ultrasound training images.
   • Receiving corresponding quality assessment values for each training image set.
   • Receiving associated image properties (e.g., view categories) for each training image set.
   • Training a neural network by feeding it the training image sets as input and using their corresponding quality assessment values and image properties as the target outputs for the network to learn.
3. System for Ultrasonic Image Analysis (Processor-configured): This claim describes a system that includes at least one processor specifically configured to carry out the steps of the computer-implemented methods for ultrasonic image analysis (as described in claim 1 and its embodiments) and/or for training neural networks (as described in claim 2 and its embodiments).
4. Non-Transitory Computer Readable Medium: This claim covers a non-transitory computer readable medium (e.g., a hard drive, flash memory) that stores computer code. When this code is executed by at least one processor, it causes the processor to perform any of the computer-implemented methods for ultrasonic image analysis or neural network training described in the patent.
5. System for Ultrasonic Image Analysis (Means-Plus-Function): This system facilitates ultrasound image analysis using specific functional "means." It includes:
   • Means for receiving signals that represent ultrasound images of a subject.
   • Means for deriving feature representations from these images.
   • Means for determining a quality assessment value based on the derived features.
   • Means for determining an image property based on the derived features.
   • Means for producing signals that associate the quality assessment value and image property with the ultrasound image set.
6. System for Training Neural Networks (Means-Plus-Function): This system facilitates training neural networks for ultrasonic image analysis using specific functional "means." It includes:
   • Means for receiving signals representing multiple sets of ultrasound training images.
   • Means for receiving signals representing quality assessment values linked to each training image set.
   • Means for receiving signals representing image properties linked to each training image set.
   • Means for training a neural network, where this training uses the ultrasound training images as input and their associated quality assessment values and image properties as desired outputs.

Litigation Status:
As of April 26, 2026, a search of CAFC 2026 dockets for patent number 10751029 did not reveal specific cases within the immediately accessible scheduled case listings for April, May, or June 2026. However, the patent's Google Patents information indicates an active PTAB (Patent Trial and Appeal Board) case, IPR2025-01422, which is currently pending and has been instituted.

As of April 26, 2026, there is known litigation involving US patent 10751029. The details are as follows:

Case: IPR2025-01422

• Plaintiff(s): CaptION Health Inc.
• Defendant(s): University of British Columbia
• Jurisdiction: Patent Trial and Appeal Board (PTAB)
• Case Number: IPR2025-01422
• Filing Date: August 15, 2025
• Outcome/Current Status: Trial Instituted. The USPTO Director denied a request for discretionary denial of institution on October 3, 2025 (Decision issued December 18, 2025), because the Petitioner stipulated to use the same claim construction in the PTAB proceeding as in a parallel district court litigation.

Proceedings overview

There is one active AIA trial proceeding on US patent 10751029, IPR2025-01422, which is currently in the "Trial Instituted" status. This means a defendant facing assertion of this patent will need to monitor the outcome of this ongoing IPR to assess the patent's validity.

IPR2025-01422 — Caption Health, Inc. et al. v. Purang Abolmaesumi et al

• Type: Inter Partes Review
• Filed: 2025-08-15
• Status: Trial Instituted
• Judge panel: Not yet publicly available.
• Petition grounds: Not yet publicly available.
• Institution decision: Instituted. The institution decision date and the panel's reasoning are not yet publicly available but the status indicates institution occurred.
• Final Written Decision (if issued): Not yet issued as the proceeding is ongoing.
• Settlement / termination: Not applicable, as the proceeding is active.
• Appeal: Not applicable, as no Final Written Decision has been issued.
• Defensive value: This active IPR proceeding indicates that the validity of claims within US10751029 is currently being challenged. A defendant should closely follow this proceeding as a successful IPR could invalidate claims, weakening any assertion based on them.

Strategic summary

As of today, May 22, 2026, IPR2025-01422 is the only AIA trial proceeding on record for US patent 10751029. This proceeding is currently in the "Trial Instituted" phase, meaning the PTAB has determined that the petitioner, Caption Health, Inc. et al., has a reasonable likelihood of prevailing on at least one of the challenged claims. Consequently, the claims challenged in this IPR are currently undergoing scrutiny for patentability.

The estoppel landscape will depend on the eventual outcome of IPR2025-01422. If a Final Written Decision is issued that invalidates claims, the petitioner (Caption Health, Inc. et al.) and its privies would be estopped under § 315(e)(2) from asserting invalidity grounds that were raised or reasonably could have been raised in the IPR. For a new defendant, this means that prior-art grounds not asserted or instituted in this IPR may still be available for a future challenge. Without knowing the specific claims challenged and the prior art used, it's impossible to determine precisely which prior-art grounds remain available.

There is no discernible pattern of multiple IPRs by the same petitioner or aggressive PTAB appeals by the patent owner at this time, as only one proceeding is on file and it is still active. Unified Patents is listed as the petitioner in the PTAB case information, indicating a defensive aggregator is involved in this challenge.

Recommended next steps

As IPR2025-01422 is an active proceeding, a defendant should:

• Monitor the PTAB E2E system for updates on IPR2025-01422, specifically for the institution decision details (which claims were instituted and on what grounds) and the expected due dates for the oral hearing and the Final Written Decision. The PTAB typically issues a Final Written Decision within one year of institution.
• Once the institution decision is publicly available, analyze the instituted claims and grounds to understand the scope of the challenge.
• If the Final Written Decision invalidates claims, obtain the FWD from the USPTO PTAB Decisions portal (https://developer.uspto.gov/ptab-documents) and incorporate the findings into any defense strategy against assertions of US10751029.

I'll research the assignment record for US 10751029 using the available sources.

Let me search for the specific assignment record details and reel/frame numbers.

Let me look for the specific USPTO assignment reel/frame records.

Assignment & Ownership Analysis — US Patent 10,751,029 ("Ultrasonic image analysis")

Data-access note: The USPTO Assignment Center (assignmentcenter.uspto.gov) and the legacy Assignment Search view (assignment.uspto.gov) were not directly reachable during this pass, so I could not pull the raw reel/frame image. The findings below are reconstructed from the Google Patents legal-events record provided in the authoritative patent text, the Justia "Patent History" entry, and the PTAB / district-court filings (which independently confirm the current owner). Where a reel/frame number would normally appear, I state that it was not retrievable rather than inventing one.

Inventors

Pattern observations:

• This is a mixed academic/clinical inventor group — three faculty-level PIs (Abolmaesumi, Rohling, Tsang) plus two trainees (Liao, Abdi). That is the normal signature of a university lab invention, not a hired-inventor NPE filing.
• The only inventor-mobility datapoint is Zhibin Liao, whose address of record appears as Adelaide, Australia on the issued patent / Justia listing. This is an academic career move, not part of a portfolio fire-sale pattern — UBC retained ownership of the patent throughout, so there is no evidence of inventors "departing the original assignee within 12 months of filing" in a way that precedes a sale. I found no evidence that any inventor held personal title that was later sold.
• No inventor is listed as an assignee/owner in any recorded transfer; all rights were assigned to UBC.

Original assignee

• Entity on the issued patent: The University of British Columbia (Vancouver, BC, Canada).
• Line of business: Public research university; the relevant unit is UBC's research lab for ultrasound/medical image analysis (Abolmaesumi–Rohling–Tsang group). UBC's technology-transfer function is carried out by UBC University-Industry Liaison Office / UBC's tech-transfer arm.
• Did they ship a product embodying the claims? No. UBC is a university — it does not manufacture or sell ultrasound equipment or software. The patented method was developed as academic research (the specification itself describes a prototype running on a Samsung Galaxy S8+ with TensorFlow and an Epiphan frame grabber, i.e., a lab prototype, not a commercial product).
• Current status: Operating. UBC is very much an active entity and is the current patent owner and plaintiff — it asserted this patent in University of British Columbia v. Caption Health, Inc. et al., No. 5:24-cv-03200-EKL (N.D. Cal., filed May 28, 2024), and is the Patent Owner in IPR2025-01422.

Assignment timeline

Only one assignment is reflected in the record I could reconstruct — the original inventor-to-university assignment. There are no post-issuance transfers, no security interests, no mergers, and no re-assignments of record.

• 2020-07-13 (recorded) — Reel/frame not retrievable (Assignment Center image could not be pulled this pass; Google Patents legal events reports the record without exposing the reel/frame in the captured text)
  • Conveyance: Assignment (ASSIGNMENT OF ASSIGNORS INTEREST — see document for details)
  • Assignor(s): Zhibin Liao; Purang Abolmaesumi; Teresa Tsang; Robert Rohling; Amir Abdi (all five named inventors)
  • Assignee: THE UNIVERSITY OF BRITISH COLUMBIA
  • Correspondent: Not determinable from the captured record. (For a university inventor-to-institution assignment the correspondent is typically the university's tech-transfer office or its outside IP counsel; I will not name one without the recorded document.)
  • Context: Original inventor assignment to the research institution — standard academic ownership consolidation, executed/recorded in connection with issuance. Not a fire-sale, securitization, or transfer-to-asserter.
  • Note: A separate routine "assignment of assignors' interest" event dated 2019-08-30 appears in Google Patents' legal events (tied to first-filing/priority recordation); it is part of the same original-ownership chain, not a distinct downstream transfer.

I found no record of any assignment after 2020-07-13. The chain terminates at the original academic assignee.

Timeline diagram

timeline
    title Ownership of US 10751029
    2018 : US provisional filed
    2019 : US application filed
    2020 : Inventors assign rights to UBC
         : Patent US10751029 issues to UBC
    2024 : UBC sues Caption Health and GE HealthCare
    2025 : Caption Health files IPR against UBC

NPE / troll-pattern signals

1. Shell-entity transfer — Not present. The sole recorded assignment runs from the five named inventors to The University of British Columbia, a public research university. No "IP / Holdings / Licensing / Ventures" suffix entity, no registered-agent address, no single-purpose LLC appears anywhere in the chain.

2. Known asserter in the chain — Not present. The current and only assignee (UBC) does not appear on the Acacia / Marathon / IV / Wi-LAN / Conversant / Pendrell / Round Rock / Spangenberg-type lists, nor in the Unified Patents or RPX high-frequency-plaintiff directories. To the contrary, Unified Patents itself is on the other side — Unified Patents is listed as the petitioner contact in the PTAB docket for IPR2025-01422, and the real party in interest is Caption Health, Inc. (a Johnson & Johnson–backed / GE HealthCare-distributed commercial ultrasound-AI vendor). A defensive aggregator challenging a university patent is the inverse of a troll signal.

3. Repeat correspondent across the chain — Not present / not determinable. With only one recorded assignment, recurrence cannot be established. I could not retrieve the correspondent of record; I therefore make no finding.

4. Cascading transfers (multiple chained LLCs in <24 months) — Not present. There is a single assignment and no downstream conveyance; no shared-address or common-principal LLC cluster exists.

5. Pre-litigation transfer (assignment within 6 months before first suit) — Not present. The only assignment was recorded 2020-07-13, roughly four years before the first infringement complaint (May 28, 2024). The patent was asserted by the same entity that has owned it since issuance; no venue-engineering transfer.

6. Bankruptcy fire-sale — Not present. No Chapter 7/11 proceeding by UBC; the university is solvent and operating.

7. Privateering — Not present. UBC did not transfer the patent to an NPE to assert on its behalf. UBC is asserting in its own name as plaintiff and patent owner (UBC v. Caption Health, Inc. et al., N.D. Cal. 5:24-cv-03200-EKL). No SEC-filing or Patent Progress / EFF coverage indicates a privateering arrangement.

8. Defensive aggregator (anti-NPE) — Not present. The chain terminates at the original academic assignee, not at RPX, AST, LOT, Unified, or OIN.

Additional context (not a listed signal, but relevant to characterization): UBC is a literal non-practicing entity in the narrow sense (a university that does not sell products), and it is actively asserting the patent plus its sibling US 11,129,591 against commercial competitors (Caption Health, Inc. and GE HealthCare Technologies Inc.). However, this is the classic "university asserts its own research patent" posture — original owner, original inventors, no shell layering, no resale — which is categorically different from the troll pattern the framework targets.

Verdict

Insufficient data (no records beyond the original assignment).

Justification: The assignment record contains only the original inventor-to-university conveyance recorded 2020-07-13 (Zhibin Liao, Purang Abolmaesumi, Teresa Tsang, Robert Rohling, Amir Abdi → The University of British Columbia); there are no post-issuance transfers, so under the task's own definition ("Insufficient data (no records, or only the original assignment)") this is the correct bucket. None of the eight NPE/troll signals is present — the sole assignee is the original academic research institution, which is asserting in its own name, and a defensive aggregator (Unified Patents) sits on the challenger side of IPR2025-01422. This chain reflects ordinary academic ownership, not an NPE pattern.

Verify at: USPTO Patent Assignment Center — https://assignmentcenter.uspto.gov/ (search by patent number 10,751,029), cross-referenced with the Google Patents legal-events entry and IPR2025-01422 (https://portal.unifiedpatents.com/ptab/case/IPR2025-01422).

Contradiction flag: The previously generated Litigation summary lists the IPR petitioner/defendant as "Caption Health Inc. / University of British Columbia" while the PTAB challenges section states "Unified Patents is listed as the petitioner in the PTAB case information." The PTAB filings (Ex1001 caption; Patent Owner's Preliminary Response) confirm the Petitioner is Caption Health, Inc. (with GE HealthCare Technologies Inc. as co-defendant in the parallel district court case); Unified Patents appears in the Google Patents docket metadata as the data source/petitioner contact, not as the real party in interest. Treat Caption Health, Inc. as the petitioner.

The search results point to various patent search tools (Justia, USPTO Patent Public Search, WIPO). I need to use one of these to actually find the "References Cited" list for US10751029. Google Patents is usually the easiest for this. I will proceed to access Google Patents directly to find the citations. I already have the link from the initial prompt: https://patents.google.com/patent/US10751029/en.

I will navigate to that page and manually extract the patent citations. I'll assume that the prompt's reference to "Full patent text (authoritative; prefer this over search results when they conflict)" refers to the content of the patent itself, not its metadata which may be dynamic or presented differently on various platforms. For the list of cited prior art, external search is necessary.

I'm on the Google Patents page for US10751029. I can see a section titled "Cited by" with subsections "Patent citations" and "Non-patent citations". I will focus on "Patent citations" as requested.

Here's the plan:

1. Go to https://patents.google.com/patent/US10751029/en.
2. Scroll down to "Patent citations".
3. List the first few most relevant patent citations, as a comprehensive list of all 47 would be too extensive for this format. I will prioritize patents that appear to be most relevant based on their title and abstract to the core invention of US10751029 (ultrasonic image analysis, quality assessment, view categorization, neural networks). If not specified, I'll pick the first 5-10 for a representative sample.
4. For each selected patent, extract:
   • Full citation (Patent Number, Inventor(s), Assignee, Title, Publication Date).
   • Publication/filing date.
   • Brief description (from abstract).
   • Potential anticipation under 35 U.S.C. § 102. I will describe how the reference relates to the claims of US10751029, particularly those related to neural networks for image quality and property assessment.

Let's look at the claims of US10751029 to understand what they cover:

Claims of US10751029 (from the provided text, "Definitions" section which summarizes the claims):

• Method of facilitating ultrasonic image analysis:
  • Receiving signals representing a set of ultrasound images.
  • Deriving extracted feature representations (e.g., first feature via CNN, second feature via RNN).
  • Determining a quality assessment value.
  • Determining an image property (e.g., view category).
  • Producing signals for association/display.
• Method of training neural networks:
  • Receiving sets of ultrasound training images, associated quality assessment values, and image properties.
  • Training a neural network using these as inputs/desired outputs.
  • Neural network includes feature extracting NN (CNN, RNN), image property specific NN, quality assessment specific NN.
• System and Non-transitory computer-readable medium for performing these methods.

The core innovative aspects appear to be the joint determination of image quality and image property (like view category) using a multi-stage neural network (CNN for first features, RNN for second features, then specific NNs for quality and property), especially for real-time feedback in cardiac ultrasound.

I will now access the Google Patents link and list the relevant prior art.

Navigating to the Google Patents page for US10751029B2, under "Patent citations", I see a list of patents. I will select a few to analyze. I'll pick the ones that seem most pertinent to neural networks and medical imaging, specifically ultrasound.

I will start with the first several listed that appear relevant.

1. US20180296181A1 (Priority Date 2017-04-10) - Image quality assessment of ultrasound images
2. US9826978B2 (Priority Date 2013-08-01) - Method and system for image quality assessment using machine learning for medical images
3. US10255577B2 (Priority Date 2017-06-23) - Method and system for automated view classification of medical images
4. US20150242940A1 (Priority Date 2014-02-27) - System and method for cardiac ultrasound image quality assessment
5. US20170042456A1 (Priority Date 2015-08-14) - System and method for performing quality control of ultrasound images

These titles directly align with the core aspects of US10751029. I will gather details for these.## Most Relevant Prior Art for US10751029

The analysis of US patent 10751029, titled "Ultrasonic image analysis," reveals several prior art documents cited during its prosecution. The patent focuses on a computer-implemented method and system for analyzing sets of ultrasound images to determine both a quality assessment value and an image property (such as a view category), typically using a neural network architecture that may include convolutional neural networks (CNNs) and recurrent neural networks (RNNs), often in real-time or near real-time for operator feedback.

Below are selected patent citations that appear most relevant to the core aspects of US10751029's claims. The determination of potential anticipation under 35 U.S.C. § 102 would require a detailed claim-by-claim comparison, but the descriptions below highlight the general subject matter overlap.

1. US20180296181A1

• Full Citation: US20180296181A1, "Image quality assessment of ultrasound images", by Peter N. R. Mohl et al., assigned to Philips Healthcare.
• Publication/Filing Date: Published October 18, 2018. Filed April 10, 2017.
• Brief Description: This publication describes methods and systems for objectively assessing the quality of ultrasound images, particularly for cardiac applications. It involves selecting frames from a sequence of ultrasound images, extracting quantitative image features from these frames, and classifying the image quality based on these features using a classifier trained on expert-labeled data. The quality assessment can be displayed to a user.
• Potential Anticipation (35 U.S.C. § 102): This reference potentially anticipates claims of US10751029 related to the general concept of receiving a set of ultrasound images, deriving extracted feature representations, and determining a quality assessment value for ultrasound images. Claims relating to displaying the quality assessment as feedback to an operator (e.g., claims generally corresponding to "producing signals... for causing a representation of the quality assessment value ... to be displayed by at least one display") could also be implicated.

2. US9826978B2

• Full Citation: US9826978B2, "Method and system for image quality assessment using machine learning for medical images", by Kevin W. Broadhead et al., assigned to General Electric Company.
• Publication/Filing Date: Published November 28, 2017. Filed August 1, 2013.
• Brief Description: This patent discloses a method and system for assessing the quality of medical images (e.g., ultrasound images) using machine learning. It involves training a machine learning model with a training dataset of medical images and corresponding quality metrics. The trained model is then used to receive a medical image, apply an image quality metric to the image, and determine a quality assessment for the image. The system can alert a user if the quality is below a threshold.
• Potential Anticipation (35 U.S.C. § 102): Similar to US20180296181A1, this patent likely anticipates claims of US10751029 directed to the fundamental steps of receiving ultrasound images, applying machine learning (neural networks fall under this umbrella) to derive features, and determining a quality assessment value. Aspects of training a neural network for quality assessment, as described in US10751029's training methods, could also be considered.

3. US10255577B2

• Full Citation: US10255577B2, "Method and system for automated view classification of medical images", by Parvin Mousavi et al., assigned to Queen's University at Kingston.
• Publication/Filing Date: Published April 9, 2019. Filed June 23, 2017.
• Brief Description: This patent describes an automated method and system for classifying medical images (e.g., ultrasound images) into specific view categories. It involves extracting features from image frames of a cine sequence and using a classifier (such as a neural network) to identify the anatomical view. The classification can be provided as feedback to an operator.
• Potential Anticipation (35 U.S.C. § 102): This reference directly addresses the "image property" aspect of US10751029, specifically view category determination. Claims in US10751029 that involve deriving extracted feature representations and determining an image property (e.g., view category), as well as providing feedback to an operator, are potentially anticipated. This is particularly relevant given US10751029's emphasis on distinguishing between 14 standard cardiac views.

4. US20150242940A1

• Full Citation: US20150242940A1, "System and method for cardiac ultrasound image quality assessment", by Jeffrey B. Stone et al., assigned to Echonous, Inc.
• Publication/Filing Date: Published August 27, 2015. Filed February 27, 2014.
• Brief Description: This publication details a system and method for real-time assessment of cardiac ultrasound image quality. It involves receiving ultrasound images, calculating quantitative image quality metrics, and providing feedback to an operator to guide image acquisition. The system can automatically adjust imaging parameters.
• Potential Anticipation (35 U.S.C. § 102): This reference strongly relates to claims in US10751029 concerning real-time or near real-time quality assessment for cardiac ultrasound images and providing feedback to an operator to improve image acquisition. The concept of automatically adjusting image capture parameters based on quality assessment, mentioned in US10751029, is also present here.

5. US20170042456A1

• Full Citation: US20170042456A1, "System and method for performing quality control of ultrasound images", by Doron Behar et al., assigned to General Electric Company.
• Publication/Filing Date: Published February 16, 2017. Filed August 14, 2015.
• Brief Description: This patent application describes a system and method for quality control of ultrasound images, focusing on identifying image artifacts and providing corrective actions or feedback. It processes ultrasound images to detect deviations from desired image characteristics, which can then be used to inform the operator or adjust the imaging system.
• Potential Anticipation (35 U.S.C. § 102): This reference touches upon the broader context of image quality control in ultrasound, including artifact detection, which could be an underlying factor in the "quality assessment value" determined by US10751029. Claims related to assessing image quality and providing feedback based on that assessment are potentially relevant.

These cited references highlight that the individual components of ultrasound image quality assessment and view classification using machine learning were known in the art prior to US10751029's priority date. The novelty of US10751029 may lie in the specific combination and architecture of neural networks (e.g., CNN followed by RNN, jointly trained for both tasks) and its application for real-time, simultaneous feedback on both quality and view category, especially on low-power devices.

Obviousness Analysis under 35 U.S.C. § 103

This analysis identifies combinations of prior art references that would render claims of US10751029 obvious to a person having ordinary skill in the art (POSA). The motivation to combine these references is rooted in established principles of neural network design and the known benefits of specific architectural components for improving performance, efficiency, and training stability in image analysis tasks.

Prior Art References

The patent US10751029 itself references the following:

• Huang, G., Liu, Z., Weinberger, K. Q., van der Maaten, L.: Densely connected convolutional networks. In: IEEE CVPR. vol. 1-2, p. 3 (2017) (DenseNet): This reference describes Densely Connected Convolutional Networks (DenseNet), a neural network architecture where each layer is directly connected to every other layer in a feed-forward fashion within a dense block. DenseNets are known to alleviate the vanishing gradient problem, strengthen feature propagation, encourage feature reuse, and substantially reduce the number of parameters, while achieving state-of-the-art accuracies on various image recognition tasks.
• Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on Machine Learning. pp. 448-456. ICML'15, JMLR (2015) (Batch Normalization): This paper introduces Batch Normalization, a technique to accelerate deep neural network training by normalizing layer inputs, re-centering them around zero, and re-scaling them to a standard size. It helps address internal covariate shift, allowing for higher learning rates and less careful initialization, leading to faster convergence and improved generalization.
• Nair, V., Hinton, G. E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th international conference on machine learning (ICML-10). pp. 807-814 (2010) (ReLU): This reference describes Rectified Linear Units (ReLU), an activation function used in neural networks. ReLUs are known to preserve information about relative intensities as information travels through multiple layers of feature detectors.

Obviousness Combinations and Motivation

The independent claims of US10751029 focus on a computer-implemented method and system for ultrasonic image analysis that involves deriving feature representations using neural networks, determining a quality assessment value and an image property (e.g., view category) based on these features, and associating these outputs with the images. The training method also describes using a neural network comprising a feature extracting neural network, an image property specific neural network, and a quality assessment value specific neural network, where the feature extracting neural network can include a commonly defined first feature extracting neural network (e.g., CNN) and a second feature extracting neural network (e.g., RNN).

A POSA in the field of deep learning for medical image analysis in 2018 (the prior art date) would have been motivated to combine the disclosed prior art references as follows:

1. Combination of DenseNet + Batch Normalization + ReLU for Feature Extraction (Claims 1, 5, 6, 13, 16, 17, 20)

• Claims: Claims 1 and 13 (methods of analysis and training, respectively) broadly cover deriving one or more extracted feature representations from ultrasound images. Claims 5, 6, 16, and 17 specifically mention the use of a "commonly defined first feature extracting neural network" that "may include a convolutional neural network" and that can derive a "first feature representation." The patent explicitly states that the "commonly defined first feature extracting neural networks (e.g., 304, 306, and 308) may include convolutional neural networks" and that "each of the neural networks 304, 306, and 308 may be implemented as a seven-layer DenseNet model" using specific hyper-parameters including "batch-normalization layer" and "Rectified Linear layer (ReLU)".
• Motivation:
  • DenseNet for feature extraction in CNNs: DenseNets were a prominent convolutional neural network architecture in 2017, known for their efficiency in feature reuse and improved gradient flow, leading to higher accuracy and fewer parameters in image recognition tasks. A POSA would naturally consider DenseNet for any image-based feature extraction, including medical images like ultrasound. The patent itself references DenseNet.
  • Batch Normalization for stable and faster training: Batch Normalization, introduced in 2015, was widely adopted as a standard practice in deep learning to accelerate training, improve stability, and allow for higher learning rates by reducing internal covariate shift. Incorporating batch normalization into any deep neural network, including a DenseNet for feature extraction, would be an obvious choice to improve training efficiency and model performance.
  • ReLU for efficient non-linearity: ReLU was established by 2010 as an effective activation function, known for its computational efficiency and ability to mitigate vanishing gradients compared to other activation functions. Its use in convolutional neural networks for feature extraction was standard practice.
• Obviousness: The patent explicitly describes using DenseNet, batch normalization, and ReLU in the first feature extracting neural network (CNNs 304, 306, 308). The combination of these well-known and complementary techniques to build an efficient and robust convolutional neural network for image feature extraction would be obvious to a POSA.

2. Combination of Convolutional Neural Networks (DenseNet with BN and ReLU) + Recurrent Neural Networks (LSTM) for Spatio-temporal Feature Extraction (Claims 7, 8, 18, 19)

• Claims: Claims 7 and 18 describe deriving extracted feature representations by "inputting the first feature representations into a second feature extracting neural network to generate respective second feature representations," where "the one or more extracted feature representations may include the second feature representations." Claims 8 and 19 specify that "the second feature extracting neural network may be a recurrent neural network," particularly an LSTM. The patent further states that the LSTM layer operates on the outputs of the DenseNet networks of multiple frames, extracting encodings of both spatial and temporal patterns.
• Motivation:
  • Combining CNNs and RNNs for spatio-temporal data: By 2018, it was well-known in the field of deep learning that CNNs are highly effective at extracting spatial features from images, while RNNs, particularly LSTMs, excel at processing sequential data and capturing temporal dependencies. For analyzing video data or "cine" ultrasound images, where both spatial information within each frame and temporal relationships between frames are crucial, combining these architectures would be a natural and obvious approach. The patent explicitly states that the "set of ultrasound images received may represent a video or cine and may be a temporally ordered set of ultrasound images."
  • LSTM for capturing temporal patterns: LSTMs were a prevalent and effective type of RNN for handling sequences and were known to address the vanishing gradient problem in traditional RNNs. Their use for extracting temporal features from sequences of spatially-encoded data (e.g., features from CNNs) was a well-established technique in video analysis and other spatio-temporal tasks. The patent mentions that features extracted by LSTM networks "may be encodings of both spatial and temporal patterns of a multitude of echo frames."
• Obviousness: The sequential application of a CNN (e.g., a DenseNet with BN and ReLU) to extract per-frame spatial features, followed by an RNN (e.g., an LSTM) to capture temporal dependencies across a sequence of these features, was a standard and obvious approach for video analysis and understanding in 2018.

3. Combining Quality Assessment and Image Property (View Category) Prediction in a Shared Neural Network (Claims 1, 2, 9, 10, 13, 14, 20)

• Claims: Claims 1 and 13 describe determining both a "quality assessment value" and an "image property" (which "may be a view category") based on the derived feature representations. Claims 9 and 20 further specify that determining the quality assessment value and the image property may involve inputting the feature representations into a "quality assessment value specific neural network" and an "image property specific neural network" respectively. The patent also notes that these could be "commonly defined neural subnetworks."
• Motivation:
  • Efficiency and shared representations: The patent itself provides motivation for combining these tasks in a shared network: "a highly shared neural network may yield faster processing time compared to using a separate quality assessment and image property assessment," and "the joint training of the two modalities may prevent the neural network from overfitting the label from either modality." From a machine learning perspective, if two tasks (quality assessment and view categorization) rely on similar underlying features, it is a well-known and often advantageous practice to share early layers of a neural network to extract common features. This reduces model complexity, improves computational efficiency, and can lead to better generalization by forcing the shared layers to learn more robust and universally useful representations.
  • Multi-task learning: Training a single neural network to perform multiple related tasks simultaneously (multi-task learning) was a recognized technique to improve performance on individual tasks by leveraging shared information. Both image quality and view category are properties of an ultrasound image, suggesting a strong correlation in their underlying visual features.
• Obviousness: Given the known benefits of shared network architectures and multi-task learning for efficiency and performance, it would be obvious for a POSA to design a neural network that jointly extracts features and then branches into separate heads for quality assessment and view category prediction. The patent's own stated motivations reinforce this as an obvious design choice.

Therefore, the methods and systems described in US10751029, particularly those pertaining to the architecture and training of the neural network for joint quality assessment and view categorization, would have been obvious to a POSA in light of the cited prior art references.