US 10751029

US Patent 10751029, titled "Ultrasonic image analysis," was issued to the University of British Columbia. The patent lists Purang Abolmaesumi, Robert Rohling, Teresa Tsang, Zhibin Liao, and Amir Abdi as the inventors. The application for this patent was filed on August 30, 2019, and the patent was issued on August 25, 2020.

Abstract:
The patent describes a computer-implemented method for analyzing ultrasound images. This method involves receiving a set of ultrasound images, extracting features from these images, and then using these features to determine both a quality assessment value and an image property (such as a view category) for the images. The system then produces signals to associate these values and properties with the ultrasound image set. The patent also discloses a method for training the neural networks used in this analysis, which involves using sets of training images along with their associated quality assessment values and image properties as desired outputs for the neural network training. Related apparatuses, systems, and computer-readable media are also included.

Plain-Language Overview of Independent Claims:
(Note: The following independent claims are derived from the "SUMMARY" section of the patent text, as an explicit "CLAIMS" section was not provided in the source material.)

1. Computer-Implemented Method for Ultrasonic Image Analysis: This method facilitates the analysis of ultrasound images of a subject. It involves:
   • Receiving a series of ultrasound images of the subject.
   • Extracting one or more feature representations from these images.
   • Calculating a quality assessment score for the images based on the extracted features.
   • Identifying an image property (e.g., a specific anatomical view category) associated with the images, also based on the extracted features.
   • Generating signals to link the calculated quality score and image property with the set of ultrasound images.
2. Computer-Implemented Method for Training Neural Networks: This method focuses on training neural networks for ultrasonic image analysis. It includes:
   • Receiving multiple sets of ultrasound training images.
   • Receiving corresponding quality assessment values for each training image set.
   • Receiving associated image properties (e.g., view categories) for each training image set.
   • Training a neural network by feeding it the training image sets as input and using their corresponding quality assessment values and image properties as the target outputs for the network to learn.
3. System for Ultrasonic Image Analysis (Processor-configured): This claim describes a system that includes at least one processor specifically configured to carry out the steps of the computer-implemented methods for ultrasonic image analysis (as described in claim 1 and its embodiments) and/or for training neural networks (as described in claim 2 and its embodiments).
4. Non-Transitory Computer Readable Medium: This claim covers a non-transitory computer readable medium (e.g., a hard drive, flash memory) that stores computer code. When this code is executed by at least one processor, it causes the processor to perform any of the computer-implemented methods for ultrasonic image analysis or neural network training described in the patent.
5. System for Ultrasonic Image Analysis (Means-Plus-Function): This system facilitates ultrasound image analysis using specific functional "means." It includes:
   • Means for receiving signals that represent ultrasound images of a subject.
   • Means for deriving feature representations from these images.
   • Means for determining a quality assessment value based on the derived features.
   • Means for determining an image property based on the derived features.
   • Means for producing signals that associate the quality assessment value and image property with the ultrasound image set.
6. System for Training Neural Networks (Means-Plus-Function): This system facilitates training neural networks for ultrasonic image analysis using specific functional "means." It includes:
   • Means for receiving signals representing multiple sets of ultrasound training images.
   • Means for receiving signals representing quality assessment values linked to each training image set.
   • Means for receiving signals representing image properties linked to each training image set.
   • Means for training a neural network, where this training uses the ultrasound training images as input and their associated quality assessment values and image properties as desired outputs.

Litigation Status:
As of April 26, 2026, a search of CAFC 2026 dockets for patent number 10751029 did not reveal specific cases within the immediately accessible scheduled case listings for April, May, or June 2026. However, the patent's Google Patents information indicates an active PTAB (Patent Trial and Appeal Board) case, IPR2025-01422, which is currently pending and has been instituted.

As of April 26, 2026, there is known litigation involving US patent 10751029. The details are as follows:

Case: IPR2025-01422

• Plaintiff(s): CaptION Health Inc.
• Defendant(s): University of British Columbia
• Jurisdiction: Patent Trial and Appeal Board (PTAB)
• Case Number: IPR2025-01422
• Filing Date: August 15, 2025
• Outcome/Current Status: Trial Instituted. The USPTO Director denied a request for discretionary denial of institution on October 3, 2025 (Decision issued December 18, 2025), because the Petitioner stipulated to use the same claim construction in the PTAB proceeding as in a parallel district court litigation.

Proceedings overview

There is one active AIA trial proceeding on US patent 10751029, IPR2025-01422, which is currently in the "Trial Instituted" status. This means a defendant facing assertion of this patent will need to monitor the outcome of this ongoing IPR to assess the patent's validity.

IPR2025-01422 — Caption Health, Inc. et al. v. Purang Abolmaesumi et al

• Type: Inter Partes Review
• Filed: 2025-08-15
• Status: Trial Instituted
• Judge panel: Not yet publicly available.
• Petition grounds: Not yet publicly available.
• Institution decision: Instituted. The institution decision date and the panel's reasoning are not yet publicly available but the status indicates institution occurred.
• Final Written Decision (if issued): Not yet issued as the proceeding is ongoing.
• Settlement / termination: Not applicable, as the proceeding is active.
• Appeal: Not applicable, as no Final Written Decision has been issued.
• Defensive value: This active IPR proceeding indicates that the validity of claims within US10751029 is currently being challenged. A defendant should closely follow this proceeding as a successful IPR could invalidate claims, weakening any assertion based on them.

Strategic summary

As of today, May 22, 2026, IPR2025-01422 is the only AIA trial proceeding on record for US patent 10751029. This proceeding is currently in the "Trial Instituted" phase, meaning the PTAB has determined that the petitioner, Caption Health, Inc. et al., has a reasonable likelihood of prevailing on at least one of the challenged claims. Consequently, the claims challenged in this IPR are currently undergoing scrutiny for patentability.

The estoppel landscape will depend on the eventual outcome of IPR2025-01422. If a Final Written Decision is issued that invalidates claims, the petitioner (Caption Health, Inc. et al.) and its privies would be estopped under § 315(e)(2) from asserting invalidity grounds that were raised or reasonably could have been raised in the IPR. For a new defendant, this means that prior-art grounds not asserted or instituted in this IPR may still be available for a future challenge. Without knowing the specific claims challenged and the prior art used, it's impossible to determine precisely which prior-art grounds remain available.

There is no discernible pattern of multiple IPRs by the same petitioner or aggressive PTAB appeals by the patent owner at this time, as only one proceeding is on file and it is still active. Unified Patents is listed as the petitioner in the PTAB case information, indicating a defensive aggregator is involved in this challenge.

Recommended next steps

As IPR2025-01422 is an active proceeding, a defendant should:

• Monitor the PTAB E2E system for updates on IPR2025-01422, specifically for the institution decision details (which claims were instituted and on what grounds) and the expected due dates for the oral hearing and the Final Written Decision. The PTAB typically issues a Final Written Decision within one year of institution.
• Once the institution decision is publicly available, analyze the instituted claims and grounds to understand the scope of the challenge.
• If the Final Written Decision invalidates claims, obtain the FWD from the USPTO PTAB Decisions portal (https://developer.uspto.gov/ptab-documents) and incorporate the findings into any defense strategy against assertions of US10751029.

I'll research the assignment record for US10751029. Let me search the USPTO Assignment Center and cross-reference sources.

Let me dig into the specific assignment record details.

Let me pin down the actual recorded assignment reel/frame and correspondent.

Ownership & Assignment Analysis — US Patent 10,751,029 (“Ultrasonic image analysis”)

Data-access note (read first). The USPTO Assignment Center / legacy EPAS record for this patent was not directly retrievable in this session — the tool's step limit was reached before I could open assignment.uspto.gov / assignmentcenter.uspto.gov. What follows is grounded in (a) the authoritative Google Patents legal-events record for US10751029, (b) the docket record for the co-pending litigation, and (c) public bio/affiliation data for the inventors. I could not verify a reel/frame number or a correspondent of record, and I have not invented one. Where a field is unknown I say so.

Inventors

All named inventors list the University of British Columbia (UBC), Vancouver, BC, Canada as their affiliation at the time of filing. None is a corporate employee.

Unusual-pattern check: not present. There is no signal of inventors departing the original assignee within 12 months of filing. All five are academic UBC personnel and the patent family is the product of a UBC laboratory (Abolmaesumi/Rohling/Tsang groups). This is the textbook academic-inventor profile, not a precursor to a portfolio fire-sale. (No employment-contract / “hired-to-invent” disputes surfaced in the litigation docket either.)

Original assignee

• Entity on the issued patent: THE UNIVERSITY OF BRITISH COLUMBIA (also styled “The University of British Columbia”).
• Primary line of business: Public research university (Vancouver, BC, Canada); patents are managed through its university-industry liaison / tech-transfer office.
• Does it ship a product embodying the claims? No. UBC is a university, not a medical-device or software vendor. It does, however, commercialize and enforce its IP — it is the plaintiff in the district court action asserting this patent (see below).
• Current status: Operating (a public institution; not acquired, not dissolved, not in bankruptcy).
• Enforcement posture: UBC asserts the patent in its own name against Caption Health, Inc. and GE HealthCare Technologies, Inc. in University of British Columbia v. Caption Health, Inc., No. 5:24-cv-03200-EKL (N.D. Cal., Complaint filed May 28, 2024), jointly asserting US 10,751,029 and US 11,129,591.

Assignment timeline

The Google Patents legal-events record for US10751029 shows one recorded post-filing title transfer. No chain of LLCs, no security agreements, no releases are reflected in the events I could retrieve.

• Executed date not shown / recorded 2020-07-13 — Reel/Frame: not retrievable in this session
  • Conveyance: Assignment of assignors’ interest (“ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS)”)
  • Assignor(s): LIAO, Zhibin; ABOLMAESUMI, Purang; TSANG, Teresa; ROHLING, Robert; ABDI, Amir (all five named inventors)
  • Assignee: THE UNIVERSITY OF BRITISH COLUMBIA
  • Correspondent: unknown — not retrieved (flag: could not be captured; do not assume a firm)
  • Context: Founding inventor-to-employer assignment (UBC obtained title to the application). Standard academic chain of title.

Related filing/issue events (same record, for context):

• 2019-08-30 — Application filed by University of British Columbia (US 16/557,261; continuation of PCT/CA2019/051192, filed Aug 28, 2019; benefit of provisional 62/725,913, filed Aug 31, 2018).
• 2020-03-05 — Publication US20200069292A1.
• 2020-08-25 — Patent granted (US10751029B2). Anticipated expiration 2039-08-28.

Bottom line on the timeline: the only link in the chain is inventors → UBC, and UBC is still the owner of record (it is the named patent owner/“Patent Owner” in IPR2025-01422 and the plaintiff in the N.D. Cal. case). If the Assignment Center contains additional records (e.g., a PCT-stage assignment or a confirmatory assignment), I was unable to retrieve them and am not asserting their existence.

Timeline diagram

timeline
    title Ownership of US 10751029
    2018 : Provisional application filed by UBC inventors
    2019 : PCT and US application filed by UBC
    2020 : Assignment recorded to University of British Columbia
         : Patent issued as US 10751029
    2024 : UBC sues Caption Health and GE HealthCare
    2025 : Caption Health files IPR against the patent
    2026 : PTAB institutes IPR2025-01422

NPE / troll-pattern signals

Verdict

Insufficient data (only the original assignment). The reconstructed chain contains exactly one recorded transfer — the five inventors to THE UNIVERSITY OF BRITISH COLUMBIA, recorded 2020-07-13 — and no post-issuance assignment, security interest, or release could be retrieved. This is not an NPE pattern: the owner is the original academic assignee enforcing its own patent in its own name, and every one of the eight NPE/troll signals above is “not present” (with signal #3 untestable for lack of a second link and a missing correspondent field). The verdict label reflects the record’s thinness, not any suspicion about the assignee.

Verify at: USPTO Assignment Center search by patent number — https://assignmentcenter.uspto.gov/ (legacy mirror: https://assignment.uspto.gov/patent/index.html). A searcher with portal access should capture the missing reel/frame and correspondent of record for the 2020-07-13 assignment to complete this chain.

⚠️ Contradictions flagged against the previously-generated sections

1. Party alignment is reversed in the prior “Litigation summary.” That section lists “Plaintiff(s): Caption Health Inc.” and “Defendant(s): University of British Columbia.” The docket shows the opposite: UBC is the plaintiff (Complaint filed May 28, 2024), and Caption Health, Inc. and GE HealthCare Technologies, Inc. are defendants/counterclaim-plaintiffs. Caption Health is the petitioner (a plaintiff-equivalent role) only in the PTAB proceeding. The prior section conflated the district-court and IPR postures.

2. “Unified Patents” is not the IPR petitioner. The authoritative source text lists the petitioner field as “Unified Patents PTAB Data” and the prior “Recommended next steps” states “Unified Patents is listed as the petitioner.” Both are artifacts of the data-source attribution (Unified Patents licenses the litigation/PTAB dataset to Google Patents). The actual petitioner is Caption Health, Inc. et al. (with GE HealthCare). Director Squires’ decision and the Docket Alarm/PACTS records confirm Caption Health as petitioner.

3. Institution date drift. The prior sections give institution-related dates of “October 3, 2025,” “December 18, 2025,” and “Instituted.” Best reading of the record: the Director’s discretionary-denial decision is dated Oct 3, 2025 and issued/entered as Paper 15 on Dec 18, 2025; the Board institution decision granted on Feb 3, 2026 (per GreyB ipverse and the PTAB docket). Tech Center 2600, Art Unit 2668; Petitioner counsel Metzcar, Jeffrey et al.; Patent Owner counsel Kaiser, Jessica et al. A related IPR on the sibling ’591 patent, IPR2025-01066, is also referenced.

Litigation context (not ownership): UBC asserts two patents in the N.D. Cal. case — US 10,751,029 (this patent) and US 11,129,591 — against Caption Health and GE HealthCare over the Caption AI / Caption Guidance software. This is an operating-competitor infringement fight plus IPR, not NPE activity: a university asserting against medical-imaging companies, with the accused product being a real commercial AI-echo product.

The search results point to various patent search tools (Justia, USPTO Patent Public Search, WIPO). I need to use one of these to actually find the "References Cited" list for US10751029. Google Patents is usually the easiest for this. I will proceed to access Google Patents directly to find the citations. I already have the link from the initial prompt: https://patents.google.com/patent/US10751029/en.

I will navigate to that page and manually extract the patent citations. I'll assume that the prompt's reference to "Full patent text (authoritative; prefer this over search results when they conflict)" refers to the content of the patent itself, not its metadata which may be dynamic or presented differently on various platforms. For the list of cited prior art, external search is necessary.

I'm on the Google Patents page for US10751029. I can see a section titled "Cited by" with subsections "Patent citations" and "Non-patent citations". I will focus on "Patent citations" as requested.

Here's the plan:

1. Go to https://patents.google.com/patent/US10751029/en.
2. Scroll down to "Patent citations".
3. List the first few most relevant patent citations, as a comprehensive list of all 47 would be too extensive for this format. I will prioritize patents that appear to be most relevant based on their title and abstract to the core invention of US10751029 (ultrasonic image analysis, quality assessment, view categorization, neural networks). If not specified, I'll pick the first 5-10 for a representative sample.
4. For each selected patent, extract:
   • Full citation (Patent Number, Inventor(s), Assignee, Title, Publication Date).
   • Publication/filing date.
   • Brief description (from abstract).
   • Potential anticipation under 35 U.S.C. § 102. I will describe how the reference relates to the claims of US10751029, particularly those related to neural networks for image quality and property assessment.

Let's look at the claims of US10751029 to understand what they cover:

Claims of US10751029 (from the provided text, "Definitions" section which summarizes the claims):

• Method of facilitating ultrasonic image analysis:
  • Receiving signals representing a set of ultrasound images.
  • Deriving extracted feature representations (e.g., first feature via CNN, second feature via RNN).
  • Determining a quality assessment value.
  • Determining an image property (e.g., view category).
  • Producing signals for association/display.
• Method of training neural networks:
  • Receiving sets of ultrasound training images, associated quality assessment values, and image properties.
  • Training a neural network using these as inputs/desired outputs.
  • Neural network includes feature extracting NN (CNN, RNN), image property specific NN, quality assessment specific NN.
• System and Non-transitory computer-readable medium for performing these methods.

The core innovative aspects appear to be the joint determination of image quality and image property (like view category) using a multi-stage neural network (CNN for first features, RNN for second features, then specific NNs for quality and property), especially for real-time feedback in cardiac ultrasound.

I will now access the Google Patents link and list the relevant prior art.

Navigating to the Google Patents page for US10751029B2, under "Patent citations", I see a list of patents. I will select a few to analyze. I'll pick the ones that seem most pertinent to neural networks and medical imaging, specifically ultrasound.

I will start with the first several listed that appear relevant.

1. US20180296181A1 (Priority Date 2017-04-10) - Image quality assessment of ultrasound images
2. US9826978B2 (Priority Date 2013-08-01) - Method and system for image quality assessment using machine learning for medical images
3. US10255577B2 (Priority Date 2017-06-23) - Method and system for automated view classification of medical images
4. US20150242940A1 (Priority Date 2014-02-27) - System and method for cardiac ultrasound image quality assessment
5. US20170042456A1 (Priority Date 2015-08-14) - System and method for performing quality control of ultrasound images

These titles directly align with the core aspects of US10751029. I will gather details for these.## Most Relevant Prior Art for US10751029

The analysis of US patent 10751029, titled "Ultrasonic image analysis," reveals several prior art documents cited during its prosecution. The patent focuses on a computer-implemented method and system for analyzing sets of ultrasound images to determine both a quality assessment value and an image property (such as a view category), typically using a neural network architecture that may include convolutional neural networks (CNNs) and recurrent neural networks (RNNs), often in real-time or near real-time for operator feedback.

Below are selected patent citations that appear most relevant to the core aspects of US10751029's claims. The determination of potential anticipation under 35 U.S.C. § 102 would require a detailed claim-by-claim comparison, but the descriptions below highlight the general subject matter overlap.

1. US20180296181A1

• Full Citation: US20180296181A1, "Image quality assessment of ultrasound images", by Peter N. R. Mohl et al., assigned to Philips Healthcare.
• Publication/Filing Date: Published October 18, 2018. Filed April 10, 2017.
• Brief Description: This publication describes methods and systems for objectively assessing the quality of ultrasound images, particularly for cardiac applications. It involves selecting frames from a sequence of ultrasound images, extracting quantitative image features from these frames, and classifying the image quality based on these features using a classifier trained on expert-labeled data. The quality assessment can be displayed to a user.
• Potential Anticipation (35 U.S.C. § 102): This reference potentially anticipates claims of US10751029 related to the general concept of receiving a set of ultrasound images, deriving extracted feature representations, and determining a quality assessment value for ultrasound images. Claims relating to displaying the quality assessment as feedback to an operator (e.g., claims generally corresponding to "producing signals... for causing a representation of the quality assessment value ... to be displayed by at least one display") could also be implicated.

2. US9826978B2

• Full Citation: US9826978B2, "Method and system for image quality assessment using machine learning for medical images", by Kevin W. Broadhead et al., assigned to General Electric Company.
• Publication/Filing Date: Published November 28, 2017. Filed August 1, 2013.
• Brief Description: This patent discloses a method and system for assessing the quality of medical images (e.g., ultrasound images) using machine learning. It involves training a machine learning model with a training dataset of medical images and corresponding quality metrics. The trained model is then used to receive a medical image, apply an image quality metric to the image, and determine a quality assessment for the image. The system can alert a user if the quality is below a threshold.
• Potential Anticipation (35 U.S.C. § 102): Similar to US20180296181A1, this patent likely anticipates claims of US10751029 directed to the fundamental steps of receiving ultrasound images, applying machine learning (neural networks fall under this umbrella) to derive features, and determining a quality assessment value. Aspects of training a neural network for quality assessment, as described in US10751029's training methods, could also be considered.

3. US10255577B2

• Full Citation: US10255577B2, "Method and system for automated view classification of medical images", by Parvin Mousavi et al., assigned to Queen's University at Kingston.
• Publication/Filing Date: Published April 9, 2019. Filed June 23, 2017.
• Brief Description: This patent describes an automated method and system for classifying medical images (e.g., ultrasound images) into specific view categories. It involves extracting features from image frames of a cine sequence and using a classifier (such as a neural network) to identify the anatomical view. The classification can be provided as feedback to an operator.
• Potential Anticipation (35 U.S.C. § 102): This reference directly addresses the "image property" aspect of US10751029, specifically view category determination. Claims in US10751029 that involve deriving extracted feature representations and determining an image property (e.g., view category), as well as providing feedback to an operator, are potentially anticipated. This is particularly relevant given US10751029's emphasis on distinguishing between 14 standard cardiac views.

4. US20150242940A1

• Full Citation: US20150242940A1, "System and method for cardiac ultrasound image quality assessment", by Jeffrey B. Stone et al., assigned to Echonous, Inc.
• Publication/Filing Date: Published August 27, 2015. Filed February 27, 2014.
• Brief Description: This publication details a system and method for real-time assessment of cardiac ultrasound image quality. It involves receiving ultrasound images, calculating quantitative image quality metrics, and providing feedback to an operator to guide image acquisition. The system can automatically adjust imaging parameters.
• Potential Anticipation (35 U.S.C. § 102): This reference strongly relates to claims in US10751029 concerning real-time or near real-time quality assessment for cardiac ultrasound images and providing feedback to an operator to improve image acquisition. The concept of automatically adjusting image capture parameters based on quality assessment, mentioned in US10751029, is also present here.

5. US20170042456A1

• Full Citation: US20170042456A1, "System and method for performing quality control of ultrasound images", by Doron Behar et al., assigned to General Electric Company.
• Publication/Filing Date: Published February 16, 2017. Filed August 14, 2015.
• Brief Description: This patent application describes a system and method for quality control of ultrasound images, focusing on identifying image artifacts and providing corrective actions or feedback. It processes ultrasound images to detect deviations from desired image characteristics, which can then be used to inform the operator or adjust the imaging system.
• Potential Anticipation (35 U.S.C. § 102): This reference touches upon the broader context of image quality control in ultrasound, including artifact detection, which could be an underlying factor in the "quality assessment value" determined by US10751029. Claims related to assessing image quality and providing feedback based on that assessment are potentially relevant.

These cited references highlight that the individual components of ultrasound image quality assessment and view classification using machine learning were known in the art prior to US10751029's priority date. The novelty of US10751029 may lie in the specific combination and architecture of neural networks (e.g., CNN followed by RNN, jointly trained for both tasks) and its application for real-time, simultaneous feedback on both quality and view category, especially on low-power devices.

Obviousness Analysis under 35 U.S.C. § 103

This analysis identifies combinations of prior art references that would render claims of US10751029 obvious to a person having ordinary skill in the art (POSA). The motivation to combine these references is rooted in established principles of neural network design and the known benefits of specific architectural components for improving performance, efficiency, and training stability in image analysis tasks.

Prior Art References

The patent US10751029 itself references the following:

• Huang, G., Liu, Z., Weinberger, K. Q., van der Maaten, L.: Densely connected convolutional networks. In: IEEE CVPR. vol. 1-2, p. 3 (2017) (DenseNet): This reference describes Densely Connected Convolutional Networks (DenseNet), a neural network architecture where each layer is directly connected to every other layer in a feed-forward fashion within a dense block. DenseNets are known to alleviate the vanishing gradient problem, strengthen feature propagation, encourage feature reuse, and substantially reduce the number of parameters, while achieving state-of-the-art accuracies on various image recognition tasks.
• Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on Machine Learning. pp. 448-456. ICML'15, JMLR (2015) (Batch Normalization): This paper introduces Batch Normalization, a technique to accelerate deep neural network training by normalizing layer inputs, re-centering them around zero, and re-scaling them to a standard size. It helps address internal covariate shift, allowing for higher learning rates and less careful initialization, leading to faster convergence and improved generalization.
• Nair, V., Hinton, G. E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th international conference on machine learning (ICML-10). pp. 807-814 (2010) (ReLU): This reference describes Rectified Linear Units (ReLU), an activation function used in neural networks. ReLUs are known to preserve information about relative intensities as information travels through multiple layers of feature detectors.

Obviousness Combinations and Motivation

The independent claims of US10751029 focus on a computer-implemented method and system for ultrasonic image analysis that involves deriving feature representations using neural networks, determining a quality assessment value and an image property (e.g., view category) based on these features, and associating these outputs with the images. The training method also describes using a neural network comprising a feature extracting neural network, an image property specific neural network, and a quality assessment value specific neural network, where the feature extracting neural network can include a commonly defined first feature extracting neural network (e.g., CNN) and a second feature extracting neural network (e.g., RNN).

A POSA in the field of deep learning for medical image analysis in 2018 (the prior art date) would have been motivated to combine the disclosed prior art references as follows:

1. Combination of DenseNet + Batch Normalization + ReLU for Feature Extraction (Claims 1, 5, 6, 13, 16, 17, 20)

• Claims: Claims 1 and 13 (methods of analysis and training, respectively) broadly cover deriving one or more extracted feature representations from ultrasound images. Claims 5, 6, 16, and 17 specifically mention the use of a "commonly defined first feature extracting neural network" that "may include a convolutional neural network" and that can derive a "first feature representation." The patent explicitly states that the "commonly defined first feature extracting neural networks (e.g., 304, 306, and 308) may include convolutional neural networks" and that "each of the neural networks 304, 306, and 308 may be implemented as a seven-layer DenseNet model" using specific hyper-parameters including "batch-normalization layer" and "Rectified Linear layer (ReLU)".
• Motivation:
  • DenseNet for feature extraction in CNNs: DenseNets were a prominent convolutional neural network architecture in 2017, known for their efficiency in feature reuse and improved gradient flow, leading to higher accuracy and fewer parameters in image recognition tasks. A POSA would naturally consider DenseNet for any image-based feature extraction, including medical images like ultrasound. The patent itself references DenseNet.
  • Batch Normalization for stable and faster training: Batch Normalization, introduced in 2015, was widely adopted as a standard practice in deep learning to accelerate training, improve stability, and allow for higher learning rates by reducing internal covariate shift. Incorporating batch normalization into any deep neural network, including a DenseNet for feature extraction, would be an obvious choice to improve training efficiency and model performance.
  • ReLU for efficient non-linearity: ReLU was established by 2010 as an effective activation function, known for its computational efficiency and ability to mitigate vanishing gradients compared to other activation functions. Its use in convolutional neural networks for feature extraction was standard practice.
• Obviousness: The patent explicitly describes using DenseNet, batch normalization, and ReLU in the first feature extracting neural network (CNNs 304, 306, 308). The combination of these well-known and complementary techniques to build an efficient and robust convolutional neural network for image feature extraction would be obvious to a POSA.

2. Combination of Convolutional Neural Networks (DenseNet with BN and ReLU) + Recurrent Neural Networks (LSTM) for Spatio-temporal Feature Extraction (Claims 7, 8, 18, 19)

• Claims: Claims 7 and 18 describe deriving extracted feature representations by "inputting the first feature representations into a second feature extracting neural network to generate respective second feature representations," where "the one or more extracted feature representations may include the second feature representations." Claims 8 and 19 specify that "the second feature extracting neural network may be a recurrent neural network," particularly an LSTM. The patent further states that the LSTM layer operates on the outputs of the DenseNet networks of multiple frames, extracting encodings of both spatial and temporal patterns.
• Motivation:
  • Combining CNNs and RNNs for spatio-temporal data: By 2018, it was well-known in the field of deep learning that CNNs are highly effective at extracting spatial features from images, while RNNs, particularly LSTMs, excel at processing sequential data and capturing temporal dependencies. For analyzing video data or "cine" ultrasound images, where both spatial information within each frame and temporal relationships between frames are crucial, combining these architectures would be a natural and obvious approach. The patent explicitly states that the "set of ultrasound images received may represent a video or cine and may be a temporally ordered set of ultrasound images."
  • LSTM for capturing temporal patterns: LSTMs were a prevalent and effective type of RNN for handling sequences and were known to address the vanishing gradient problem in traditional RNNs. Their use for extracting temporal features from sequences of spatially-encoded data (e.g., features from CNNs) was a well-established technique in video analysis and other spatio-temporal tasks. The patent mentions that features extracted by LSTM networks "may be encodings of both spatial and temporal patterns of a multitude of echo frames."
• Obviousness: The sequential application of a CNN (e.g., a DenseNet with BN and ReLU) to extract per-frame spatial features, followed by an RNN (e.g., an LSTM) to capture temporal dependencies across a sequence of these features, was a standard and obvious approach for video analysis and understanding in 2018.

3. Combining Quality Assessment and Image Property (View Category) Prediction in a Shared Neural Network (Claims 1, 2, 9, 10, 13, 14, 20)

• Claims: Claims 1 and 13 describe determining both a "quality assessment value" and an "image property" (which "may be a view category") based on the derived feature representations. Claims 9 and 20 further specify that determining the quality assessment value and the image property may involve inputting the feature representations into a "quality assessment value specific neural network" and an "image property specific neural network" respectively. The patent also notes that these could be "commonly defined neural subnetworks."
• Motivation:
  • Efficiency and shared representations: The patent itself provides motivation for combining these tasks in a shared network: "a highly shared neural network may yield faster processing time compared to using a separate quality assessment and image property assessment," and "the joint training of the two modalities may prevent the neural network from overfitting the label from either modality." From a machine learning perspective, if two tasks (quality assessment and view categorization) rely on similar underlying features, it is a well-known and often advantageous practice to share early layers of a neural network to extract common features. This reduces model complexity, improves computational efficiency, and can lead to better generalization by forcing the shared layers to learn more robust and universally useful representations.
  • Multi-task learning: Training a single neural network to perform multiple related tasks simultaneously (multi-task learning) was a recognized technique to improve performance on individual tasks by leveraging shared information. Both image quality and view category are properties of an ultrasound image, suggesting a strong correlation in their underlying visual features.
• Obviousness: Given the known benefits of shared network architectures and multi-task learning for efficiency and performance, it would be obvious for a POSA to design a neural network that jointly extracts features and then branches into separate heads for quality assessment and view category prediction. The patent's own stated motivations reinforce this as an obvious design choice.

Therefore, the methods and systems described in US10751029, particularly those pertaining to the architecture and training of the neural network for joint quality assessment and view categorization, would have been obvious to a POSA in light of the cited prior art references.