Obviousness

The obviousness of US patent 10991097 under 35 U.S.C. § 103 can be assessed by considering combinations of the prior art explicitly mentioned or acknowledged within the patent document itself. A person having ordinary skill in the art (POSA) in the field of AI-driven medical image analysis would be motivated to combine these known techniques to address existing challenges in histological image segmentation.

Common General Knowledge and Explicit Prior Art:

The patent US10991097 acknowledges several existing technologies and challenges in the background and detailed description:

1. Convolutional Neural Networks (CNNs): Acknowledged as deep learning algorithms that analyze digital images by assigning one class label to each input image. The patent notes their limitation for slides containing multiple tissue types [cite: "A Convolutional Neural Network (“CNN”) is a deep learning algorithm that analyzes digital images by assigning one class label to each input image. Slides, however, include more than one type of tissue, including the borders between neighboring tissue classes."].
2. Fully Convolutional Networks (FCNs): Acknowledged as capable of assigning classification labels to each pixel within an image, thus more useful for images with multiple classifications and generating overlay maps. However, the patent states that traditional FCNs for digital slides are impractical due to the extensive annotation time and computational requirements for pixel-level labeling of high-resolution images [cite: "A Fully Convolutional Network (FCN) can analyze an image and assign classification labels to each pixel within the image, so a FCN is more useful for analyzing images that depict objects with more than one classification.", "However, FCN deep learning algorithms that analyze digital slides would require training data sets of images with each pixel labeled as a tissue class, which requires too much annotation time and processing time to be practical."].
3. ResNet-18 Image Recognition Model: Explicitly mentioned as a "known CNN" that forms the basis for the tile-resolution FCN PhiNet architecture described in the patent [cite: "The tissue class locator 216 includes a tile-resolution fully convolutional network (FCN) black box deep learning model based on a known CNN ResNet-18 image recognition model.", "FIG. 6B illustrates the differences between the ResNet-18 algorithm on the left, and the tile-resolution FCN PhiNet shown in FIG. 6A and on the right half of FIG. 6B."].
4. Two-class UNet Models with Binary Classification: Acknowledged as "known in the art" for cell segmentation (e.g., cell vs. background). The patent highlights their limitation in accurately counting overlapping cells [cite: "Two-class UNet models with binary classification (cell vs background) are known in the art, but the three-class UNet model allows a type of classification that is not binary, which requires adaptation with the use of a different loss function.", "In traditional two-class cell outlining models that only label whether a pixel contains a cell outer edge or not, each clump of two or more overlapping cells would be counted as one cell."].
5. U.S. Provisional Patent Application No. 62/889,521: This provisional application, titled "Determining Therapeutic Tumor-Infiltrating Lymphocytes (TILS) from Histopathology Slide Images," filed on August 20, 2019, is incorporated by reference, indicating its status as prior art for the present patent (filed December 31, 2019) [cite: "An example of a TILS process and engine is disclosed, for example, in U.S. Provisional Patent Application No. 62/889,521, titled “Determining Therapeutic Tumor-Infiltrating Lymphocytes (TILS) from Histopathology Slide Images,” filed on Aug. 20, 2019, which is incorporated herein by reference"].

Obviousness Combinations and Motivation:

The independent claims of US10991097 generally cover methods and systems for:

• Tile-based tissue classification using multi-tile analysis (Claims 1, 11, 27, 30).
• Cell detection and outlining using polygons (Claims 14, 31).
• Combining tile and cell classifications, with cell classification overriding tile classification (Claims 17, 32).

A POSA would have been motivated to combine the known prior art to achieve the claimed inventions for the following reasons:

Combination 1: ResNet-18 + Traditional FCNs → Tile-resolution FCN (PhiNet) for Tissue Classification (Claims 1, 11, 27, 30)

• Rationale: The patent explicitly states the limitations of traditional CNNs (one label per image) and traditional FCNs (computational intensity for pixel-level segmentation of large images) for analyzing complex histological slides with diverse tissue types [cite: "There is a need to classify different regions as different tissue classes, in part to study the borders between neighboring tissue classes and the presence of immune cells among tumor cells.", "The high number of pixels makes it infeasible to use traditional FCNs to segment digital images of slides."].
• Motivation: A POSA, faced with these known problems, would be motivated to adapt existing, proven deep learning architectures like ResNet-18 to enable efficient, multi-class segmentation of large digital histology images. The adaptation to a "tile-resolution" FCN (PhiNet) with "additional layers" for a "classification-segmentation task" on tiles, as described in the patent, directly addresses the computational and annotation challenges of pixel-level FCNs while providing more granular classification than a whole-image CNN [cite: "the added layers convert a classification task into a classification-segmentation task. This means that instead of receiving and classifying a whole image as one tissue class label, the added layers allow the tile-resolution FCN to classify each small tile in the user-defined grid as a tissue class."]. The concept of dividing an image into tiles for processing is a standard image processing technique, and applying an adapted convolutional network for tile-level classification would be an obvious step for a POSA seeking to balance computational efficiency with localized analysis. The "multi-tile analysis" and context-awareness (medium tiles providing context for small central tiles) described in the patent would be an expected refinement to improve accuracy by leveraging local neighborhood information, a common approach in image recognition.

Combination 2: Two-class UNet Models + General AI Image Segmentation Principles → Three-class UNet Model for Cell Detection (Claims 14, 31)

• Rationale: The patent acknowledges that "Two-class UNet models with binary classification (cell vs background) are known in the art" [cite: "Two-class UNet models with binary classification (cell vs background) are known in the art, but the three-class UNet model allows a type of classification that is not binary, which requires adaptation with the use of a different loss function."]. The core problem identified is the inaccuracy in counting individual cells when they overlap, a critical issue for quantitative analysis like Tumor-Infiltrating Lymphocyte (TIL) assessment [cite: "This facilitates the counting of each individual cell, especially when two or more cells overlap each other. In one example, tumor infiltrating lymphocytes will overlap tumor cells. In traditional two-class cell outlining models that only label whether a pixel contains a cell outer edge or not, each clump of two or more overlapping cells would be counted as one cell."].
• Motivation: A POSA specializing in biomedical image analysis would be motivated to improve the precision of cell detection and counting, especially for overlapping cells. Modifying a known two-class UNet to a three-class model (background, cell outer edge, cell interior) to explicitly distinguish cell boundaries from interiors would be an obvious solution to this known problem of overlapping cells. This adaptation would be driven by the clear need for more accurate individual cell segmentation, and the patent itself details this adaptation, including the need for a "different loss function" [cite: "Two-class UNet models with binary classification (cell vs background) are known in the art, but the three-class UNet model allows a type of classification that is not binary, which requires adaptation with the use of a different loss function."].

Combination 3: Tile-resolution FCN (PhiNet) for Tissue Classification + Three-class UNet for Cell Detection + U.S. Provisional Patent Application No. 62/889,521 → Integrated System with Cell-overriding-Tile Classification (Claims 17, 32)

• Rationale: Independent claims 17 and 32 describe an integrated system where both tile-level tissue classes and cell objects are identified, and the cell object's predicted class overrides the tile's predicted class if they correspond. This represents a hierarchical approach to segmentation.
• Motivation: A POSA aiming to provide a comprehensive and highly accurate diagnostic tool would be strongly motivated to integrate different levels of AI analysis. The limitations of tile-level classification (a tile might contain multiple cell types, making a single majority class assignation potentially misleading) would lead a POSA to seek refinement. The detailed, pixel-level information from a cell detection model (like the three-class UNet) offers this refinement. Combining the tile-based tissue classification (from PhiNet, as in Combination 1) with the individual cell detection and classification (from the UNet, as in Combination 2) and allowing the more precise cell-level data to override broader tile-level classifications is a logical and obvious step to enhance diagnostic accuracy. This is especially true given the clinical importance of specific cell types, such as Tumor-Infiltrating Lymphocytes (TILs), which are explicitly mentioned in the patent as being detectable by the system and taught in the incorporated provisional application [cite: "the digital tissue segmenter 201 is configured to determine tumor-infiltrating lymphocytes (TILS). An example of a TILS process and engine is disclosed, for example, in U.S. Provisional Patent Application No. 62/889,521, titled “Determining Therapeutic Tumor-Infiltrating Lymphocytes (TILS) from Histopathology Slide Images,” filed on Aug. 20, 2019, which is incorporated herein by reference"]. A POSA would understand that a specific cell type identified by a highly accurate cell model should take precedence over a coarser tissue classification for a region, particularly in contexts like predicting immunotherapy response. The integration would create a more robust and clinically relevant system.

In summary, the advancements described in US10991097, while useful, appear to be a logical progression for a person having ordinary skill in the art, driven by well-understood problems in medical image analysis and building upon existing, known AI architectures and techniques. The motivations for combining these elements—namely, to improve efficiency, accuracy, and clinical relevance in the analysis of complex histological images—would have been readily apparent.