Derivative works

Defensive Disclosure for U.S. Patent 11,275,900

Publication Date: May 14, 2026
Reference ID: DPD-2026-0514-US11275900
Title: Derivative Implementations and Obvious Variations of Hierarchical, Multi-Label Text Classification Systems in High-Noise Environments

This document discloses a series of derivative works, extensions, and alternative implementations related to the core teachings of U.S. Patent 11,275,900. The purpose of this disclosure is to place these variations into the public domain, thereby establishing them as prior art for any future patent applications.

---

Axis 1: Material & Component Substitution

Derivative 1.1: Transformer-Based Feature Extraction

• Description: This variation replaces the Doc2vec or Word2vec feature extraction component (recited in the patent as "assigning word vectors or paragraph vectors") with a transformer-based deep learning model, such as BERT (Bidirectional Encoder Representations from Transformers), RoBERTa, or a domain-specific variant pre-trained on cybersecurity corpora. Instead of static word embeddings, this system generates context-aware embeddings for each token in the input text (e.g., a forum post title). The final feature vector for the machine classifier is derived from the pooled output of the transformer's last hidden state (e.g., the [CLS] token's embedding). This method provides a more nuanced semantic understanding of the text, particularly for handling slang, misspellings, and polysemy common in dark web forums.

• Enabling Description: A system is configured with a pre-trained bert-base-uncased model. A new discussion topic string is tokenized using the WordPiece tokenizer, adding special [CLS] and [SEP] tokens. The tokenized input is passed through the BERT model. The resulting 768-dimension vector corresponding to the [CLS] token is extracted and used as the input feature set for a downstream classifier, such as a multi-layer perceptron with a sigmoid activation function for multi-label classification. The core logic of using a probability threshold to add parent tags from a pre-defined hierarchy remains, but it is applied to the output of this new classifier.

• Mermaid Diagram:

  graph TD
      A[Input: Dark Web Topic Text] --> B{BERT Tokenizer};
      B --> C[BERT Model];
      C --> D[Extract [CLS] Token Embedding];
      D --> E{Multi-Layer Perceptron Classifier};
      E --> F[Output: Tag Probabilities];
      F --> G{Thresholding Logic};
      G -- Prediction Probability > α --> H[Add Parent Tags];
      G -- Prediction Probability <= α --> I[Final Prediction List];
      H --> I;
  

Derivative 1.2: Graph-Based Feature Extraction

• Description: This derivative treats the entire corpus of dark web forums as a heterogeneous graph, where nodes represent users, posts, and named entities (e.g., malware names, CVE numbers), and edges represent relationships (e.g., "author of," "mentions"). A Graph Neural Network (GNN), such as GraphSAGE or a Graph Attention Network (GAT), is used to learn embeddings for each post node. These embeddings capture not only the text of the post but also its relational context within the forum ecosystem. This feature vector is then used for classification.

• Enabling Description: The system first parses a dataset of 200,000 forum posts to build a graph. Nodes are created for each post, user, and unique term. Edges link users to their posts and posts to the terms they contain. A GraphSAGE model is trained on this graph to generate 256-dimension embeddings for each post node. When a new topic requires classification, its corresponding node embedding is retrieved and fed into a Random Forest classifier. The subsequent parent-tag addition logic operates on the classifier's output probabilities.

• Mermaid Diagram:

  graph TD
      subgraph Pre-processing
          A[Corpus of Forum Posts] --> B(Graph Construction);
          B --> C{Nodes: Posts, Users, Terms};
          B --> D{Edges: Author, Mentions};
      end
      subgraph Classification
          E[New Topic Post] --> F(Find Node in Graph);
          F --> G[GraphSAGE Model];
          G --> H[Generate Node Embedding];
          H --> I{Classifier};
          I --> J[Prediction List w/ Probabilities];
          J --> K(Enforce Tag Hierarchy);
      end
      C --> G;
      D --> G;
  

---

Axis 2: Operational Parameter Expansion

Derivative 2.1: Real-Time Industrial IoT Log Classification

• Description: This variation applies the core method to classify high-velocity, high-volume log data streams from industrial machinery. The "discussion topic" is a log entry or a window of log entries, and the "tags" represent machine states (e.g., normal, overheating, vibration_anomaly). The tag hierarchy represents subsystem dependencies (e.g., a bearing_fail tag is a child of motor_assembly_fault). The system must operate with sub-second latency to enable real-time alerts.

• Enabling Description: A Kafka stream ingests log data at a rate of 10,000 messages per second from a factory floor. A Flink processing job consumes these messages. For each message, a feature vector is generated using a fast text embedding model like fastText trained on engineering logs. A pre-trained LightGBM model classifies the vector. The output probabilities are compared against thresholds (α=0.95 for adding a "system_critical" parent tag) to generate a final classification, which is then pushed to a monitoring dashboard. The model is retrained nightly on the previous day's labeled data.

• Mermaid Diagram:

  sequenceDiagram
      participant IoT_Device
      participant Kafka_Broker
      participant Flink_Processor
      participant Classifier_Service
      participant Monitoring_Dashboard
  
      IoT_Device->>Kafka_Broker: Stream Log Message
      loop Real-time Processing
          Flink_Processor->>Kafka_Broker: Consume Message
          Flink_Processor->>Classifier_Service: Request Classification(Log Text)
          Classifier_Service-->>Flink_Processor: Return Tag Probabilities
          Flink_Processor->>Flink_Processor: Apply Hierarchy Logic (α, β)
          Flink_Processor->>Monitoring_Dashboard: Push Alert (Final Tags)
      end
  

Derivative 2.2: Nanoscale Genomic Sequence Functional Tagging

• Description: The invention is adapted to automatically assign functional labels to newly discovered gene sequences. The "text" is a DNA or protein sequence. The "tags" are functional annotations from a Gene Ontology (GO) hierarchy (e.g., 'molecular function', 'cellular component', 'biological process'). The parent/child relationships are explicitly defined by the GO tree. This allows for high-throughput functional prediction in bioinformatics.

• Enabling Description: A DNA sequence is converted into a sequence of k-mers (e.g., 3-mers like 'ACG', 'CGT'). These k-mers are treated as "words." A Doc2vec model is trained on the entire RefSeq database to learn embeddings for complete gene sequences. The resulting vector for a new sequence is fed into a deep neural network classifier. The output layer has neurons corresponding to thousands of GO terms. The hierarchical consistency logic is applied to the output softmax probabilities, ensuring that if a specific function like GO:0003723 (RNA binding) is predicted, its parent GO:0003674 (molecular_function) is also added to the final annotation list.

• Mermaid Diagram:

  graph TD
      A[Input DNA Sequence] --> B(Generate k-mers);
      B --> C[Pre-trained Gene2Vec Model];
      C --> D[Sequence Feature Vector];
      D --> E{Multi-label DNN Classifier};
      E --> F[Probabilities for GO Terms];
      F --> G{Apply GO Hierarchy Rules};
      G --> H[Final Functional Annotation List];
  

---

Axis 3: Cross-Domain Application

Derivative 3.1: Aerospace - Avionics Fault Log Analysis

• Description: This system is applied to classify maintenance and fault logs from aircraft avionics systems. The unstructured text written by pilots and technicians is automatically tagged with a predefined fault hierarchy (e.g., System -> Navigation -> GPS -> Signal_Loss). This standardizes reporting and allows for predictive maintenance by identifying recurring, low-level issues that may precede a major failure.

• Enabling Description: A dataset of 500,000 historical maintenance logs is used to create the ground truth and tag hierarchy. Text is pre-processed to handle aviation-specific acronyms. A SciBERT model, pre-trained on scientific text, is fine-tuned on this data to extract features from new log entries. The classifier is an SVM. When a technician enters "GPS signal dropping intermittently on approach," the system predicts Signal_Loss. Based on a probability > 0.8, it adds the parent tags GPS and Navigation automatically.

• Mermaid Diagram:

  erDiagram
      MAINTENANCE_LOG {
          int LogID PK
          string RawText
          datetime Timestamp
      }
      CLASSIFICATION {
          int LogID PK, FK
          string TagID PK, FK
          float Probability
      }
      TAG_HIERARCHY {
          string TagID PK
          string TagName
          string ParentTagID FK
      }
      MAINTENANCE_LOG ||--|{ CLASSIFICATION : "is classified by"
      TAG_HIERARCHY ||--o{ TAG_HIERARCHY : "has parent"
      CLASSIFICATION }|--|| TAG_HIERARCHY : "uses tag"
  

Derivative 3.2: AgTech - Automated Crop Disease Identification

• Description: The system analyzes reports from farmers submitted via a mobile app, which may include text descriptions and images. The goal is to provide a preliminary diagnosis of crop diseases. The text classification component analyzes the farmer's description (e.g., "yellow spots on lower leaves of my tomato plants"). The tag hierarchy is a taxonomy of plant diseases.

• Enabling Description: The system uses a multi-modal architecture. A CNN (e.g., ResNet) processes the image, while a text model (as described in the patent) processes the description. The feature vectors from both models are concatenated and fed into a final classifier. The hierarchy (e.g., Fungal -> Blight -> Early_Blight) is enforced on the text classification output before being combined with the image analysis for a final recommendation.

• Mermaid Diagram:

  graph TD
      subgraph Inputs
          A[Farmer's Text Description]
          B[Image of Crop]
      end
      subgraph Processing
          A --> C[Text Feature Extractor];
          B --> D[Image Feature Extractor (CNN)];
          C & D --> E(Concatenate Features);
          E --> F{Final Classifier};
      end
      subgraph Output
          F --> G[Disease Tag Probabilities];
          G --> H(Apply Disease Taxonomy Hierarchy);
          H --> I[Diagnostic Report];
      end
  

---

Axis 4: Integration with Emerging Tech

Derivative 4.1: AI-Driven Threshold Optimization

• Description: This derivative integrates a Reinforcement Learning (RL) agent to dynamically tune the add parent threshold (α) and remove child threshold (β). The agent's goal is to maximize the F1 score over time as new data arrives and concept drift occurs. This automates the difficult process of selecting optimal thresholds.

• Enabling Description: A PPO (Proximal Policy Optimization) agent is used. The state is a vector representing the classifier's performance metrics (precision, recall, F1 score) over the last 1000 classifications. The action space is discrete: increase/decrease α/β by 0.05. The reward is the change in the F1 score. The agent periodically takes an action, the system runs with the new thresholds for a period, and the resulting change in performance provides the reward signal to train the agent's policy.

• Mermaid Diagram:

  graph TD
      A(Start) --> B{Observe Current F1 Score};
      B --> C[RL Agent Selects Action (adjust α, β)];
      C --> D{System Classifies New Data w/ New Thresholds};
      D --> E{Calculate New F1 Score};
      E --> F[Calculate Reward (ΔF1)];
      F --> G{Update RL Agent's Policy};
      G --> B;
  

Derivative 4.2: Blockchain for Auditable Intelligence

• Description: The entire classification process is logged on a private blockchain to provide an immutable and auditable trail for high-stakes applications like law enforcement or national security intelligence. Every classification is a transaction containing the source data hash, the feature vector, the predicted tags with probabilities, the thresholds used, and the final output.

• Enabling Description: A Hyperledger Fabric network is established. When the system classifies a dark web topic, a chaincode (smart contract) function is invoked. This function takes the classification details as arguments. It validates the inputs and commits a new transaction to the ledger. An access control layer ensures that only authorized analysts can query the blockchain to trace how and why a particular piece of intelligence was categorized.

• Mermaid Diagram:

  sequenceDiagram
      participant Analyst_UI
      participant Classification_Engine
      participant Blockchain_Network
  
      Analyst_UI->>Classification_Engine: Submit Topic for Classification
      Classification_Engine->>Classification_Engine: Process & Generate Tags/Probabilities
      Classification_Engine->>Blockchain_Network: Invoke Chaincode('recordClassification', data)
      Blockchain_Network->>Blockchain_Network: Validate & Commit Transaction
      Blockchain_Network-->>Classification_Engine: Transaction Success
      Classification_Engine-->>Analyst_UI: Display Final Tags
  

---

Axis 5: The "Inverse" or Failure Mode

Derivative 5.1: Conservative Classification with "Human Review" Default State

• Description: This variation is designed for sensitive content moderation (e.g., hate speech, misinformation). To minimize the risk of incorrect automated action, the system defaults to a "Requires Human Review" state if classification confidence is low. Parent tags are only added if the child tag's probability is exceptionally high (e.g., > 0.98), ensuring that any automated hierarchical tagging is done with maximum certainty.

• Enabling Description: A global confidence threshold (T_global = 0.70) is set. After the classifier generates probabilities for all possible tags for a given text, the maximum probability (P_max) is identified. If P_max < T_global, the system immediately outputs a single "Human Review" tag and halts. If P_max >= T_global, the standard hierarchical logic proceeds, but using a very high add parent threshold (α = 0.98).

• Mermaid Diagram:

  stateDiagram-v2
      [*] --> Processing
      Processing --> Human_Review: if P_max < 0.70
      Processing --> Hierarchical_Tagging: if P_max >= 0.70
      Hierarchical_Tagging --> Tagged_Safe: if child_P > 0.98, add parents
      Hierarchical_Tagging --> Human_Review: if child_P <= 0.98
      Tagged_Safe --> [*]
      Human_Review --> [*]
  

---

Combination Prior Art Scenarios

1. Integration with STIX/TAXII for Threat Intelligence Sharing: The hierarchical tags produced by the system (e.g., malware -> ransomware -> Conti) are programmatically mapped to STIX 2.1 Cyber-observable Objects. A Malware object is created with the name "Conti" and the type "ransomware." This structured object is then published to a TAXII 2.1 server, making the intelligence from the dark web forum immediately available to any connected security tool (SIEM, SOAR) that subscribes to the TAXII feed.

2. Mapping to MITRE ATT&CK Framework: The classification system is enhanced with a post-processing module that maps the output tags to the MITRE ATT&CK framework. For a post discussing a new phishing technique, the system might output tags like phishing, credential-access, and spearphishing-attachment. The post-processing module uses a dictionary to map these tags to ATT&CK Technique IDs, such as T1566.001 (Phishing: Spearphishing Attachment). This contextualizes the raw intelligence within a standardized model of adversary behavior, allowing security teams to assess their defensive posture against the discussed threat.

3. Integration with SPDX for SBOM Vulnerability Management: The system is deployed to monitor open-source developer forums and code repositories. When discussions about vulnerabilities in a specific software package arise, it classifies the text with tags like buffer-overflow, log4j, v2.17.1. This output is then used to automatically generate a vulnerability disclosure in the SPDX (Software Package Data Exchange) format. The SPDX document explicitly links the CVE or vulnerability description to the specific software component and version discussed, creating a machine-readable artifact that can be used to enrich a Software Bill of Materials (SBOM).