Derivative works — US Patent 11689383

Defensive Disclosure: Derivatives of US Patent 11689383

This document provides technical disclosures of derivative variations and combinations of the invention described in US Patent 11689383, "System, method, and computer-readable medium for streaming real-time data from a user device." The aim is to establish prior art that renders obvious or non-novel future incremental improvements by competitors. This analysis focuses on extrapolating from the core concepts of transmitting a URL via a second connection to initiate a browser-based WebRTC video stream from a mobile device to a recipient during an emergency call, while audio continues on a first connection.

Derivative Variations based on Core Claims (referencing Claim 1 implicitly)

The following derivatives expand upon the method, system, and computer-readable medium aspects of US11689383.

1. Material & Component Substitution: Satellite Network for First Connection and Data Link

Enabling Description: The primary communication pathways for both the "first connection" (emergency voice call) and the "second connection" (electronic message carrying the URL link) are implemented over a satellite communication network. This requires the mobile device to possess integrated satellite communication capabilities (e.g., Iridium, Globalstar, or Starlink direct-to-cell compatibility). The emergency call's audio content is transmitted via the satellite voice channel, while the URL link is delivered via a satellite data burst message (e.g., Short Burst Data, SMS over satellite, or a low-bandwidth IP over satellite connection). The subsequent WebRTC session for real-time video streaming from the mobile device is established over the satellite data link, routed through a satellite ground station acting as an intermediary and WebRTC signaling server. The recipient's dispatch unit terminal (DUT) integrates a satellite modem and associated communication gateway for direct receipt and processing of both the satellite-relayed audio and video streams. This configuration specifically addresses emergency scenarios in areas devoid of terrestrial cellular or internet infrastructure, such as remote wilderness, maritime environments, or disaster-stricken regions.

Mermaid Diagram:

sequenceDiagram
    actor UserDevice: Mobile Device (Satellite-enabled)
    participant SatNetwork: Satellite Network
    participant Server: Server (Satellite Gateway)
    participant CallCenter: Call Center / Recipient

    UserDevice->>SatNetwork: Initiate Emergency Call (Audio - First Connection via Satellite Voice)
    SatNetwork->>CallCenter: Route Emergency Call
    activate CallCenter
    activate SatNetwork

    Server->>UserDevice: Transmit URL (Satellite Data Burst - Second Connection)
    activate Server
    UserDevice->>UserDevice: User Clicks URL, Launches Browser
    UserDevice->>SatNetwork: Establish WebRTC Session (Video Stream via Satellite Data)
    SatNetwork->>Server: Route WebRTC Video Stream
    Server->>CallCenter: Send Real-time Video Stream
    deactivate SatNetwork
    deactivate Server
    deactivate CallCenter

2. Material & Component Substitution: Specialized Low-Latency Hardware-Accelerated Video Encoder/Decoder

Enabling Description: To minimize end-to-end latency and improve video quality under varied network conditions, the mobile device is equipped with a dedicated, low-power hardware-accelerated video encoder, such as an ASIC-based H.265 (HEVC) or AV1 encoder. This hardware encoder directly interfaces with the mobile device's camera sensor and is exposed to the WebRTC API via a browser extension, WebAssembly module, or direct operating system integration, allowing the browser to leverage it for real-time video compression. Concurrently, the recipient's dispatch unit terminal (DUT) incorporates a corresponding hardware-accelerated H.265/AV1 video decoder. This dedicated decoder offloads video processing from the general-purpose CPU, enabling rapid decoding and display of the incoming real-time video stream with sub-100ms processing latency. This specialized hardware approach ensures high-fidelity, ultra-low latency video transmission critical for dynamic emergency situations, even on computationally limited mobile devices and high-throughput dispatch systems.

Mermaid Diagram:

flowchart TD
    subgraph Mobile Device
        UD_Cam[Camera Sensor] --Raw Video--> UD_HW_Encoder[Hardware HEVC/AV1 Encoder]
        UD_HW_Encoder --Encoded Video (Low Latency)--> UD_Browser[Web Browser (WebRTC)]
        UD_Browser --WebRTC Stream--> Network
    end

    Network --WebRTC Stream--> Server[Server (Optional Transcoding)]
    Server --WebRTC Stream--> Recipient_DUT[Recipient DUT]

    subgraph Recipient DUT
        Recipient_DUT --WebRTC Stream--> DUT_HW_Decoder[Hardware HEVC/AV1 Decoder]
        DUT_HW_Decoder --Decoded Video (Sub-100ms)--> DUT_Display[Display Screen]
        DUT_Audio[Audio (First Connection)] --Synchronize--> DUT_Display
    end

3. Operational Parameter Expansion: High-Resolution, Multi-Spectral Video for Industrial Incident Response

Enabling Description: The real-time data stream is expanded to include high-resolution (e.g., 4K or 8K UHD) video captured by a specialized mobile device, integrated with multi-spectral imaging capabilities (e.g., near-infrared, thermal, ultraviolet bands). This mobile device could be a ruggedized smartphone with attachable or built-in multi-spectral sensors. The WebRTC session is configured to transmit these multiple video channels (visible light + spectral bands) using advanced, high-efficiency codecs such as AV1 or VVC (Versatile Video Coding), with adaptive bitrate streaming to accommodate varying network capacities. The recipient's dispatch unit terminal in an industrial or HAZMAT response center features a multi-spectral display capable of rendering and false-coloring these additional spectral data streams. This allows operators to detect specific chemical signatures, heat anomalies, or structural damage invisible to the naked eye, providing unprecedented situational awareness for industrial accidents or environmental hazards. The system dynamically adjusts which spectral channels are transmitted based on incident type and available bandwidth.

Mermaid Diagram:

graph TD
    A[Mobile Device (Multi-Spectral Camera)] --Captures--> B{Raw Multi-Spectral Video Data (4K/8K)}
    B --Encoded (AV1/VVC, Multi-Channel)--> C[Web Browser (WebRTC Session)]
    C --Stream (High Bandwidth/Adaptive)--> D[Network]
    D --Stream--> E[Server (Adaptive Streaming / Transcoding)]
    E --Stream--> F[Recipient DUT (Multi-Spectral Display)]
    F --Renders--> G{Detailed Multi-Spectral Incident View}

    subgraph Real-time Data Flow
        B --> C
        C --> D
        D --> E
        E --> F
    end

4. Operational Parameter Expansion: Ultra-Low Bandwidth Stills and Metadata for Remote Disaster Zones

Enabling Description: For environments with severely compromised or extremely limited network connectivity (e.g., remote disaster zones, deep rural areas), the system automatically defaults to an ultra-low bandwidth mode. In this mode, continuous real-time video streaming is replaced by the transmission of highly compressed still images (e.g., JPEG 2000, WebP, or AVIF) captured at a reduced frequency (e.g., one image every 5-30 seconds). Each still image is transmitted via the WebRTC data channel and is meticulously interleaved with essential metadata: high-precision GPS coordinates, barometric altitude, device orientation (compass heading), timestamp, battery status, and ambient environmental sensor readings (temperature, humidity, air pressure, CO2 levels from mobile device sensors). The recipient's dispatch terminal processes these intermittent data packets, reconstructing a visual timeline from the stills and dynamically overlaying the sensor and location data on a geospatial mapping interface. This "telemetry-rich still image stream" provides crucial, bandwidth-efficient situational awareness where continuous video is infeasible.

Mermaid Diagram:

sequenceDiagram
    actor MobileDevice: User Device (Ultra-Low Bandwidth Mode)
    participant Network: Constrained Network (e.g., LEO Satellite, Mesh, HF Radio Link)
    participant Server: Server (Data Aggregation & Reconstruction)
    participant Recipient: Dispatch Unit Terminal

    MobileDevice->>Network: Emergency Call (Audio - First Conn. - degraded)
    Note over MobileDevice,Network: Audio may be severely compressed, intermittent, or text-based
    
    Server->>MobileDevice: Send URL (SMS/Satellite Burst - Second Conn. - delayed)
    MobileDevice->>MobileDevice: User Clicks, Browser/WebRTC Lite
    
    loop Intermittent Still Images & Metadata
        MobileDevice->>Network: Capture Still Image (Highly Compressed) + Full Telemetry Data
        MobileDevice->>Network: Transmit (WebRTC Data Channel, prioritized)
        Network-->>Server: Forward Stills & Metadata (potentially delayed)
        Server-->>Recipient: Reconstruct & Display Image, Overlay Telemetry
    end
    
    Recipient->>Recipient: Comprehensive Situational Awareness from Stills & Data

5. Cross-Domain Application: Remote Agricultural Diagnostics for Livestock

Enabling Description: This invention is adapted for precision agriculture, specifically for remote veterinary diagnostics and livestock management. A farm worker or animal handler (user) initiates a call to an agricultural or veterinary support center (recipient) upon observing signs of distress or injury in livestock. The mobile device (e.g., a ruggedized tablet or smartphone) obtains the unique animal ID (e.g., via NFC tag scan, RFID reader integration, or manual input) and its GPS location. The support center obtains the mobile device's phone number and sends a URL via SMS. Upon activation, the mobile device's browser launches a WebRTC session, streaming real-time video of the animal, often augmented with specialized thermal imaging from the device's camera for fever detection. This video, along with the animal ID, GPS coordinates, and any collected biometric data (e.g., heart rate from a connected collar), is transmitted to a veterinarian at the support center. The veterinarian uses this integrated real-time data to conduct remote diagnostic assessments, monitor herd health, and provide immediate guidance for animal care or determine the necessity of an on-site visit, improving response times and efficiency in large-scale farming operations.

Mermaid Diagram:

sequenceDiagram
    actor Farmer: Farmer (Mobile Device w/ RFID/NFC, Optional Thermal Cam)
    participant Network: Cellular/Internet Network
    participant VetCenter: Agricultural/Veterinary Support Center
    participant Veterinarian: Veterinarian (Recipient DUT)

    Farmer->>Network: Initiate Call (VoIP/Cellular - First Connection)
    Note over Farmer: Identifies distressed animal
    Network->>VetCenter: Route Call
    activate VetCenter

    VetCenter->>Farmer: Send URL (SMS - Second Connection)
    Farmer->>Farmer: Clicks URL, Browser Launches WebRTC
    Farmer->>Network: Stream Real-time Video (Animal) + Animal ID + GPS + (Optional Biometrics/Thermal)
    Network->>VetCenter: Forward Video + Data
    VetCenter->>Veterinarian: Display Video + Data Overlays
    deactivate VetCenter
    Veterinarian->>Farmer: Provide Remote Diagnosis & Guidance (via First Connection Audio)

6. Cross-Domain Application: Construction Site Safety Incident Reporting with Indoor Positioning

Enabling Description: The system is deployed for immediate safety incident reporting within large-scale construction projects or industrial complexes. A construction worker (user) encounters a safety hazard (e.g., equipment malfunction, structural integrity issue, or personnel injury) and initiates an "emergency call" to the site's central safety control center (recipient). The mobile device, which may be a ruggedized tablet or smartphone, leverages advanced indoor positioning systems (IPS) such as Ultra-Wideband (UWB) tags, Bluetooth Low Energy (BLE) beacons, or Wi-Fi triangulation to provide precise sub-meter location data within the complex, complementing GPS outdoors. The safety control center obtains the worker's device identifier and transmits a URL via SMS. Upon clicking, the worker's browser establishes a WebRTC session, streaming real-time video of the incident. This video stream, along with the highly accurate indoor/outdoor location data and potentially sensor data from the mobile device (e.g., accelerometer data indicating impact, gas sensor readings if integrated), is displayed on a dynamic 3D map interface at the safety control center. This enables safety officers to precisely pinpoint the incident location, visualize the scene, and guide the worker on immediate containment or first aid, while dispatching appropriate emergency teams to the exact coordinates.

Mermaid Diagram:

flowchart LR
    A[Construction Worker Mobile Device] --1. Emergency Call (Audio)--> B(Safety Control Center)
    A --Collects--> C(Indoor/Outdoor Location Data)
    B --2. Obtain Device ID--> D{Server}
    D --3. Transmit URL (SMS)--> A
    A --4. Click URL, Launch Browser--> E[WebRTC Session]
    E --5. Stream Real-time Video + Location Data (IPS/GPS)--> F(Safety Control Center Display)
    F --6. Display Video, Location Overlay on 3D Map--> G[Safety Officer]

    subgraph Connections
        A --First Connection (Voice)--> B
        D --Second Connection (SMS)--> A
        A --WebRTC Session--> F
    end

7. Integration with Emerging Tech: AI-Driven Video Content Analysis and Alert Triggering

Enabling Description: The real-time video stream originating from the mobile device via the WebRTC session is not directly sent to the recipient's display. Instead, it is first routed through an intermediary server equipped with an AI-driven video analytics module. This module leverages advanced computer vision algorithms, including object detection (e.g., for firearms, smoke, fire, specific vehicle types), activity recognition (e.g., detecting falls, struggles, large crowds, rapid movement), and environmental state analysis (e.g., water level changes, structural collapse). The AI continuously processes the incoming video frames, performing real-time inference. Upon detecting predefined critical events or objects, the AI system generates immediate, prioritized alerts or visual overlays (e.g., bounding boxes, heat maps, text labels) on the video feed that is subsequently forwarded to the dispatch unit terminal. For example, if the AI identifies an unconscious person or active flames, an audible alert is triggered at the DUT, and the relevant area of the video is highlighted, providing enhanced contextual information and reducing cognitive load on the human operator, potentially enabling faster and more accurate emergency response.

Mermaid Diagram:

graph TD
    A[Mobile Device] --WebRTC Stream (Raw Video)--> B[Server (AI Pre-processing)]
    B --Input Video Feed--> C{AI Video Analytics Module}
    C --Real-time Inference--> C1(Object Detection: e.g., Fire, Weapon, Person)
    C --Real-time Inference--> C2(Activity Recognition: e.g., Fall, Struggle)
    C --Output: Analyzed Video + Alerts/Overlays--> D[Dispatch Unit Terminal (DUT)]
    D --Display Video + AI-generated Visualizations--> E[Dispatch Operator]
    
    style C1 fill:#f9f,stroke:#333,stroke-width:2px
    style C2 fill:#f9f,stroke:#333,stroke-width:2px

8. Integration with Emerging Tech: IoT Sensor Data Fusion and Blockchain for Evidentiary Integrity

Enabling Description: This derivative enhances the system by fusing the real-time video stream with rich, contextual data from proximate Internet of Things (IoT) sensors and ensuring the integrity of this combined dataset using blockchain technology. The mobile device, upon initiating the WebRTC session, also collects data from paired or nearby IoT devices (e.g., smart wearable sensors providing biometric data like heart rate and body temperature, environmental sensors measuring air quality, radiation levels, or chemical presence in hazardous situations, or integrated vehicle telemetry). This heterogeneous IoT data is synchronized with video frames using precise timestamps and embedded within the WebRTC data channel or streamed alongside the video. Before being displayed at the recipient's DUT, critical segments of the fused video and IoT data, along with unique identifiers and location metadata, are securely committed to a permissioned blockchain network. Each data block contains cryptographic hashes of preceding data, establishing an immutable and auditable chain of custody. The dispatch terminal displays the real-time video with synchronized IoT data overlays and a clear indicator of blockchain verification status, providing verifiable evidence for post-incident analysis, forensics, or legal proceedings.

Mermaid Diagram:

flowchart TD
    A[Mobile Device] --WebRTC Stream--> B(WebRTC Server)
    A --Collects--> C[Co-located IoT Sensors (e.g., Biometrics, Environmental)]
    C --IoT Data--> A
    
    B --Video + Synchronized IoT Data--> D{Data Fusion & Blockchain Integration Module}
    D --Commits Hashed Data--> E[Blockchain Network (Immutable Ledger)]
    D --Real-time Fused Feed--> F[Dispatch Unit Terminal (DUT)]
    
    F --Displays--> G{Video + Synchronized IoT Overlays + Blockchain Verification Status}

9. The "Inverse" or Failure Mode: Privacy-Preserving Low-Power Mode for Sensitive Situations

Enabling Description: The system incorporates a "privacy-preserving low-power mode" that activates under specific conditions: critically low mobile device battery (e.g., below 10%), explicit user activation of a privacy setting, or detection of a sensitive environment (e.g., in a hospital, school, or private residence where full video streaming might be inappropriate or restricted). In this mode, the WebRTC session automatically degrades from a full video stream to transmitting only highly anonymized or obfuscated visual data. This could involve motion vectors without explicit pixel data, extreme pixelation/blurring of the video feed, abstract heat map representations of movement, or periodic still images with significant facial/identifiable object blurring. This obfuscated visual data is combined with continuous audio (if privacy concerns allow, otherwise audio may also be modulated or converted to speech-to-text summaries) and precise location data. The primary goal is to conserve battery life and protect privacy while still providing critical contextual information (e.g., "Is the user moving?", "What is the general environment's activity level?", "Is audio present?") to the dispatch unit. The recipient's DUT is specifically designed to interpret and display these privacy-enhanced visual cues and summarized information, ensuring effective communication in sensitive and resource-constrained scenarios.

Mermaid Diagram:

stateDiagram-v2
    [*] --> NormalOperation
    NormalOperation --> LowBattery : Battery < 10%
    NormalOperation --> PrivacyModeTriggered : User Activates Privacy / Environment Detected
    
    LowBattery --> LowPowerPrivacyMode : Automatic Transition
    PrivacyModeTriggered --> LowPowerPrivacyMode : User/System Activated
    
    LowPowerPrivacyMode --> TransmitAudioLocation : Audio + GPS
    LowPowerPrivacyMode --> TransmitObfuscatedVideo : + Pixelated/Blurred Video / Motion Vectors
    LowPowerPrivacyMode --> DisplaySummarizedInfo : Dispatch Displays Essential Data
    
    TransmitAudioLocation --> DispatchDisplay : Contextual Audio & Location
    TransmitObfuscatedVideo --> DispatchDisplay : Abstract Visual Cues
    
    LowPowerPrivacyMode --> NormalOperation : Battery Charged / Privacy Deactivated / All Clear

10. The "Inverse" or Failure Mode: Limited-Functionality "Safe-Mode" for Compromised Devices

Enabling Description: In situations where the mobile device's integrity is compromised (e.g., detection of malware, unauthorized root access, or suspicious OS activity through device attestation) or when a full WebRTC video connection cannot be established due to severe network degradation or explicit user refusal of camera access, the system enters a "limited-functionality safe-mode." In this mode, the WebRTC session attempts to transmit only a minimal, verifiable stream of still images (e.g., one image per minute) from the most accessible camera (front or rear), critical device health metrics (e.g., battery level, signal strength, detected malware signature, available storage), and periodic, coarse-grained location updates (e.g., cell tower IDs, Wi-Fi SSID scans, or last known GPS fixes). All transmitted data, including the still images and metadata, is subjected to enhanced end-to-end encryption using a hardware security module (HSM) if available on the device, and digitally signed with a unique device key. A simplified, tamper-evident manifest accompanies each data packet. This ensures that even in a compromised state, the dispatch receives critical, authenticated, and minimal information, potentially aiding in identifying the user's general whereabouts and device status, without risking further exploitation or misrepresentation.

Mermaid Diagram:

flowchart TD
    A[Mobile Device] --Detects Compromise/Network Failure--> B{Safe-Mode Activation Logic}
    B --Activates--> C[Limited-Functionality WebRTC Session]
    
    C --Transmit Encrypted & Signed Still Images (Periodic)--> D(Network)
    C --Transmit Encrypted & Signed Device Health Metrics--> D
    C --Transmit Encrypted & Signed Coarse Location Updates--> D
    
    D --Encrypted & Authenticated Stream--> E[Server (Decryption/Verification/Attestation)]
    E --Verified & Minimal Data--> F[Dispatch Unit Terminal]
    F --Displays--> G{Still Images + Health/Location Overlay + Integrity Status}

    style B fill:#faa,stroke:#333,stroke-width:2px
    style E fill:#aaf,stroke:#333,stroke-width:2px

Combination Prior Art Scenarios

These scenarios describe combinations of the core invention (US11689383) with existing open-source standards, demonstrating obviousness for certain enhancements.

1. Combination with Session Initiation Protocol (SIP) and SIPREC for Unified Communication Recording

Description: The emergency call (the "first connection") is initiated and managed using the open-source Session Initiation Protocol (SIP), a widely adopted standard for multimedia communication over IP networks. This means the call setup, tear-down, and mid-call changes are controlled by a SIP server at the emergency call center (PSAP). When the URL is transmitted via the "second connection" (e.g., SMS) to trigger the WebRTC video stream, the WebRTC signaling itself is also brokered or coordinated by the existing SIP infrastructure, treating the WebRTC stream as an additional media channel within the established SIP session. Crucially, the entire communication, encompassing both the audio content from the SIP voice call and the real-time video stream from the WebRTC session, is unified and recorded using the SIP Recording (SIPREC) standard. The unique identifier for the mobile device, as described in US11689383, is explicitly integrated into the SIP call metadata and SIPREC session data, ensuring a comprehensive, synchronized, and legally compliant record of the multi-modal emergency interaction.
Prior Art Value: The integration of WebRTC streams into an existing SIP-based communication environment for emergency services is a natural and obvious extension for professionals skilled in telecommunications and VoIP. Leveraging SIPREC for unified recording of both voice and supplemental media (like WebRTC video) is a standard practice for compliance and archival in call centers, particularly those handling critical communications. The use of a unique identifier to correlate these streams would be a logical necessity.

2. Combination with MPEG-DASH (Dynamic Adaptive Streaming over HTTP) for Server-Side Adaptive Delivery

Description: The real-time video stream received from the mobile device via the WebRTC session at the server (Server 120 in FIG. 1) is immediately processed and transcoded into an adaptive bitrate (ABR) format using the open-source MPEG-DASH (Dynamic Adaptive Streaming over HTTP) standard. While WebRTC provides the initial low-latency, peer-to-peer (or relayed) connection, the server simultaneously segments the incoming stream into multiple quality representations (e.g., 360p, 720p, 1080p, 4K) and stores them for adaptive delivery. This allows dispatch unit terminals or other authorized viewers within the PSAP network to access the same incident video feed via standard HTTP, capable of dynamically adjusting the video quality based on their local network conditions, device capabilities, or explicit user preferences. This also facilitates efficient archival and VOD (Video on Demand) playback within the emergency response system, allowing operators to rewind, fast-forward, or review specific moments of the incident stream with optimal performance on diverse viewing platforms (e.g., desktop displays, tablets, or even large situation room monitors).
Prior Art Value: Integrating real-time WebRTC input with an adaptive streaming standard like MPEG-DASH for broader consumption, archiving, and flexible playback within an enterprise or public safety context is a straightforward engineering task. The benefits of adaptive streaming (network resilience, device compatibility, VOD features) are well-known, and applying them to a WebRTC ingest stream would be an obvious optimization.

3. Combination with OpenStreetMap (OSM) for Enhanced Geospatial Context and Visualization

Description: The GPS coordinates and/or indoor location data transmitted by the mobile device through the WebRTC session (as optionally described in US11689383 for real-time data or claim 7) are explicitly integrated and rendered onto an interactive map interface powered by OpenStreetMap (OSM) data at the recipient's dispatch unit terminal. The unique identifier associated with the mobile device is used to track the real-time position and movement of the user on a highly detailed, open-source basemap. This integration extends to leveraging other publicly available and crowd-sourced spatial information from OSM, such as building footprints, points of interest (e.g., hospitals, schools), public transportation routes, and even fine-grained details like fire hydrants or accessibility ramps, which are dynamically overlaid onto the map. This provides dispatch operators with significantly enhanced geospatial context, allowing them to visualize the incident location within a rich, real-world framework and communicate precise, landmark-based instructions to emergency responders. The map interface also displays the mobile device's compass heading and estimated speed to provide dynamic situational awareness.
Prior Art Value: The use of open-source mapping platforms like OpenStreetMap for visualizing real-time location data is a common and obvious application in fields requiring geospatial awareness. Integrating the location data from a real-time streaming event (like a WebRTC video stream during an emergency) onto such a map, and overlaying additional contextual data from the map itself, is a direct and uninventive enhancement.