Derivative works — US Patent 8941708

Defensive Disclosure for U.S. Patent 8,941,708

Publication Date: May 13, 2026
Subject: Derivative Works and Obvious Variations of U.S. Patent 8,941,708 for Video Conferencing Layout Modification
Technology Area: Video Composition, User Interfaces, Real-Time Communication

This document discloses a series of methods, systems, and applications that build upon, vary, and expand the core concepts described in U.S. Patent 8,941,708. The purpose of this disclosure is to place these derivative concepts into the public domain, thereby establishing them as prior art for any future patent applications.

Axis 1: Material & Component Substitution

Derivative 1.1: Radial and Multi-Axis Layout Control

Enabling Description: The linear slider (axis) is replaced with a radial dial or a two-dimensional control plane (e.g., a virtual joystick or trackpad). In a radial implementation, the user rotates a dial. The angle of rotation (0-360 degrees) is divided into angular intervals, each mapped to a predefined layout. This allows for a circular, continuous loop of layouts. In a 2D plane implementation, the X-axis controls the balance between a primary speaker and other participants (as in the original patent), while the Y-axis controls a different parameter, such as the number of participant windows displayed or the aspect ratio of the main window. The video composing unit receives a coordinate pair (x, y) and selects a layout from a 2D matrix of predefined layouts corresponding to that position.

Mermaid Diagram:

graph TD
    A[User interacts with 2D Pad] --> B{Input (X, Y) Coordinates};
    B --> C{Layout Selection Logic};
    C -->|X-axis: Focus| D[Layout A: Speaker Dominant];
    C -->|Y-axis: Presence| E[Layout B: Grid View];
    C -->|X,Y Combination| F[Layout C: Hybrid PIP/POP];
    D --> G[Video Composing Unit];
    E --> G;
    F --> G;
    G --> H[Render Composite Video];

Derivative 1.2: Haptic Feedback Physical Controller

Enabling Description: The user interface object is a physical rotary encoder or linear slider on a dedicated control surface (e.g., a mixing console or custom remote). The intervals along the axis are not just visual but are implemented with haptic detents. As the user moves the physical control, a haptic engine provides tactile feedback (a "click" or "bump") when crossing the threshold from one layout interval to another. The controller sends a simple integer or float value corresponding to its position to the video conference endpoint via USB HID or Bluetooth, which then directs the video composing unit.

Mermaid Diagram:

sequenceDiagram
    participant User;
    participant PhysicalDial;
    participant HapticEngine;
    participant VideoEndpoint;
    User->>PhysicalDial: Rotates dial;
    PhysicalDial->>HapticEngine: Crosses interval threshold;
    HapticEngine-->>User: Generates haptic detent (click);
    PhysicalDial->>VideoEndpoint: Sends new position value (e.g., 0.75);
    VideoEndpoint->>VideoEndpoint: Selects layout for interval;
    VideoEndpoint->>VideoEndpoint: Compose new video signal;

Derivative 1.3: Gaze-Based Axis Control

Enabling Description: For hands-free environments, an eye-tracking sensor is used as the input device. A static, on-screen visual element represents the axis. The user's gaze position along this visual axis is continuously monitored. If the user's gaze dwells on a specific interval of the axis for a predetermined duration (e.g., >500ms), that action is detected as a user selection. The system then triggers the video composing unit to switch to the layout associated with the selected interval. This is applicable in accessibility, medical, or industrial settings.

Mermaid Diagram:

stateDiagram-v2
    [*] --> Idle
    Idle --> Tracking: User looks at screen
    Tracking --> Idle: User looks away
    Tracking --> Dwelling: Gaze enters layout interval
    Dwelling --> Tracking: Gaze leaves interval
    Dwelling --> Selecting: Dwell time > 500ms
    Selecting --> Idle: Layout change command sent
    state Selecting {
        note right of Selecting
            Send layout change
            request to composer
        end note
    }

Axis 2: Operational Parameter Expansion

Derivative 2.1: Microscopic Imaging Composition

Enabling Description: The system is applied to a multi-modal microscope (e.g., a confocal microscope with phase-contrast and DIC optics). The "video conference streams" are live feeds from different sensors or imaging modalities of the same sample. The axis control allows a researcher to fluidly transition the main display view between different data sources. For example, one end of the slider shows the fluorescence channel in full screen. Moving the slider fades in a picture-in-picture of the phase-contrast view. Further movement makes the two views a 50/50 split screen. The "video composing unit" is a software module within the microscope's control application, using GPU shaders to composite the raw sensor data into the final image.

Mermaid Diagram:

graph LR
    subgraph Microscope
        A[Fluorescence Sensor]
        B[Phase Contrast Sensor]
        C[DIC Sensor]
    end
    subgraph Control UI
        D{Slider Position}
    end
    subgraph Image Composer (GPU)
        E[Compositing Logic]
    end
    A --> E;
    B --> E;
    C --> E;
    D --> E;
    E --> F[Display Output];

Derivative 2.2: Adaptive Bitrate-Layout Coupling for Unstable Networks

Enabling Description: The system operates in a low-bandwidth or high-latency network environment. The intervals on the layout control axis are associated not only with a visual layout but also with a bitrate budget profile. When the user moves the slider to a "Focus" layout (one large image), the system allocates the majority of the available downstream bandwidth to that single video stream, requesting a high-resolution, high-framerate version from the server (or peer). When the slider is moved to an "Overview" grid layout, the system signals the video composing unit (or SFU) to request low-resolution, low-framerate versions of all streams, ensuring the total bitrate fits within network capacity. The layout and stream quality are thus controlled by a single user action.

Mermaid Diagram:

sequenceDiagram
    participant User;
    participant UIControl;
    participant EndpointLogic;
    participant VideoServer_SFU;
    User->>UIControl: Moves slider to 'Grid' position;
    UIControl->>EndpointLogic: Notify: Position is in 'Grid' interval;
    EndpointLogic->>VideoServer_SFU: Request low-bitrate streams for all participants;
    VideoServer_SFU-->>EndpointLogic: Sends low-bitrate streams;
    EndpointLogic->>EndpointLogic: Compose 'Grid' layout;
    User->>UIControl: Moves slider to 'Focus' position;
    UIControl->>EndpointLogic: Notify: Position is in 'Focus' interval;
    EndpointLogic->>VideoServer_SFU: Request high-bitrate for speaker, pause others;
    VideoServer_SFU-->>EndpointLogic: Sends high-bitrate stream;
    EndpointLogic->>EndpointLogic: Compose 'Focus' layout;

Axis 3: Cross-Domain Application

Derivative 3.1: Aerospace Cockpit Multi-Function Display (MFD) Control

Enabling Description: In an aircraft glass cockpit, a physical bezel knob or rocker switch serves as the axis control. Each detent in the knob's rotation corresponds to an interval mapped to a critical flight display layout. For example, position 1 is a full-screen Primary Flight Display (PFD). Position 2 is a 60/40 split between the PFD and the Navigation Display (ND). Position 3 is a 60/40 split between the PFD and the Engine Indicating and Crew Alerting System (EICAS). Position 4 is a quad-split of PFD, ND, EICAS, and a Flight Management System (FMS) page. The "video composing unit" is the certified avionics display processor, which composites these different data sources into a single, safety-critical display output.

Mermaid Diagram:

graph TD
    A(Bezel Knob) -- selects position --> B{Display Layout Logic};
    B -- Position 1 --> C[PFD Full Screen];
    B -- Position 2 --> D[PFD/ND 60-40 Split];
    B -- Position 3 --> E[PFD/EICAS 60-40 Split];
    subgraph DisplayProcessor
        C --> F(Render Output);
        D --> F;
        E --> F;
    end
    F --> G([Cockpit MFD]);

Derivative 3.2: AgTech Multi-Layer Data Visualization

Enabling Description: A farm management application on a tablet or in-cab terminal for a tractor uses the axis control to switch between views of geospatial data for a specific field. The axis intervals are mapped to different data layers. Interval 1 shows a high-resolution RGB satellite/drone image. Interval 2 overlays an NDVI (Normalized Difference Vegetation Index) map using a color-graded transparency. Interval 3 replaces the RGB view with a soil moisture map from ground sensors. Interval 4 shows a yield map from the previous harvest. The slider allows a farmer to seamlessly "slide through" different layers of data about their field, rather than toggling checkboxes in a complex menu.

Mermaid Diagram:

classDiagram
    class DataLayer {
        +string name
        +render()
    }
    class RgbLayer {
    }
    class NdviLayer {
    }
    class SoilMoistureLayer {
    }
    DataLayer <|-- RgbLayer
    DataLayer <|-- NdviLayer
    DataLayer <|-- SoilMoistureLayer
    class MapView {
        -DataLayer[] layers
        +displayComposition(layout)
    }
    class LayoutController {
        -float sliderPosition
        +getLayoutForPosition()
    }
    LayoutController "1" -- "1" MapView : Controls
    MapView "1" -- "N" DataLayer : Displays

Axis 4: Integration with Emerging Tech

Derivative 4.1: AI-Optimized Conversational Flow Layout

Enabling Description: A machine learning model, trained on video conference data, analyzes the meeting in real-time. It processes audio to identify the active speaker, uses computer vision to detect screen sharing or whiteboard presentations, and applies sentiment analysis to gauge engagement. Based on this multi-modal analysis, the AI determines an "optimal" layout for the current state of the conversation. It then programmatically moves the UI slider to the corresponding interval. For example, during a heated debate with multiple speakers, it moves the slider to a grid view. When one person begins a presentation, it moves the slider to a "Focus" view on the presenter. The user can manually override the AI at any time by moving the slider themselves.

Mermaid Diagram:

sequenceDiagram
    participant RealTimeFeeds;
    participant AI_Analyzer;
    participant LayoutController;
    participant VideoComposer;
    loop Real-time Analysis
        RealTimeFeeds->>AI_Analyzer: Audio, Video, Screen Share data;
        AI_Analyzer->>AI_Analyzer: Analyze speaker, content, sentiment;
        AI_Analyzer->>LayoutController: Recommend optimal layout (e.g., 'Presentation');
    end
    LayoutController->>VideoComposer: Set layout to 'Presentation';
    VideoComposer->>VideoComposer: Re-compose video signal;

Derivative 4.2: IoT-Triggered Industrial Monitoring Layout

Enabling Description: In a factory or infrastructure control room, the video display shows a grid of surveillance cameras. These cameras are linked to IoT sensors (temperature, pressure, vibration) on the machinery they monitor. When a sensor's reading exceeds a predefined threshold (e.g., a pump's temperature is critical), it sends an alert. This alert programmatically drives the layout control slider to a specific interval. This interval is associated with a "Focus+Data" layout, which enlarges the video feed from the relevant camera to fill the screen and simultaneously overlays a real-time data chart from the triggering IoT sensor. This instantly brings the operator's attention to the problem with full context.

Mermaid Diagram:

graph TD
    A[IoT Sensor] -- Value > Threshold --> B(Alert System);
    B -- Triggers Layout Change --> C(Layout Controller);
    C -- Selects 'Focus+Data' Layout --> D(Video Composer);
    E[Camera Feed] --> D;
    A -- Real-time Data --> F(Data Overlay);
    F --> D;
    D --> G([Operator Display]);

Axis 5: The "Inverse" or Failure Mode

Derivative 5.1: Graceful Degradation / Safe-Mode Layout

Enabling Description: The video composing unit continuously monitors its own resource utilization (CPU, memory) and the integrity of incoming video streams (packet loss, jitter). If a critical failure is detected, such as CPU utilization exceeding 95% or the primary speaker's stream being lost, the system invokes a safe mode. This action programmatically forces the layout slider to a pre-configured "fail-safe" interval. This layout is designed to be minimally resource-intensive, for example, showing only participant name tags and a single, low-resolution "last good frame" still image for each. This prevents a catastrophic crash of the conference and allows audio to continue while the system attempts to recover.

Mermaid Diagram:

stateDiagram-v2
    state "Normal Operation" as Normal
    state "Fail-Safe Mode" as Failsafe

    [*] --> Normal
    Normal --> Normal: All systems nominal
    Normal --> Failsafe: CPU > 95% OR PacketLoss > 20%
    Failsafe --> Normal: System resources recover
    Failsafe --> Failsafe: Continue displaying low-resource layout

Combination Prior Art Scenarios

Combination with WebRTC: The core method is implemented in client-side JavaScript within a web application. The layout axis is a standard HTML5 <input type="range"> element. Its value is transmitted to all other participants via a WebRTC RTCDataChannel. Each peer's browser listens for messages on this channel and uses the received value to dynamically update the CSS Grid or Flexbox styles applied to the container holding the <video> elements. The "video composing" is thus fully decentralized, occurring in the browser of each viewer.
Combination with GStreamer: A centralized video conferencing bridge is built using the GStreamer multimedia framework. A videomixer element is used to compose the final video stream. The user's layout control position is sent via a WebSocket message to a control application managing the GStreamer pipeline. This application translates the position (e.g., a float from 0.0 to 1.0) into specific properties for the videomixer pads (e.g., xpos, ypos, width, height, alpha). Moving the slider in the UI results in a real-time, smooth animation of video frames resizing and repositioning in the GStreamer pipeline.
Combination with OBS Studio and NDI Protocol: A user running Open Broadcaster Software (OBS) Studio for a live stream uses this method as a way to control scene layouts. The video streams from conference participants are brought into OBS as sources via the Network Device Interface (NDI) protocol. A custom OBS script or plugin provides the on-screen slider. Moving the slider triggers the script to programmatically switch between different OBS scenes, where each scene is pre-configured with a different layout of the NDI video sources. This allows a single stream producer to dynamically change the composition of a complex broadcast with a simple, intuitive control.