Derivative works — US Patent 8265353

Defensive Disclosure: US Patent 8265353 - Method of Reconstructing an Image Acquired Using Several Imagery Modes

This defensive disclosure aims to broaden the prior art landscape around US patent 8265353, "Method of reconstructing an image acquired using several imagery modes," thereby rendering potential future incremental improvements by competitors as obvious or non-novel. The derivations below explore various substitutions, operational parameters, cross-domain applications, integrations with emerging technologies, and inverse/failure modes, providing technical descriptions and visual representations.

Derivative Variations for Core Claims (Claim 1 & Claim 12)

The core claims describe a method and apparatus for forming a motion-compensated image of a mobile object by fusing data from two different imaging techniques and independent motion sensor systems. The following derivatives expand upon this foundational concept.

1. Material & Component Substitution

This section explores alternative materials, imaging modalities, and sensor technologies that achieve the same functional result of motion-compensated multi-modal imaging.

Derivative 1.1: Multi-modality Ultrasound & Optical Coherence Tomography with MEMS IMUs.

Enabling Description:
A system is disclosed for forming a motion-compensated image of a mobile object, specifically a vibrating micro-device. A plurality of first images are obtained using high-frequency ultrasound imaging (e.g., employing a 50 MHz+ transducer array for acoustic attenuation/reflection). Concurrently, first movement measurements are acquired via an array of surface-mounted Micro-Electro-Mechanical Systems (MEMS) Inertial Measurement Units (IMUs), each providing 6-Degrees of Freedom (6-DOF) positional and orientational data (e.g., using a combination of triaxial accelerometers, gyroscopes, and magnetometers). These ultrasound images are associated with corresponding first movement states of the micro-device, determined from the synchronized IMU data. Simultaneously, a plurality of second images are obtained using Optical Coherence Tomography (OCT) with a swept-source laser (e.g., 1300 nm center wavelength, 100 kHz A-scan rate), providing cross-sectional optical reflectance data. Second movement measurements are acquired using a second set of MEMS IMUs or integrated piezoresistive strain gauges within the micro-device packaging. These OCT images are associated with their respective second movement states. An image processing unit (e.g., an FPGA or GPU-accelerated processor) then performs data fusion of the motion-compensated ultrasound and OCT data streams to reconstruct a high-resolution 3D volumetric image, resolving internal structure and surface dynamics while mitigating motion artifacts through elastic registration techniques.

graph TD
    A[Mobile Object (Micro-device)] --> B(High-Frequency Ultrasound Imaging);
    A --> C(MEMS IMU Array 1);
    B --> D{First Images};
    C --> E{First Measurements};
    D & E --> F[Associate First Images with First Movement States];

    A --> G(Swept-Source OCT Imaging);
    A --> H(MEMS IMU Array 2 / Strain Gauges);
    G --> I{Second Images};
    H --> J{Second Measurements};
    I & J --> K[Associate Second Images with Second Movement States];

    F & K --> L[Image Processing Unit];
    L --> M[Motion-Compensated 3D Volumetric Image];

Derivative 1.2: Terahertz Attenuation & Photoacoustic Emission Imaging with Fiber Optic Strain Sensors.

Enabling Description:
A method is described for reconstructing an image of a mobile object, specifically a composite material component (e.g., a drone wing undergoing vibrational stress). A plurality of first images are obtained using Terahertz (THz) time-domain spectroscopy for attenuation mapping (e.g., 0.1-10 THz frequency range, picosecond pulse duration), revealing internal delaminations or voids. First movement measurements are acquired using embedded fiber optic Bragg grating (FBG) strain sensors distributed across the composite structure, providing localized strain and vibration data (e.g., 1 kHz sampling rate). These THz images are associated with the material's dynamic strain states. Concurrently, a plurality of second images are obtained using Photoacoustic Tomography (PAT), wherein pulsed laser light (e.g., 532 nm, nanosecond pulse) illuminates the material, generating ultrasonic waves detected by an external ultrasonic transducer array, providing molecular absorption contrast (e.g., revealing embedded foreign objects or local heating). Second movement measurements are gathered from additional FBG sensors or integrated piezoelectric accelerometers. The system then forms a motion-compensated composite structural image by combining the THz attenuation maps with PAT functional maps. This fusion leverages the FBG sensor data for precise motion compensation and spatial registration between the two modalities, accounting for both rigid-body motion and local elastic deformation during the imaging period.

graph TD
    A[Mobile Object (Composite Material)] --> B(Terahertz Time-Domain Spectroscopy);
    A --> C(Embedded FBG Strain Sensors 1);
    B --> D{First Images (THz Attenuation)};
    C --> E{First Measurements (Strain/Vibration)};
    D & E --> F[Associate THz Images with Strain States];

    A --> G(Photoacoustic Tomography);
    A --> H(Embedded FBG Strain Sensors 2 / Piezoelectric Accelerometers);
    G --> I{Second Images (PAT Absorption)};
    H --> J{Second Measurements (Strain/Vibration)};
    I & J --> K[Associate PAT Images with Strain States];

    F & K --> L[Image Reconstruction Unit];
    L --> M[Motion-Compensated Composite Structural Image];

Derivative 1.3: Electrical Impedance Tomography & Bioluminescence Imaging with Bio-impedance Sensors.

Enabling Description:
An apparatus is disclosed for forming an image of a mobile object, specifically a living plant (e.g., a plant stem responding to environmental stimuli). Means are provided for obtaining a plurality of first images using Electrical Impedance Tomography (EIT) via a multi-electrode array (e.g., 10 kHz, 32-electrode system), mapping impedance changes related to water potential or sap flow as an attenuation technique. Means for obtaining first movement measurements are integrated bio-impedance sensors measuring subtle dielectric property changes due to turgor pressure variations or localized growth movements (e.g., 100 Hz sampling rate). The means for associating correlates EIT images with these bio-mechanical states. Concurrently, means are provided for obtaining a plurality of second images via bioluminescence imaging, detecting light emitted by genetically modified reporter proteins (emission technique, e.g., luciferase activity indicating metabolic stress or gene expression). Means for obtaining second movement measurements are derived from a second set of high-sensitivity bio-impedance sensors or haptic feedback sensors detecting tissue turgidity changes. A dedicated image reconstruction unit synthesizes a motion-compensated 3D functional map of the plant, integrating the EIT data with the bioluminescence signals, using the bio-impedance data for precise registration and motion artifact suppression (e.g., employing a variational non-rigid registration algorithm).

graph TD
    A[Mobile Object (Living Plant)] --> B(Electrical Impedance Tomography (EIT));
    A --> C(Embedded Bio-impedance Sensors 1);
    B --> D{First Images (EIT)};
    C --> E{First Measurements (Turgor/Growth)};
    D & E --> F[Associate EIT Images with Bio-mechanical States];

    A --> G(Bioluminescence Imaging);
    A --> H(Embedded Bio-impedance Sensors 2 / Haptic Sensors);
    G --> I{Second Images (Bioluminescence)};
    H --> J{Second Measurements (Turgor/Turgidity)};
    I & J --> K[Associate Bioluminescence Images with Bio-mechanical States];

    F & K --> L[Image Reconstruction Unit];
    L --> M[Motion-Compensated 3D Plant Functional Map];

2. Operational Parameter Expansion

This section describes the technology operating at extreme scales (nanoscale to industrial scale) and under extreme conditions (temperature, pressure, frequency).

Derivative 2.1: Ultra-high Resolution Nanoscale Electron Microscopy & X-ray Fluorescence Imaging with Atomic Force Probes.

Enabling Description:
A method is described for forming an image of a mobile object, specifically protein complexes undergoing conformational changes at the nanoscale (sub-nanometer resolution). A plurality of first images are obtained using Cryo-Electron Tomography (Cryo-ET) as the attenuation technique (e.g., 300 keV electron beam, operating at 77K), capturing structural details. First movement measurements are acquired by integrated atomic force microscopy (AFM) probes directly interacting with the sample surface, providing picometer-scale positional feedback (e.g., 10 kHz sampling rate). These Cryo-ET image stacks are associated with precise conformational states determined by the AFM. Concurrently, a plurality of second images are obtained using X-ray Fluorescence (XRF) microscopy (emission technique, e.g., scanning X-ray microprobe at 10 keV) to map elemental composition within the same sample volume, indicating functional states. Second movement measurements are gathered from a second array of AFM probes or high-frequency interferometric displacement sensors. A specialized reconstruction algorithm fuses the Cryo-ET structural data with XRF elemental maps, applying motion compensation derived from the AFM positional data using non-rigid registration techniques (e.g., optical flow methods adapted for cryo-EM data) to resolve subtle conformational dynamics and chemical shifts in a motion-corrected 3D volume, all performed under cryogenic conditions.

graph TD
    A[Mobile Object (Protein Complex)] --> B(Cryo-Electron Tomography);
    A --> C(AFM Probes 1 / Interferometric Sensors);
    B --> D{First Images (Cryo-ET)};
    C --> E{First Measurements (Picometer Position)};
    D & E --> F[Associate Cryo-ET Images with Conformational States];

    A --> G(X-ray Fluorescence Microscopy);
    A --> H(AFM Probes 2 / Interferometric Sensors);
    G --> I{Second Images (XRF)};
    H --> J{Second Measurements (Picometer Position)};
    I & J --> K[Associate XRF Images with Conformational States];

    F & K --> L[Nanoscale Reconstruction Algorithm (Cryogenic)];
    L --> M[Motion-Compensated 3D Nanoscale Volume];

Derivative 2.2: Large-Scale Ground Penetrating Radar (GPR) & Passive Seismic Tomography with Distributed MEMS Accelerometers.

Enabling Description:
A method is disclosed for forming an image of a mobile object, specifically a large-scale geological formation or infrastructure (e.g., a bridge foundation over an active fault line) undergoing slow deformation or seismic shifts. A plurality of first images are obtained using Ground Penetrating Radar (GPR) arrays (e.g., 100 MHz-1 GHz stepped-frequency radar) for shallow subsurface attenuation mapping, detecting voids or material changes. First movement measurements are acquired by a distributed network of ultra-low-noise MEMS accelerometers (e.g., with 100 nG sensitivity, sampled at 1 kHz) strategically placed on the surface and embedded in boreholes, detecting micro-seismic activity and structural vibrations. GPR scans are associated with these dynamic states, often over extended periods (e.g., weekly scans associated with cumulative displacement). Concurrently, a plurality of second images are obtained via passive seismic tomography (emission technique, utilizing natural seismic noise or distant controlled sources in the 0.1-100 Hz range) to map deeper subsurface velocity anomalies, indicating rock density and fluid content. Second movement measurements are gathered from a second array of broadband seismometers or fiber-optic distributed acoustic sensing (DAS) cables. A massive parallel processing unit combines the GPR and seismic tomographic data, applying motion compensation (e.g., using time-lapse image registration and geodetic models) derived from the fused accelerometer/seismometer data, to produce a long-term, motion-corrected 4D model of subsurface deformation and material integrity under various environmental stressors.

graph TD
    A[Mobile Object (Geological Formation/Infrastructure)] --> B(GPR Arrays);
    A --> C(Distributed MEMS Accelerometers 1);
    B --> D{First Images (GPR Attenuation)};
    C --> E{First Measurements (Micro-seismic/Vibration)};
    D & E --> F[Associate GPR Scans with Dynamic States];

    A --> G(Passive Seismic Tomography);
    A --> H(Distributed Seismometers / DAS Cables);
    G --> I{Second Images (Seismic Velocity Anomalies)};
    H --> J{Second Measurements (Seismic Activity)};
    I & J --> K[Associate Seismic Data with Dynamic States];

    F & K --> L[Massive Parallel Processing Unit];
    L --> M[Motion-Compensated 4D Subsurface Model];

Derivative 2.3: High-Frequency Millimeter-Wave Radar & Calorimetry Imaging at Extreme Temperatures with Pyrometric Strain Gauges.

Enabling Description:
An apparatus is disclosed for forming an image of a mobile object, specifically turbine blades operating within a high-temperature industrial process (e.g., 1500°C). Means are provided for obtaining a plurality of first images using a focused millimeter-wave radar system (e.g., 94 GHz FMCW radar, 100 Hz frame rate) to detect internal defects and cracks via reflection/attenuation changes. Means for obtaining first movement measurements are integrated pyrometric strain gauges (e.g., fiber Bragg gratings embedded in sapphire, capable of operating up to 1800°C) directly integrated into the blade, providing real-time temperature and strain data under extreme thermal loads. The means for associating correlates millimeter-wave images with these high-temperature strain states. Concurrently, means are provided for obtaining a plurality of second images via high-speed infrared calorimetry (emission technique, e.g., uncooled microbolometer array operating in the 8-14 µm range, 500 Hz frame rate) to map thermal energy dissipation and hot spots, indicating areas of material stress or degradation. Means for obtaining second movement measurements are acquired from additional pyrometric strain gauges and non-contact laser Doppler vibrometers. A specialized image processor fuses the millimeter-wave structural data with the calorimetric thermal maps, leveraging the high-temperature strain data for precise motion compensation and thermal deformation correction (e.g., using a coupled thermo-mechanical deformation model), reconstructing a high-fidelity 3D structural and thermal integrity map of the turbine blade operating under extreme conditions.

graph TD
    A[Mobile Object (Turbine Blade @ 1500°C)] --> B(Millimeter-Wave Radar);
    A --> C(Pyrometric Strain Gauges 1);
    B --> D{First Images (Radar Defects)};
    C --> E{First Measurements (Temp/Strain)};
    D & E --> F[Associate Radar Images with High-Temp States];

    A --> G(High-Speed Infrared Calorimetry);
    A --> H(Pyrometric Strain Gauges 2 / Laser Vibrometers);
    G --> I{Second Images (Thermal Maps)};
    H --> J{Second Measurements (Temp/Strain/Vibration)};
    I & J --> K[Associate Calorimetry Images with High-Temp States];

    F & K --> L[Image Processor (Extreme Temp)];
    L --> M[Motion-Compensated 3D Structural/Thermal Map];

3. Cross-Domain Application

This section demonstrates the applicability of the patent's mechanism in three unrelated industries.

Derivative 3.1: Aerospace - Satellite In-Orbit Inspection using Passive Microwave Radiometry & Active X-ray Backscatter with Optical Tracking.

Enabling Description:
A method is described for forming an image of a mobile object, specifically a satellite or space debris in low-Earth orbit. A plurality of first images are obtained using a multi-frequency passive microwave radiometer array (e.g., 10-300 GHz, multi-beam) as an attenuation technique, sensing thermal emission variations attenuated by internal structures, thus detecting anomalies in insulation or fluid tanks. First movement measurements are acquired by external optical tracking systems (e.g., star trackers and lidar ranging sensors) on a companion inspection satellite, providing precise 6-DOF relative motion data (e.g., sub-millimeter precision). Microwave images are associated with these relative orbital states. Concurrently, a plurality of second images are obtained using an active X-ray backscatter imager (emission technique, e.g., scanning electron beam inducing characteristic X-ray emission from materials, 5-50 keV) to identify material composition and hidden defects. Second movement measurements are obtained from a second optical tracking system or onboard inertial navigation units. A space-hardened processing unit combines the microwave radiometry and X-ray backscatter data, using the high-precision optical tracking data for motion compensation and alignment (e.g., Kalman filter-based fusion for state estimation), to reconstruct a comprehensive 3D integrity map of the target satellite, accounting for its complex orbital dynamics and internal oscillations.

graph TD
    A[Mobile Object (Satellite in Orbit)] --> B(Passive Microwave Radiometer);
    A --> C(Optical Tracking System 1);
    B --> D{First Images (Microwave Attenuation)};
    C --> E{First Measurements (6-DOF Relative Motion)};
    D & E --> F[Associate Microwave Images with Orbital States];

    A --> G(Active X-ray Backscatter Imager);
    A --> H(Optical Tracking System 2 / Onboard IMU);
    G --> I{Second Images (X-ray Backscatter)};
    H --> J{Second Measurements (6-DOF Relative Motion)};
    I & J --> K[Associate X-ray Images with Orbital States];

    F & K --> L[Space-Hardened Processing Unit];
    L --> M[Motion-Compensated 3D Satellite Integrity Map];

Derivative 3.2: Agri-Tech - Subsurface Root & Soil Analysis using Electrical Resistivity Tomography & Neutron Tomography with Haptic Soil Sensors.

Enabling Description:
An apparatus is disclosed for forming an image of a mobile object, specifically a plant root system and its surrounding soil in an agricultural field. Means are provided for obtaining a plurality of first images using Electrical Resistivity Tomography (ERT) with an array of soil-embedded electrodes (e.g., 64-electrode array, 1-10 Hz measurement cycle) to map soil moisture content and root biomass (attenuation technique). Means for obtaining first movement measurements are a network of haptic soil sensors (e.g., piezoelectric films, fiber optic deformation sensors) detecting localized root pressure, growth, and soil displacement (e.g., 10 Hz sampling). The means for associating correlates ERT data with these root/soil interaction states. Concurrently, means are provided for obtaining a plurality of second images using low-dose neutron tomography (emission technique, e.g., using neutron-activated tracers within the plant or soil, 10-minute acquisition) to map water distribution within the root tissue or nutrient uptake. Means for obtaining second movement measurements are derived from a second array of haptic soil sensors or accelerometers embedded in the root zone. An agricultural data processing system integrates the ERT and neutron tomography data, applying motion compensation (e.g., using biomechanical models of root growth and soil mechanics) based on the haptic sensor feedback, to reconstruct a dynamic 3D model of root architecture, water uptake, and soil interaction, compensating for continuous root growth and soil settlement.

graph TD
    A[Mobile Object (Plant Root System & Soil)] --> B(Electrical Resistivity Tomography (ERT));
    A --> C(Haptic Soil Sensors 1);
    B --> D{First Images (ERT Soil/Root Map)};
    C --> E{First Measurements (Root Pressure/Soil Displacement)};
    D & E --> F[Associate ERT Data with Root/Soil States];

    A --> G(Low-Dose Neutron Tomography);
    A --> H(Haptic Soil Sensors 2 / Soil Accelerometers);
    G --> I{Second Images (Neutron Water/Nutrient Map)};
    H --> J{Second Measurements (Root Growth/Soil Movement)};
    I & J --> K[Associate Neutron Data with Root/Soil States];

    F & K --> L[Agricultural Data Processing System];
    L --> M[Dynamic 3D Root/Soil Interaction Model];

Derivative 3.3: Manufacturing/Robotics - Robotic Arm Integrity Monitoring using Ultrasonic Phased Array & Acoustic Emission with Laser Interferometry.

Enabling Description:
A method is described for continuous quality control and predictive maintenance of a multi-axis robotic arm (mobile object) during its operational cycles. A plurality of first images are obtained using an ultrasonic phased array system (e.g., 5 MHz, 128-element array) to detect internal cracks, voids, or delaminations in structural components (attenuation technique). First movement measurements are acquired by high-precision laser interferometers (e.g., sub-micron resolution, 10 kHz sampling) tracking the 6-DOF position and vibration of the robotic arm joints. Ultrasonic scans are associated with these dynamic motion states. Concurrently, a plurality of second images are obtained using an acoustic emission (AE) monitoring system (emission technique, e.g., broadband piezoelectric sensors, 1 MHz sampling) to pinpoint active material degradation events (e.g., crack propagation, friction, impact). Second movement measurements are obtained from additional laser interferometers or accelerometers mounted on the arm. A manufacturing control system fuses the ultrasonic structural data with the acoustic emission functional data. The laser interferometry data is used for precise motion compensation and dynamic alignment (e.g., using real-time Kalman filtering and inverse kinematic models), to generate a continuous 3D health map of the robotic arm, predicting potential failure points and enabling proactive maintenance schedules.

graph TD
    A[Mobile Object (Robotic Arm)] --> B(Ultrasonic Phased Array System);
    A --> C(Laser Interferometers 1);
    B --> D{First Images (Ultrasonic Defects)};
    C --> E{First Measurements (6-DOF Position/Vibration)};
    D & E --> F[Associate Ultrasonic Scans with Motion States];

    A --> G(Acoustic Emission Monitoring System);
    A --> H(Laser Interferometers 2 / Accelerometers);
    G --> I{Second Images (AE Event Locations)};
    H --> J{Second Measurements (Vibration/Displacement)};
    I & J --> K[Associate AE Data with Motion States];

    F & K --> L[Manufacturing Control System];
    L --> M[Continuous 3D Robotic Arm Health Map];

4. Integration with Emerging Tech

This section integrates the patent's principles with AI-driven optimization, IoT sensors, and blockchain for data integrity.

Derivative 4.1: AI-Optimized Multimodal Medical Imaging with IoT-Enabled Bio-sensors.

Enabling Description:
A method is disclosed for forming an image of a patient (mobile object) using combined Positron Emission Tomography (PET) and Magnetic Resonance Imaging (MRI). A plurality of first images (MRI, attenuation) and second images (PET, emission) are obtained. First and second movement measurements are acquired via a distributed network of IoT-enabled wearable bio-sensors (e.g., galvanic skin response sensors, respiration belts, ECG patches, miniature accelerometers, optical plethysmography modules) providing high-frequency, synchronized physiological data streams wirelessly (e.g., via Bluetooth Low Energy). An AI-driven optimization module, comprising deep learning networks (e.g., U-Net for image reconstruction, Recurrent Neural Networks for motion prediction), dynamically processes the raw image data and sensor measurements. This AI module performs: (1) real-time anomaly detection in sensor data, (2) adaptive gating and phase association based on predicted cyclic movements, (3) iterative motion estimation and correction using generative adversarial networks, (4) spatial synchronization refinement, and (5) enhanced image reconstruction. Weighting factors (a, b, c, d, etc., from the patent's minimization function) are adaptively tuned by a reinforcement learning agent to maximize image quality (e.g., sharpness, signal-to-noise ratio, artifact reduction) based on the current patient state and data characteristics. The final image of the mobile object is formed by the AI-optimized fusion of the motion-corrected PET and MRI data.

graph TD
    A[Patient (Mobile Object)] --> B(MRI Acquisition);
    A --> C(PET Acquisition);
    A --> D(IoT-Enabled Bio-sensors);

    B --> E{First Images (MRI)};
    C --> F{Second Images (PET)};
    D --> G{First & Second Measurements};

    E & F & G --> H[AI-Driven Optimization Module];
    H --> H1(Real-time Anomaly Detection);
    H --> H2(Adaptive Gating & Phase Association);
    H --> H3(Iterative Motion Estimation & Correction);
    H --> H4(Spatial Synchronization Refinement);
    H --> H5(Reinforcement Learning for Weighting Factors);

    H --> I[AI-Optimized Fused Image];

Derivative 4.2: Real-time Industrial Component Inspection with Edge AI and Secure Blockchain Data Logging.

Enabling Description:
An apparatus is disclosed for real-time inspection of mobile objects, such as hot-rolled steel beams on a production line. Means for obtaining a plurality of first images use eddy current testing (ECT, attenuation, e.g., multi-coil array at 100 kHz) to detect surface and subsurface flaws. Means for obtaining first movement measurements are high-speed vision systems integrated with edge AI for feature tracking and optical flow analysis (e.g., 200 FPS cameras, YOLOv8 object detection, implemented on an NVIDIA Jetson platform). ECT data is associated with dynamically identified motion states. Concurrently, means for obtaining a plurality of second images use infrared thermography (IRT, emission, e.g., 30 Hz microbolometer array) to detect thermal gradients indicating material stress or heat loss. Means for obtaining second movement measurements are a second set of edge AI vision systems or laser displacement sensors. An edge computing unit processes the imaging data and sensor measurements, performing real-time motion estimation and fusion using specialized convolutional neural networks. Crucially, all raw data, processed motion fields, and final image reconstruction parameters (e.g., sensor calibration data, motion vectors, fusion weights) are cryptographically hashed and immutably logged onto a distributed ledger (blockchain, e.g., Hyperledger Fabric) in real-time. This provides a verifiable audit trail for quality assurance, compliance, and supply chain transparency, ensuring that all components of the "forming an image" method are tamper-proof and traceable from acquisition to final reconstruction.

graph TD
    A[Mobile Object (Steel Beam)] --> B(Eddy Current Testing);
    A --> C(Edge AI Vision System 1);
    B --> D{First Images (ECT Flaw Map)};
    C --> E{First Measurements (Feature Tracking/Optical Flow)};
    D & E --> F[Associate ECT Data with Motion States];

    A --> G(Infrared Thermography);
    A --> H(Edge AI Vision System 2 / Laser Displacement);
    G --> I{Second Images (IRT Thermal Map)};
    H --> J{Second Measurements (Feature Tracking/Displacement)};
    I & J --> K[Associate IRT Data with Motion States];

    F & K --> L[Edge Computing Unit];
    L --> M[Motion-Compensated Fused Image];
    M --> N[Blockchain Data Logging (Hashed Data)];

Derivative 4.3: Smart City Infrastructure Monitoring with IoT-integrated Strain Gauges and Predictive Maintenance AI.

Enabling Description:
A method is described for long-term monitoring of structural integrity in smart city infrastructure (mobile object, e.g., a bridge section experiencing environmental stress). A plurality of first images are obtained using Ground Penetrating Radar (GPR) for internal defect detection and material density mapping (e.g., vehicle-mounted GPR, monthly scans). First movement measurements are acquired by an IoT network of wirelessly connected strain gauges, accelerometers, and displacement sensors (e.g., LoRaWAN-enabled sensors, 1 Hz reporting frequency) embedded within the bridge structure, reporting real-time vibration and deformation data. GPR scans are correlated with these structural movement states over time. Concurrently, a plurality of second images are obtained using thermographic cameras (passive infrared, emission, e.g., drone-mounted, quarterly scans) to detect thermal anomalies indicating structural weaknesses or active corrosion. Second movement measurements are from additional IoT sensors, including inclinometers and GPS-based differential displacement sensors. A central cloud-based AI platform (e.g., Azure IoT Hub, Google Cloud AI Platform) aggregates and synchronizes all IoT sensor data and imaging streams. A predictive maintenance AI model uses this combined, motion-compensated dataset to forecast structural fatigue, schedule inspections, and alert authorities to potential failures, forming a dynamic, continuously updated image of the bridge's structural health. The AI system also uses inverse kinematic models to refine motion estimates based on the observed sensor data.

graph TD
    A[Mobile Object (Bridge Section)] --> B(GPR Acquisition);
    A --> C(IoT Sensor Network 1 (Strain, Accel, Disp));
    B --> D{First Images (GPR)};
    C --> E{First Measurements (Vibration, Deformation)};
    D & E --> F[Correlate GPR Scans with Structural States];

    A --> G(Thermographic Camera Acquisition);
    A --> H(IoT Sensor Network 2 (Inclino, GPS Diff));
    G --> I{Second Images (Thermal)};
    H --> J{Second Measurements (Inclination, Displacement)};
    I & J --> K[Correlate Thermal Images with Structural States];

    F & K --> L[Cloud-Based AI Platform];
    L --> L1(Data Aggregation & Synchronization);
    L --> L2(Predictive Maintenance AI Model);
    L --> L3(Inverse Kinematic Refinement);

    L --> M[Dynamic Structural Health Map];

5. The "Inverse" or Failure Mode

This section describes versions of the invention designed to fail safely or operate in a low-power/limited-functionality mode.

Derivative 5.1: Minimal Viable Image Reconstruction for Emergency Diagnostics in Low-Power Mode.

Enabling Description:
A method is disclosed for forming an image of a patient (mobile object) in a medical imaging system (e.g., a combined X-ray CT and SPECT scanner) operating in a low-power, emergency mode due to limited battery or system resources. Instead of full 3D volumetric reconstruction, the system prioritizes minimal, critical diagnostic information through 2D projection images. A plurality of first images are obtained as sparse X-ray attenuation projections (reduced dose, fewer angles, e.g., 4 projections at 90-degree intervals). First movement measurements are acquired from a single, low-power accelerometer (e.g., MEMS accelerometer with wake-on-motion, 10 Hz sampling). These are associated with simplified "gross movement" states (e.g., "breathing in," "breathing out," "stationary"). Concurrently, a plurality of second images are obtained as limited-angle SPECT projections (reduced acquisition time, fewer detector rotations, e.g., 2 rotations over 180 degrees) and second movement measurements from a simple respiratory belt sensor. The "forming an image" step simplifies the displacement field calculation to rigid body transformations only (translation and rotation) and employs a reduced-complexity registration algorithm (e.g., phase correlation). Weighting factors in the patent's minimization function are fixed (e.g., a=0, b=1, c=0.5, d=0.5, i=0) to prioritize the more robust attenuation data and its motion estimate. The output is a series of motion-compensated 2D anatomical and functional projection images, or a coarsely reconstructed 3D volume, providing sufficient information for immediate, critical diagnostic decisions while conserving power (e.g., maintaining system operation for up to 4 hours on internal battery).

stateDiagram
    [*] --> Normal_Operation: Full power, full features
    Normal_Operation --> Low_Power_Mode: Battery low / Resource constraint
    Low_Power_Mode --> Limited_Functionality: Further degradation / Critical data only
    
    state Normal_Operation {
        Full_3D_Reconstruction
        High_Res_Imaging_Modes
        Dense_Displacement_Fields
    }

    state Low_Power_Mode {
        Simplified_Motion_Estimates
        Sparse_Projection_Images
        Reduced_Acquisition_Parameters
        Minimal_Viable_Image: Coarse 3D / 2D Projections
    }

    state Limited_Functionality {
        Sensor_Priority: Most robust data only
        Boolean_Status_Output: OK/NOT OK for critical aspects
        Emergency_Alerts
    }

    Limited_Functionality --> Normal_Operation: Power restored / Resources available
    Low_Power_Mode --> Normal_Operation: Power restored / Resources available

Derivative 5.2: Degraded Mode Industrial Inspection with Failure Prediction and Redundant Sensing.

Enabling Description:
An apparatus is disclosed for industrial inspection of a moving metal part (mobile object), employing magnetic particle inspection (MPI) for surface flaws and ultrasonic testing (UT) for subsurface defects. In the event of a system fault, if one imaging modality (e.g., MPI camera system) or an independent sensor system (e.g., optical encoder for motion) fails, the system transitions into a degraded mode. The primary imaging technique (ultrasonic phased array, e.g., 10 MHz, 64-element) continues to acquire first images. Redundant accelerometers (e.g., industrial-grade piezoelectric accelerometers) and strain gauges provide first movement measurements, compensating for the lost optical encoder data by estimating the object's velocity and displacement. For the failed magnetic particle inspection, the system attempts to infer surface flaw information from ultrasonic data characteristics (e.g., specific UT wave scattering patterns) or relies on historical data/pre-trained machine learning models for "soft" emission imaging. If a secondary imaging technique (e.g., IR thermography as a proxy for MPI) also degrades, the system automatically switches to a "safety-only" mode. In this mode, only the most robust sensor data (e.g., a single high-accuracy laser displacement sensor) is used to estimate gross movement (e.g., part presence/absence, speed), and a highly simplified "OK/NOT OK" status for critical dimensions or major defects is generated. A dedicated failure prediction module (e.g., using Bayesian inference) constantly assesses sensor health and adjusts the weighting factors in the image formation algorithm based on the confidence scores of the available data streams, ensuring the most reliable output is generated given the system's degraded state, thus providing continuous, albeit reduced, inspection capability.

stateDiagram
    [*] --> Normal_Operation_Industrial: All systems active
    Normal_Operation_Industrial --> Degraded_Mode: MPI Camera OR Optical Encoder Failure
    Degraded_Mode --> Safety_Only_Mode: Secondary Imaging OR Redundant Sensor Failure
    
    state Normal_Operation_Industrial {
        MPI_Imaging
        Ultrasonic_Testing
        Optical_Encoder_Motion
        Full_Defect_Map
    }

    state Degraded_Mode {
        Ultrasonic_Testing_Only
        Redundant_Accelerometer_Motion
        Inferred_Surface_Flaws: From UT / ML Model
        Partial_Defect_Map
    }

    state Safety_Only_Mode {
        Laser_Displacement_Motion
        Boolean_Pass_Fail_Status
        Critical_Dimension_Check
        System_Shut_Down_Flag
    }

    Safety_Only_Mode --> Normal_Operation_Industrial: All systems restored
    Degraded_Mode --> Normal_Operation_Industrial: Failed component repaired

Derivative 5.3: Limited-Functionality Environmental Monitoring with Autonomous Power Management.

Enabling Description:
A method is described for environmental monitoring of remote, mobile objects (e.g., a seismic sensor array with a chemical sensor package deployed on a glacier, undergoing slow deformation). The system operates with limited solar power and autonomously manages its energy consumption. In "low-power" states (e.g., battery voltage below 30%), it obtains a plurality of first images as infrequent, low-resolution satellite radar interferometry (InSAR, attenuation) data (e.g., weekly acquisitions, 10m resolution). First movement measurements are acquired from low-frequency GPS receivers (e.g., single-frequency L1 GPS, 1-hour update rate). These are associated with coarse deformation states. Concurrently, it obtains a plurality of second images as infrequent, low-resolution hyperspectral imagery (emission, e.g., identifying chemical signatures from meltwater with 50m resolution, daily snapshot), with second movement measurements from a reduced set of accelerometers (e.g., 3-axis MEMS accelerometer, 0.1 Hz sampling). The "forming an image" process is conducted in a computationally efficient manner, perhaps by reducing the number of iterations in the reconstruction algorithm or downsampling data (e.g., bilateral filtering for noise reduction, 2x downsampling). The system intelligently conserves power by: (1) powering down non-essential sensors/modalities, (2) transmitting only aggregated motion compensation parameters instead of raw data, (3) dynamically adjusting the desired image resolution based on available energy, and (4) prioritizing the most critical data streams (e.g., rapid deformation events vs. subtle chemical changes) via a fuzzy logic controller. The goal is to provide continuous, albeit spatially and temporally sparse, motion-compensated environmental insights, even under severe energy constraints, until full power is restored.

stateDiagram
    [*] --> Full_Power_Environmental: All sensors/modalities active
    Full_Power_Environmental --> Low_Power_Environmental: Battery_Level < 30%
    Low_Power_Environmental --> Critical_Data_Only: Battery_Level < 10% / Sensor_Failure
    
    state Full_Power_Environmental {
        High_Res_InSAR
        High_Res_Hyperspectral
        Dense_GPS_Tracking
        Full_Sensor_Network
    }

    state Low_Power_Environmental {
        Infrequent_Low_Res_InSAR
        Infrequent_Low_Res_Hyperspectral
        Low_Freq_GPS_Tracking
        Reduced_Accelerometer_Set
        Aggregated_Data_Transmission
    }

    state Critical_Data_Only {
        Only_Most_Critical_Sensors
        Minimal_Data_Transmission
        Emergency_Alert_Mode
        System_Hibernation
    }

    Critical_Data_Only --> Full_Power_Environmental: Solar_Recharge_Sufficient
    Low_Power_Environmental --> Full_Power_Environmental: Solar_Recharge_Sufficient

Combination Prior Art Scenarios with Open-Source Standards

These scenarios illustrate how the methods and apparatus described in US8265353, when combined with existing open-source standards, would be rendered obvious to a person skilled in the art.

1. DICOM (Digital Imaging and Communications in Medicine) Standard for Image and Data Handling

Scenario: The method of US8265353 is implemented within a medical imaging workflow that strictly adheres to the DICOM standard (ISO 12052). Specifically, for a patient undergoing a combined CT/PET scan for cardiac motion compensation, the first images (CT scans) and second images (PET scans) are acquired and immediately formatted as DICOM CT and DICOM PET objects, respectively. The first and second movement measurements, derived from independent ECG and respiration sensors, are encapsulated within DICOM Secondary Capture objects or custom private DICOM tags, ensuring all ancillary data is part of the DICOM patient record. The "first movement states" and "second movement states" (e.g., end-systole, end-diastole, specific respiration phases) are represented within DICOM Structured Reports or as DICOM Real-time Measurement objects. The estimated "enhanced displacement fields" are stored as DICOM Spatial Registration objects, and the final motion-compensated image of the patient is reconstructed and saved as a DICOM Enhanced CT, Enhanced PET, or a dedicated DICOM Blended image object. This ensures end-to-end interoperability, standardized archival, and retrieval of all image and motion data components described by the patent within a ubiquitous medical imaging framework.

2. OpenCV (Open Source Computer Vision Library) for Motion Estimation and Image Processing

Scenario: The digital image processing components of US8265353, particularly the "estimating movement" (steps Et 7, Et 8, Et 9) and "spatial synchronization" (Et 6), are implemented using functions and algorithms readily available in the open-source OpenCV library. For instance, partial attenuation images ($f_t$) and emission images ($g_t$) are loaded and processed using OpenCV's Mat objects. Feature detection (e.g., using cv::SIFT, cv::ORB) and matching algorithms are applied to identify corresponding points across images from different phases. Optical flow estimation (e.g., cv::calcOpticalFlowFarneback) is used to derive dense displacement fields ($mf, mg$). Image registration for spatial synchronization (deformation operator $D$) is performed using cv::findHomography or cv::estimateAffine2D to align coordinate systems. The convex functions ($\phi, \phi_1, \phi_2, \phi_3, \phi_4$) within the patent's minimization equations are instantiated using standard image similarity metrics (e.g., Sum of Squared Differences (SSD) or Normalized Cross-Correlation (NCC)), which are directly calculable or implementable with OpenCV's array operations. The final motion-compensated image reconstruction (Et 10) leverages OpenCV's image warping (cv::warpPerspective, cv::warpAffine) and blending functionalities to combine the motion-corrected partial images.

3. ROS (Robot Operating System) for Sensor Integration and Data Synchronization

Scenario: An apparatus implementing the principles of US8265353 is a mobile robotic platform (e.g., an autonomous inspection robot) configured with multiple imaging modalities and independent sensor systems, all integrated and synchronized via the Robot Operating System (ROS) framework. For example, a 3D LIDAR sensor serves as the "first imaging technique" (attenuation, providing point clouds), and a stereo camera provides "second images" (visual spectrum). Independent Inertial Measurement Units (IMUs) and wheel encoders serve as the "first and second sensor systems" for obtaining motion measurements. Each sensor publishes its data (e.g., sensor_msgs/PointCloud2, sensor_msgs/Image, sensor_msgs/Imu, nav_msgs/Odometry) as ROS messages on dedicated topics. ROS's built-in time synchronization mechanisms (e.g., ros::Time::now(), message_filters::sync_policies) ensure that all data streams are aligned to a common timeline, addressing the synchronization steps (Et 3, Et 13, Et 14). Separate ROS nodes are developed to perform "means for associating" (correlating images with movement states using, for instance, a tf listener to manage coordinate transformations) and "means for forming an image" (another node performing scan matching for LIDAR-based displacement fields, visual odometry for camera-based motion, and fusing these using a custom ROS service or action based on the patent's minimization function). The ROS TF (Transform) tree is explicitly used to manage coordinate frames and apply rigid body transformations for the deformation operator $D$ and various displacement fields ($m, mf, mg$). This demonstrates that the patent's apparatus claims are straightforwardly realized within a standard robotics operating environment.