motorsignal
/
MotorSignal-App
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
MotorSignal-App/PHASE10_FIELD_TEST_REPORT.md

10 KiB
Raw Blame History

Phase 10: Real-World Field Validation & Forensic Ride Logging Final Report

Executive Summary

Attribute Status / Value
Executive Verdict READY FOR REAL-WORLD FIELD TEST
Validation Classification Comprehensive Software & Synthetic Validation Passed (Physical Real-World On-Motorcycle Validation is Pending Rider Execution)
Telemetry Pipeline Architecture 20Hz Heartbeat, Sensor Fusion EKF, GPS Multi-Path Filter, Lean/Pitch IMU Drift Compensation
Forensic Event Engine Asynchronous, Bounded 1,000-Event Prioritized Drop Queue, SQLite Batch Persistence
Event De-duplication State-Machine Level (Single Warning on Entry, Single Recovery Event with Excursion Duration/Metrics on Exit)
Database Schema Version Schema 5 (field_test_events, field_test_summaries with full indexing on rideId, timestamp, eventType, severity)
Test Suite Results 100 Tests Executed, 100 Passed (0 Failed, 0 Flaky)
Release Artifact Status Built Cleanly (app-release.apk 63.3 MB)

1. Executive Verdict & Validation Boundary

+==================================================================================================+
|                                    EXECUTIVE VERDICT                                             |
|                                                                                                  |
|                      >>> READY FOR REAL-WORLD FIELD TEST <<<                                     |
|                                                                                                  |
|  DISTINCTION NOTICE:                                                                             |
|  1. Software Validation: COMPLETE. Telemetry ingest, quality scoring, event de-duplication,    |
|     forensic event batching, crash recovery, and export systems are fully verified.             |
|  2. Physical Validation: PENDING. Real physical dynamics, engine vibration, cellular handover,   |
|     and thermal throttling require mounting the device to a motorcycle for a 30-60 min test.   |
+==================================================================================================+

2. Forensic Event Architecture & Taxonomy

The forensic system captures 40 standard event categories across critical system boundaries without blocking the real-time 20Hz telemetry loop:

flowchart TD
    A[Hardware Sensors\nGPS / IMU / Compass] --> B[TelemetryService 20Hz]
    B --> C[TelemetryValidator]
    C -->|Validated Sample + QualityScore| D[FieldTestLogger]
    D -->|State Machine Analysis| E{Threshold Cross / State Change?}
    E -->|Yes| F[Bounded Prioritized Queue\nCap: 1000 Events]
    E -->|Duplicate State| G[Accumulate Max Excursions]
    F -->|Flush Every 3s| H[(SQLite Database\nfield_test_events)]
    F -->|Live Stream| I[In-Memory Circular Buffer\n500 Events -> UI Timeline]

Forensic Event Categorization (36 Enum Types)

  1. Lifecycle & Session: RIDE_STARTED, RIDE_STOPPED, APP_PAUSED, APP_RESUMED, FOREGROUND_SERVICE_STARTED, FOREGROUND_SERVICE_STOPPED
  2. GPS Pipeline: GPS_STALE, GPS_RECOVERED, GPS_JUMP_REJECTED, GPS_ACCURACY_DEGRADED, GPS_INITIALIZED, GPS_ERROR
  3. IMU & Motion Pipeline: IMU_SATURATION, IMU_RECOVERED, SENSOR_INITIALIZED, SENSOR_ERROR, CALIBRATION_STARTED, CALIBRATION_COMPLETED
  4. Telemetry Heartbeat: TELEMETRY_GAP, TELEMETRY_RECOVERED, TIMESTAMP_ERROR, SAMPLE_ACCEPTED, SAMPLE_SUSPECT, SAMPLE_REJECTED
  5. Dynamics & Performance: ACCELERATION_EVENT, BRAKING_EVENT, PERFORMANCE_RUN_STARTED, PERFORMANCE_RUN_COMPLETED, PERFORMANCE_RUN_REJECTED
  6. Database & Storage: DATABASE_FLUSH, DATABASE_QUEUE_WARNING, DATABASE_QUEUE_OVERFLOW, DATABASE_RECOVERY
  7. Road Safety & Alerts: ROAD_ALERT_RECEIVED, TTS_ANNOUNCED, AUDIO_FAILURE, NETWORK_ERROR
  8. Export Subsystem: EXPORT_STARTED, EXPORT_COMPLETED, EXPORT_FAILED

3. Event De-duplication Strategy

High-frequency 20Hz telemetry can emit 1,200 samples per minute. To prevent database bloating and log inundation, stateful de-duplication was implemented:

Outage / Fault State Transition In (Entry) Sustained Outage Behavior Transition Out (Exit / Recovery)
GPS Fix Staleness (>2000ms at >3 MPH) Emits 1 GPS_STALE warning with initial age and speed Accumulates maxGpsAgeMs, minAccuracy, maxAccuracy silently Emits 1 GPS_RECOVERED info event with total outage duration, max GPS age, and speed
GPS Accuracy Degradation (>35m) Emits 1 GPS_ACCURACY_DEGRADED warning Updates worst accuracy tracker Emits 1 GPS_RECOVERED info event when precision drops below \le 15\text{m}
IMU Saturation (>120\text{ m/s}^2) Emits 1 IMU_SATURATION error Prevents redundant event generation Emits 1 IMU_RECOVERED info event upon returning to normal dynamic range
Telemetry Gap (>2000\text{ms}) Emits 1 TELEMETRY_GAP warning with delta time Tracks max gap length Emits 1 TELEMETRY_RECOVERED info event with elapsed gap seconds

4. Database Schema & Migration (Version 5)

Drift database upgraded cleanly to schemaVersion = 5 with optimized B-tree indices:

-- Forensic Events Table
CREATE TABLE IF NOT EXISTS field_test_events (
    id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
    ride_id INTEGER REFERENCES rides(id),
    timestamp INTEGER NOT NULL,
    event_type TEXT NOT NULL,
    severity TEXT NOT NULL,
    message TEXT NOT NULL,
    telemetry_timestamp INTEGER,
    gps_age_ms INTEGER,
    speed REAL,
    latitude REAL,
    longitude REAL,
    gps_accuracy REAL,
    lean_angle REAL,
    longitudinal_g REAL,
    lateral_g REAL,
    overall_confidence REAL,
    metadata TEXT
);
CREATE INDEX IF NOT EXISTS idx_field_test_events_ride_id ON field_test_events(ride_id);
CREATE INDEX IF NOT EXISTS idx_field_test_events_timestamp ON field_test_events(timestamp);
CREATE INDEX IF NOT EXISTS idx_field_test_events_type ON field_test_events(event_type);
CREATE INDEX IF NOT EXISTS idx_field_test_events_severity ON field_test_events(severity);

-- Forensic Ride Summaries Table
CREATE TABLE IF NOT EXISTS field_test_summaries (
    id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
    ride_id INTEGER NOT NULL REFERENCES rides(id),
    start_time INTEGER NOT NULL,
    end_time INTEGER,
    duration_seconds INTEGER NOT NULL,
    total_samples INTEGER NOT NULL,
    valid_samples INTEGER NOT NULL,
    suspect_samples INTEGER NOT NULL,
    rejected_samples INTEGER NOT NULL,
    valid_percentage REAL NOT NULL,
    suspect_percentage REAL NOT NULL,
    rejected_percentage REAL NOT NULL,
    gps_fix_count INTEGER NOT NULL,
    gps_stale_event_count INTEGER NOT NULL,
    gps_jump_reject_count INTEGER NOT NULL,
    max_gps_age_ms INTEGER,
    average_gps_accuracy REAL,
    worst_gps_accuracy REAL,
    telemetry_gap_count INTEGER NOT NULL,
    max_telemetry_gap_ms INTEGER,
    imu_saturation_count INTEGER NOT NULL,
    max_speed_mph REAL,
    max_lean_angle REAL,
    max_longitudinal_g REAL,
    max_lateral_g REAL,
    acceleration_event_count INTEGER NOT NULL,
    braking_event_count INTEGER NOT NULL,
    zero_to_60_count INTEGER NOT NULL,
    zero_to_100_count INTEGER NOT NULL,
    best_zero_to_60 REAL,
    best_zero_to_100 REAL,
    database_samples_written INTEGER NOT NULL,
    database_samples_lost INTEGER NOT NULL,
    event_count INTEGER NOT NULL,
    overall_confidence REAL NOT NULL,
    ride_quality_score INTEGER NOT NULL,
    ride_quality_rating TEXT NOT NULL,
    was_recovered_after_crash INTEGER NOT NULL,
    was_interrupted INTEGER NOT NULL,
    export_integrity TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_field_test_summaries_ride_id ON field_test_summaries(ride_id);

5. Forensic Export Bundle Specification

The export subsystem (field_test_export_service.dart) produces a self-contained forensic archive:

  1. field_test_events.csv:
    • timestamp,ride_id,event_type,severity,message,speed,latitude,longitude,gps_accuracy,gps_age_ms,lean_angle,longitudinal_g,lateral_g,confidence,metadata
  2. field_test_session.json: Complete structured forensic bundle including session metadata, computed quality summary, serialized event timeline, performance runs, and diagnostic counters.
  3. telemetry.csv: Complete 20Hz raw/filtered sensor sample trace.
  4. ride.gpx: Standard XML GPS track trace with custom extensions for lean angle, speed, and acceleration.

6. Comprehensive Verification Summary

Static Analysis

flutter analyze lib/ test/
Analyzing 2 items...
No issues found! (ran in 3.7s)

Test Suite Execution

flutter test
00:11 +100: All tests passed! (100 total unit, integration, and replay tests)

30-Minute Synthetic Replay Test Results

  • Urban Stop/Start (10 min): Verified stationary wander suppression and 0-35 MPH transitions.
  • Highway Cruise (15 min): Verified stable 0.02\text{g} baseline and 0-60 MPH launch detection.
  • Mountain Cornering (5 min): Verified up to 41.5^\circ lean angle captures and -0.88\text{g} braking event detection.
  • Fault Injections: Verified graceful detection of 10s GPS overpass dropout, 1 teleport jump rejection, 1 3.0s telemetry gap, and 1 IMU saturation spike.
  • Summary Quality Score: Verified \ge 85 / 100 (EXCELLENT rating).

7. Rider Field-Testing Guide (Physical Real-World Test)

To execute the physical validation test on your motorcycle:

  1. Mount Phone: Secure phone in a rigid motorcycle mount (RAM mount, Quad Lock, or stem mount).
  2. Launch Application: Open MotorSignal and verify GPS has locked (Green dot / Accuracy \le 10\text{m}).
  3. Arm Field Test Mode: Go to Settings \to Field Test Mode and ensure the switch is ON (Logger Status displays ARMED).
  4. Start Ride: Tap Start Ride on the main dashboard.
  5. Ride Without Interaction (30–60 Minutes):
    • Perform a mix of stoplight urban riding, open cruising, and corners.
    • You do NOT need to look at or touch the phone during the ride.
  6. Finish Ride: At the end of your trip, tap Finish Ride.
  7. Inspect Forensic Report:
    • Go to Settings \to Field Test Mode \to View Reports.
    • Review your Ride Quality Score, Sensor Health, Performance Runs, and Event Timeline.
    • Tap the Share/Export button in the AppBar to export the complete ZIP package (field_test_events.csv, field_test_session.json, ride.gpx).