715 lines
38 KiB
Markdown
715 lines
38 KiB
Markdown
# MotorSignal — Phase 2 Cloud Sync & Data Ownership Architecture Discovery
|
||
|
||
**Document Version**: 2.0.0-DISCOVERY
|
||
**Date**: August 31, 2026
|
||
**Status**: ARCHITECTURAL SPECIFICATION & DESIGN REVIEW
|
||
**Baseline**: Commit `00819ec` (Tag: `pre-feature-development-baseline`), Flutter 3.44.6, Dart 3.12.2, Drift Schema 5, Riverpod 2.6.1
|
||
|
||
---
|
||
|
||
## 1. Executive Summary
|
||
|
||
MotorSignal is an offline-first motorcycle telemetry acquisition, ride-recording, dynamic analysis, and forensic logging platform. Its core telemetry subsystem operates under an absolute **architectural freeze** (authoritative 20 Hz acquisition cadence, complementary sensor fusion, 3.0 MPH kickstand isolation gate, 3-second crash-durability batch writes, and Drift Schema 5 persistence).
|
||
|
||
Phase 1 established the foundation for user identity abstraction (`AuthUser`, `AuthState`, `AuthService`, `FirebaseAuthService`, Riverpod providers, and dual-theme `AccountScreen`), ensuring that user authentication is 100% optional and operates without any coupling to the high-frequency telemetry pipeline.
|
||
|
||
This Phase 2 Discovery outlines the architectural blueprint for introducing **cloud synchronization, multi-device backup, and user data ownership** while strictly adhering to the fundamental principle:
|
||
|
||
$$\text{Authoritative Pipeline:}\quad \text{Sensors (20 Hz)} \longrightarrow \text{Local SQLite (Schema 5)} \longrightarrow \text{Sync Outbox} \longrightarrow \text{Cloud}$$
|
||
|
||
Cloud availability is never a prerequisite for riding. Unauthenticated, anonymous, and offline rides remain primary first-class citizens. Raw 20 Hz sub-second telemetry traces ($18,\!000+$ samples/hour) are separated from relational summary documents to maintain minimal Firestore costs, avoid document size limits ($1\text{ MB}$), and prevent UI/telemetry thread stuttering.
|
||
|
||
---
|
||
|
||
## 2. Current Cloud/Sync Architecture
|
||
|
||
The repository currently exhibits two distinct synchronization and identity mechanisms:
|
||
|
||
```
|
||
┌────────────────────────────────────────────────────────────────────────┐
|
||
│ PRESENTATION & SETTINGS │
|
||
│ AccountScreen | EvolutionSettingsView | SettingsScreen | Router │
|
||
└───────────────────┬────────────────────────────────────┬───────────────┘
|
||
│ (AuthNotifier) │ (GitHubSyncService)
|
||
┌───────────────────▼─────────────────┐ ┌────────────────▼───────────────┐
|
||
│ PHASE 1 AUTH SUBSYSTEM │ │ EXISTING GITHUB SYNC │
|
||
│ AuthUser | AuthState | AuthService │ │ GitHubAuthService (PAT Storage)│
|
||
│ FirebaseAuthService (Optional Init)│ │ GitHubSyncService (Contents API│
|
||
└─────────────────────────────────────┘ └────────────────▲───────────────┘
|
||
│ triggers upload
|
||
┌────────────────────────────────────────────────────────┴───────────────┐
|
||
│ AUTHORITATIVE LOCAL APPLICATION │
|
||
│ RideStateMachine ────(finishRide)────► AppDatabase (Drift Schema 5) │
|
||
│ ▲ │ │
|
||
│ │ (20 Hz Stream) ▼ │
|
||
│ TelemetryService (Frozen) FieldTestExportService │
|
||
└────────────────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
1. **GitHub Sync (`GitHubSyncService`)**:
|
||
* Uses Personal Access Tokens (PAT) stored in `SharedPreferences`.
|
||
* Directly invoked by `RideStateMachine.finishRide()` upon session termination.
|
||
* Uses `FieldTestExportService` and `ExportService` to generate 5 text files (README, JSON, CSVs, GPX) and commits them sequentially via the GitHub Contents API to `bponders12-cell/ride_logs`.
|
||
* Tracks upload state via `SharedPreferences` flags (`github_uploaded_ride_$rideId`).
|
||
2. **Phase 1 Authentication (`AuthService`)**:
|
||
* Abstraction layer wrapping Firebase Authentication with Google Sign-In and Email/Password.
|
||
* Manages identity lifecycle completely isolated from `RideStateMachine` and `TelemetryService`.
|
||
* Unconfigured/offline mode is automatically handled without app crashes or network blocking.
|
||
|
||
---
|
||
|
||
## 3. Existing GitHub Sync Analysis
|
||
|
||
### Current Implementation Characteristics
|
||
* **Coupling**: `RideStateMachine` directly reads `githubSyncServiceProvider` in `finishRide()`. This creates direct coupling between state machine lifecycle and an external network service.
|
||
* **Network & Retry Mechanism**: Lacks exponential backoff, persistent outbox queueing, or network reachability listeners; if network fails during sequential PUT calls, the ride enters a `failed` state and requires manual user retry or app restart.
|
||
* **Payload Footprint**: Uploads individual uncompressed text files directly to a git repository, generating 5 separate HTTP PUT requests per ride session.
|
||
|
||
### Recommendation
|
||
* **Transition Strategy**: Refactor GitHub sync from a standalone hardcoded service into a concrete implementation of a unified `SyncProvider` (`GitHubSyncProvider`), managed by a central, provider-agnostic `SyncManager`.
|
||
* **Coexistence**: Firebase Sync (MotorCloud) and GitHub Sync should coexist as independent sync destinations. Riders can enable either, both, or neither.
|
||
* **Deprecation Horizon**: Retain GitHub sync for forensic engineering field-testing; position Firebase/Cloud storage as the primary consumer backup and multi-device synchronization engine.
|
||
|
||
---
|
||
|
||
## 4. Phase 1 Authentication Integration Analysis
|
||
|
||
Phase 1 introduced clean, decoupled identity types:
|
||
* `AuthUser`: Contains `uid`, `email`, `displayName`, `photoUrl`, `isAnonymous`, `providerId`.
|
||
* `AuthState`: Represents `status` (`initial`, `loading`, `authenticated`, `unauthenticated`, `anonymous`, `error`), holding `isFirebaseConfigured`.
|
||
* `authNotifierProvider`: Riverpod provider exposing reactive auth updates.
|
||
|
||
### Decoupled Identity Association
|
||
* When a ride is completed, the local database record remains pure (Schema 5).
|
||
* When the `SyncManager` prepares an outbox payload for cloud upload, it inspects the current `AuthUser` at sync time.
|
||
* If the user is authenticated (or anonymous), the cloud document metadata is populated with `userId: authUser.uid`.
|
||
* If the user is unauthenticated / local-only, cloud sync remains disabled for that ride until an account is linked or manual backup is requested.
|
||
|
||
---
|
||
|
||
## 5. Local Data Ownership Model
|
||
|
||
MotorSignal follows a strict **Local-First Authoritative Model**:
|
||
|
||
```
|
||
[Rider Device]
|
||
Local SQLite (Drift Schema 5) <── Master Source of Truth for Raw Data
|
||
│
|
||
├── Rides (Summary & Aggregates)
|
||
├── TelemetrySamples (20 Hz Raw Sub-Second Points)
|
||
├── RideEvents (Discrete Driving Dynamic Events)
|
||
├── PerformanceRuns (0-60 / 0-100 Benchmarks)
|
||
└── GarageMotorcycles (Vehicle Profiles)
|
||
|
||
[Cloud Storage (Firebase / Remote)]
|
||
Cloud Records <── Read-Replica, Backup, & Multi-Device Relay
|
||
│
|
||
├── Firestore: /users/{uid}/rides/{rideId} (Aggregates & Events)
|
||
└── Cloud Storage: /users/{uid}/rides/{rideId}/telemetry.zip (Compressed Raw Stream)
|
||
```
|
||
|
||
### Core Ownership Rules
|
||
1. **Local Primacy**: Deleting a remote cloud backup must never delete the local SQLite record on the device that recorded it unless the user explicitly executes a local wipe.
|
||
2. **Immutability of Historical Telemetry**: Once a ride transitions to `RideState.finished`, its raw sensor trace is cryptographically immutable. Cloud sync uploads a frozen snapshot.
|
||
3. **Multi-Device Replication**: When a second device (Phone B) logs into the same account, cloud sync pulls ride summaries and telemetry bundles into Phone B's local database, marking them as `isSynced: true`.
|
||
|
||
---
|
||
|
||
## 6. Proposed Cloud Architecture
|
||
|
||
A hybrid storage model separating **structured queryable metadata** from **high-volume raw telemetry streams** is architecturally superior:
|
||
|
||
```
|
||
┌────────────────────────────────────────────────────────────────────────┐
|
||
│ CLOUD ENGINE │
|
||
│ │
|
||
│ ┌────────────────────────────────┐ ┌───────────────────────────────┐ │
|
||
│ │ CLOUDFIRESTORE (METADATA) │ │ CLOUD STORAGE (RAW BUNDLES) │ │
|
||
│ │ │ │ │ │
|
||
│ │ • Ride Summaries & Stats │ │ • telemetry.csv.gz / .zip │ │
|
||
│ │ • Discrete Events & Alerts │ │ • forensic_session.json.gz │ │
|
||
│ │ • Performance Benchmarks │ │ • Full GPX Trace │ │
|
||
│ │ • Garage Motorcycle Profiles │ │ • High-Res IMU Stream │ │
|
||
│ │ • Sync Metadata & ETag │ │ • Future Video / Thumbnails │ │
|
||
│ └────────────────────────────────┘ └───────────────────────────────┘ │
|
||
└───────────────────────────────────▲────────────────────────────────────┘
|
||
│ Uploads via SyncProvider
|
||
┌───────────────────────────────────┴────────────────────────────────────┐
|
||
│ MOTORCLOUD SYNC MANAGER │
|
||
│ Background Isolate Pipeline: Compress 20Hz ──► Generate JSON/CSV │
|
||
│ Exponential Backoff Queue ──► Network State Gate (Wi-Fi / Cellular) │
|
||
└───────────────────────────────────▲────────────────────────────────────┘
|
||
│ Consumes Outbox Items
|
||
┌───────────────────────────────────┴────────────────────────────────────┐
|
||
│ LOCAL PERSISTENCE & TELEMETRY │
|
||
│ AppDatabase (Schema 5) ◄── Authoritative 20 Hz Clock │
|
||
└────────────────────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
---
|
||
|
||
## 7. Cloud Provider Comparison
|
||
|
||
| Evaluation Metric | Option A: Firestore Only | Option B: Firestore + Cloud Storage (Recommended) | Option C: Custom REST Backend | Option D: Supabase (Postgres) |
|
||
| :--- | :--- | :--- | :--- | :--- |
|
||
| **Telemetry Volume ($18\text{k}$ samples/hr)** | ❌ Exceeds doc limit ($1\text{ MB}$); extreme write cost | ✅ Compressed bundle ($<500\text{ KB}$) in Storage; zero write cost per sample | ✅ Efficient via custom binary / gRPC endpoints | ⚠️ Large row volume in Postgres; high egress |
|
||
| **Queryability (Stats, History, Filter)** | ✅ Rich querying by date, bike, speed | ✅ Rich querying for metadata in Firestore | ✅ Full SQL / Postgres queries | ✅ Direct relational queries via SQL |
|
||
| **Cost Efficiency** | ❌ Highly expensive (18,000 writes/ride) | ✅ Extremely cheap (1 doc write + 1 storage upload per ride) | ⚠️ Requires dedicated VPS/Server hosting | ✅ Affordable at baseline tier |
|
||
| **Offline Resilience & SDK** | ✅ Mature SDK with offline cache | ✅ Mature SDK with background resumable upload | ⚠️ Must build custom offline sync engine | ✅ Good offline client ecosystem |
|
||
| **Maintenance Burden** | Low | Low (Serverless) | High (Database, Auth, API, OS maintenance) | Medium (Managed Postgres) |
|
||
|
||
**Verdict**: **Option B (Firestore + Firebase Cloud Storage)** provides the optimal balance of zero infrastructure maintenance, low cloud cost, high query performance for UI history, and full storage capacity for forensic telemetry streams.
|
||
|
||
---
|
||
|
||
## 8. Recommended Provider Architecture
|
||
|
||
To support future backend evolution (or multi-destination sync), cloud sync must be defined behind an abstract provider contract:
|
||
|
||
```dart
|
||
abstract class SyncProvider {
|
||
String get providerId; // 'firebase', 'github', 'supabase'
|
||
Future<bool> isAvailable();
|
||
Future<SyncUploadResult> uploadRide(SyncPayload payload);
|
||
Future<List<RemoteRideHeader>> fetchRemoteHeaders({DateTime? since});
|
||
Future<SyncDownloadResult> downloadRideBundle(String remoteId);
|
||
Future<void> deleteRemoteRide(String remoteId);
|
||
}
|
||
```
|
||
|
||
The application interacts solely with `SyncManager`, which coordinates the local outbox, monitors network connectivity, dispatches tasks to registered `SyncProvider`s, and reports status to Riverpod.
|
||
|
||
---
|
||
|
||
## 9. Proposed Cloud Data Model
|
||
|
||
### Firestore Document Schema
|
||
|
||
#### User Profile Document: `/users/{uid}`
|
||
```json
|
||
{
|
||
"uid": "aB3kL99zX...",
|
||
"email": "rider@motorsignal.app",
|
||
"displayName": "Apex Pilot",
|
||
"photoUrl": "https://...",
|
||
"isAnonymous": false,
|
||
"createdAt": "2026-08-31T01:00:00.000Z",
|
||
"lastActiveAt": "2026-08-31T12:00:00.000Z",
|
||
"schemaVersion": 1,
|
||
"stats": {
|
||
"totalRides": 42,
|
||
"totalDistanceMiles": 1284.5,
|
||
"totalMovingTimeSeconds": 89400,
|
||
"maxSpeedMph": 118.4,
|
||
"maxLeanLeft": 46.2,
|
||
"maxLeanRight": 48.7
|
||
}
|
||
}
|
||
```
|
||
|
||
#### Ride Metadata Document: `/users/{uid}/rides/{remoteRideId}`
|
||
```json
|
||
{
|
||
"remoteRideId": "ride_20260831_080000_aB3k",
|
||
"localRideId": 14,
|
||
"deviceUuid": "e7b1a2c3-d4e5-4f6a-8b9c-0d1e2f3a4b5c",
|
||
"startTime": "2026-08-31T08:00:00.000Z",
|
||
"endTime": "2026-08-31T08:45:12.000Z",
|
||
"distanceMiles": 24.8,
|
||
"topSpeedMph": 74.2,
|
||
"averageSpeedMph": 41.5,
|
||
"maxLeanLeft": 32.1,
|
||
"maxLeanRight": 35.4,
|
||
"maxAccelG": 0.58,
|
||
"maxBrakingG": -0.88,
|
||
"movingTimeSeconds": 2160,
|
||
"stoppedTimeSeconds": 552,
|
||
"elevationGainMeters": 312.0,
|
||
"elevationLossMeters": 298.0,
|
||
"qualityScore": 98,
|
||
"qualityRating": "EXCELLENT",
|
||
"motorcycle": {
|
||
"year": 2001,
|
||
"make": "Yamaha",
|
||
"model": "YZF-R6",
|
||
"variant": "Factory Stock"
|
||
},
|
||
"storageBundle": {
|
||
"storagePath": "users/aB3kL99zX.../rides/ride_20260831_080000_aB3k/telemetry.zip",
|
||
"bundleSizeCompressedBytes": 348120,
|
||
"sampleCount": 27120,
|
||
"contentSha256": "8f4b2c1..."
|
||
},
|
||
"software": {
|
||
"version": "1.0.2",
|
||
"build": 102,
|
||
"gitCommit": "00819ec",
|
||
"telemetrySchemaVersion": 1,
|
||
"exportSchemaVersion": 1
|
||
},
|
||
"syncMetadata": {
|
||
"uploadedAt": "2026-08-31T08:46:00.000Z",
|
||
"clientTimestamp": "2026-08-31T08:45:50.000Z",
|
||
"isDeleted": false
|
||
}
|
||
}
|
||
```
|
||
|
||
#### Discrete Events Subcollection: `/users/{uid}/rides/{remoteRideId}/events/{eventId}`
|
||
```json
|
||
{
|
||
"eventId": "evt_01",
|
||
"timestamp": "2026-08-31T08:14:22.050Z",
|
||
"eventType": "Hard Braking",
|
||
"value": -0.88,
|
||
"latitude": 37.7749,
|
||
"longitude": -122.4194
|
||
}
|
||
```
|
||
|
||
#### Performance Benchmarks Subcollection: `/users/{uid}/rides/{remoteRideId}/performance/{runId}`
|
||
```json
|
||
{
|
||
"runId": "perf_01",
|
||
"type": "0-60",
|
||
"timeMs": 3420.0,
|
||
"topSpeedReachedMph": 60.2,
|
||
"timestamp": "2026-08-31T08:20:10.000Z"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 10. Local Sync Outbox Design
|
||
|
||
### Preservation of Schema 5
|
||
To preserve the frozen Drift Schema 5 database without requiring high-risk migrations, the local synchronization queue is implemented via an **Outbox File Manifest** (`sync_outbox_manifest.json`) backed by `SharedPreferences` index pointers.
|
||
|
||
### Outbox Model Structure
|
||
```dart
|
||
class SyncItem {
|
||
final int localRideId;
|
||
final String idempotencyKey;
|
||
final String providerId;
|
||
final SyncStatus status;
|
||
final int attemptCount;
|
||
final DateTime createdAt;
|
||
final DateTime? lastAttemptAt;
|
||
final DateTime? nextRetryAt;
|
||
final DateTime? uploadedAt;
|
||
final String? remoteId;
|
||
final String? lastError;
|
||
final String? bundleLocalPath;
|
||
|
||
const SyncItem({
|
||
required this.localRideId,
|
||
required this.idempotencyKey,
|
||
required this.providerId,
|
||
required this.status,
|
||
this.attemptCount = 0,
|
||
required this.createdAt,
|
||
this.lastAttemptAt,
|
||
this.nextRetryAt,
|
||
this.uploadedAt,
|
||
this.remoteId,
|
||
this.lastError,
|
||
this.bundleLocalPath,
|
||
});
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## 11. Sync State Machine
|
||
|
||
```
|
||
┌───────────────────────┐
|
||
│ LOCAL_ONLY │
|
||
└──────────┬────────────┘
|
||
│ Ride finished / Cloud sync enabled
|
||
▼
|
||
┌───────────────────────┐
|
||
│ QUEUED │
|
||
└──────────┬────────────┘
|
||
│
|
||
┌────────────┴────────────┐
|
||
No Network / Metered Network Connected & Authed
|
||
▼ ▼
|
||
┌─────────────────────────┐ ┌─────────────────────────┐
|
||
│ WAITING_FOR_NETWORK │ │ PREPARING │ (Isolate Bundle Compression)
|
||
└────────────┬────────────┘ └────────────┬────────────┘
|
||
│ Network restored ▼
|
||
└────────────────────────►┌─────────────────────────┐
|
||
│ UPLOADING │
|
||
└───────────┬─────────────┘
|
||
│
|
||
┌─────────────┴─────────────┐
|
||
Upload Success Upload Failure
|
||
▼ ▼
|
||
┌─────────────────────────┐ ┌─────────────────────────┐
|
||
│ SYNCED │ │ RETRY_WAIT │
|
||
└─────────────────────────┘ └────────────┬────────────┘
|
||
│ Exponential backoff
|
||
▼
|
||
┌─────────────────────────┐
|
||
│ FAILED (Max Retries) │
|
||
└─────────────────────────┘
|
||
```
|
||
|
||
---
|
||
|
||
## 12. Retry & Failure Strategy
|
||
|
||
1. **Exponential Backoff Formula**:
|
||
$$T_{\text{delay}} = \min\left(T_{\text{max}},\; T_{\text{base}} \times 2^{\text{attempt}} + \text{jitter}\right)$$
|
||
* $T_{\text{base}} = 5\text{ seconds}$
|
||
* $T_{\text{max}} = 30\text{ minutes}$
|
||
* Maximum automated retry limit: **5 attempts**, after which status becomes `FAILED_MANUAL_REQUIRED`.
|
||
2. **Network Constraint Filtering**:
|
||
* Settings configurable: `Sync over Cellular` vs `Wi-Fi Only`.
|
||
* Unmetered network gate: Automatically suspends large bundle uploads if device switches to roaming or restricted metered networks.
|
||
3. **Decoupled Execution**: All retry timers and network listeners execute asynchronously in background services. The 20 Hz telemetry clock is completely isolated and unaffected by sync failures or retries.
|
||
|
||
---
|
||
|
||
## 13. Duplicate & Idempotency Strategy
|
||
|
||
To prevent duplicate cloud entries when uploading across flaky connections:
|
||
1. **Deterministic Remote ID**:
|
||
$$\text{remoteRideId} = \text{SHA256}(\text{deviceInstallationUuid} + \text{"\_"} + \text{startTimeIso8601})$$
|
||
2. **Idempotent Upsert**:
|
||
* Firestore writes use `set(..., SetOptions(merge: true))` targeted at `/users/{uid}/rides/{remoteRideId}`.
|
||
* Storage uploads overwrite the canonical destination `users/{uid}/rides/{remoteRideId}/telemetry.zip`.
|
||
3. **Payload Verification**:
|
||
* Cloud Storage upload includes `contentSha256` in custom metadata. If an upload was interrupted, the client checks remote file hash before re-uploading bytes.
|
||
|
||
---
|
||
|
||
## 14. Anonymous Account Linking
|
||
|
||
```
|
||
[Local Anonymous Rider] ──► Records Rides ──► AuthState.anonymous (Local ID: anon_123)
|
||
│
|
||
│ Rider signs in with Google / Email
|
||
▼
|
||
[Firebase Auth: linkWithCredential]
|
||
│
|
||
┌─────────────────────────────┴─────────────────────────────┐
|
||
▼ ▼
|
||
[Linking Successful] [Linking Conflict / New Account]
|
||
• UID remains constant • New UID generated
|
||
• Remote documents retain same path • Outbox queue updates local metadata
|
||
• Zero data re-upload required • Local rides queued for backup to new UID
|
||
```
|
||
|
||
### Safety Rules:
|
||
* If account linking fails (e.g., Google account already exists on another UID), prompt the rider: *"Link this device's rides to your existing Google Account?"*
|
||
* If approved, the Outbox re-tags local un-synced rides with the existing account UID and schedules background sync.
|
||
* Signing out immediately clears in-memory user tokens and halts active sync streams, leaving local SQLite databases completely untouched.
|
||
|
||
---
|
||
|
||
## 15. Multi-Device Synchronization
|
||
|
||
### Ingestion Protocol for Secondary Devices (Phone B)
|
||
1. **Header Sync**: Phone B queries `/users/{uid}/rides` where `syncMetadata.uploadedAt > lastSyncTime`.
|
||
2. **Local Placeholder Insertion**: Phone B inserts summary headers into its local Drift `Rides` table with a flag indicating remote origin.
|
||
3. **Selective / On-Demand Telemetry Download**:
|
||
* Summary, route map coordinates, and stats display immediately in History.
|
||
* Raw 20 Hz telemetry bundles (`telemetry.zip`) are downloaded on-demand only when the rider opens Forensic Analysis, Replay, or Export for that session.
|
||
4. **Deletes & Tombstones**:
|
||
* Deleting a ride on Phone A sets `syncMetadata.isDeleted = true` (soft delete) in Firestore.
|
||
* Phone B syncs the tombstone and marks its local copy as deleted or removes it per user preference.
|
||
|
||
---
|
||
|
||
## 16. Security Model
|
||
|
||
### Firestore Security Rules (`firestore.rules`)
|
||
```javascript
|
||
rules_version = '2';
|
||
service cloud.firestore {
|
||
match /databases/{database}/documents {
|
||
match /users/{userId} {
|
||
allow read, write: if request.auth != null && request.auth.uid == userId;
|
||
|
||
match /rides/{rideId} {
|
||
allow read, write: if request.auth != null && request.auth.uid == userId;
|
||
|
||
match /{allSubcollections=**} {
|
||
allow read, write: if request.auth != null && request.auth.uid == userId;
|
||
}
|
||
}
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
### Cloud Storage Security Rules (`storage.rules`)
|
||
```javascript
|
||
rules_version = '2';
|
||
service firebase.storage {
|
||
match /b/{bucket}/o {
|
||
match /users/{userId}/rides/{rideId}/{allPaths=**} {
|
||
allow read, write: if request.auth != null && request.auth.uid == userId;
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
### Security Highlights
|
||
* **Zero Cross-User Data Exposure**: All user data is keyed directly under `/users/{request.auth.uid}/`.
|
||
* **Zero Hardcoded Secrets**: Client configuration relies purely on `google-services.json` / `GoogleService-Info.plist` without committing API secrets or GitHub PATs to source control.
|
||
|
||
---
|
||
|
||
## 17. Privacy Model
|
||
|
||
Motorcycle telemetry contains sensitive personal data (precise GPS home locations, maximum speed, dynamic acceleration). The privacy model enforces user-selectable tiers:
|
||
|
||
```
|
||
[Privacy Tier 1: Local Only (Default)]
|
||
• No data leaves device; 100% offline persistence.
|
||
|
||
[Privacy Tier 2: Private Cloud Backup]
|
||
• Full forensic sync to user's encrypted, authenticated private storage.
|
||
• Not visible to public search, community leaderboards, or third parties.
|
||
|
||
[Privacy Tier 3: Community & Fleet Sharing (Future)]
|
||
• Anonymized aggregate stats (road condition hazard alerts, cornering heatmaps).
|
||
• Strips first/last 0.5 miles of GPS coordinates to protect origin/destination privacy.
|
||
```
|
||
|
||
---
|
||
|
||
## 18. Camera & Media Future Compatibility
|
||
|
||
When camera hardware or action cameras (GoPro / Insta360 / BLE Dashcams) are integrated in later phases:
|
||
|
||
```
|
||
Ride Session
|
||
├── Firestore: /users/{uid}/rides/{rideId} (Metadata & Video Manifest)
|
||
├── Cloud Storage: .../telemetry.zip (Raw 20 Hz Telemetry)
|
||
└── Cloud Storage: .../media/
|
||
├── video_segment_01.mp4 (Resumable chunked upload)
|
||
├── thumbnail.jpg (Instant preview)
|
||
└── bookmarks.json (Dynamic telemetry trigger highlights)
|
||
```
|
||
|
||
* **Storage Isolation**: Video files ($100\text{ MB} - 4\text{ GB}$) bypass Firestore entirely and utilize Firebase Storage Resumable Upload URLs.
|
||
* **Telemetry Synchronicity**: Video timestamps are indexed against the monotonic 20 Hz telemetry timestamps for frame-accurate playback.
|
||
|
||
---
|
||
|
||
## 19. GitHub Provider Compatibility
|
||
|
||
The proposed `SyncManager` architecture natively incorporates GitHub sync alongside Firebase:
|
||
|
||
```dart
|
||
class GitHubSyncProvider implements SyncProvider {
|
||
final GitHubSyncService _service;
|
||
@override
|
||
String get providerId => 'github';
|
||
...
|
||
}
|
||
|
||
class FirebaseSyncProvider implements SyncProvider {
|
||
final FirebaseFirestore _firestore;
|
||
final FirebaseStorage _storage;
|
||
@override
|
||
String get providerId => 'firebase';
|
||
...
|
||
}
|
||
```
|
||
|
||
* Both providers consume the same standardized `SyncPayload` generated by `FieldTestExportService`.
|
||
* A rider can configure GitHub PAT for engineering test uploads and Firebase for consumer account sync simultaneously.
|
||
|
||
---
|
||
|
||
## 20. Performance & Isolate Strategy
|
||
|
||
To maintain a rock-solid 20 Hz telemetry loop and a 60/120 FPS UI:
|
||
|
||
1. **Background Isolate Compression (`compute()`)**:
|
||
* Reading 18,000 samples from SQLite, encoding JSON/CSV, and executing GZIP/ZIP compression must run in a dedicated background worker isolate:
|
||
```dart
|
||
final zipBytes = await compute(_packageRideBundleIsolate, exportParams);
|
||
```
|
||
2. **Streamed Storage Uploads**:
|
||
* Compressed files are written to temporary application storage and streamed directly to Cloud Storage via resumable background tasks, avoiding RAM spikes.
|
||
|
||
---
|
||
|
||
## 21. Failure Scenario Matrix
|
||
|
||
| # | Scenario | Failure Mode | Architectural Recovery Strategy |
|
||
| :--- | :--- | :--- | :--- |
|
||
| **1** | Phone has no internet | Network connection dropped | Ride saves to local Drift SQLite; marked `WAITING_FOR_NETWORK` in outbox. |
|
||
| **2** | Firebase is unavailable | HTTP 503 / Backend timeout | Outbox backs off exponentially ($5\text{s} \rightarrow 30\text{m}$). Telemetry continues. |
|
||
| **3** | User is logged out | Missing auth token | Sync paused with `AUTH_REQUIRED`; local ride remains available in UI. |
|
||
| **4** | User is anonymous | Ephemeral local UID | Sync operates using Firebase Anonymous Auth; ready for account linking. |
|
||
| **5** | Auth token expires | HTTP 401 | `AuthService` auto-refreshes token; queue retries automatically. |
|
||
| **6** | Ride finishes during loss | No network on finish | `RideStateMachine` finishes locally in 0ms; queues outbox asynchronously. |
|
||
| **7** | App killed during upload | Process termination | Outbox marks upload incomplete; resumes chunked upload on next app boot. |
|
||
| **8** | Battery dies during upload | Sudden device shutdown | Incomplete remote files overwritten idempotently on next startup. |
|
||
| **9** | Upload partially completes | Firestore OK, Storage failed | Outbox tracks sub-step completion; retries only the storage payload. |
|
||
| **10** | Same ride uploaded twice | Retried network request | Deterministic `remoteRideId` ensures idempotent Firestore merge (no duplicates). |
|
||
| **11** | User signs out mid-upload | Session invalidated | Active upload stream cancelled immediately; local file preserved. |
|
||
| **12** | User switches accounts | New account credentials | Outbox isolates queue per `uid`; clears foreign upload cache. |
|
||
| **13** | Cloud record deleted | Remote delete via console | Local database retains authoritative copy; re-syncs if requested. |
|
||
| **14** | Local ride deleted | Rider deletes from History | Tombstone event sent to cloud if sync enabled; otherwise purely local wipe. |
|
||
| **15** | Corrupt export bundle | CRC32 mismatch | Isolate regenerates bundle from authoritative SQLite records. |
|
||
| **16** | Cloud quota exceeded | HTTP 402/429 | Outbox sets `QUOTA_EXCEEDED`; notifies user via Settings/Account badge. |
|
||
| **17** | Firestore write fails | Security rule or index error | Logs error code to diagnostics; schedules retry without crashing app. |
|
||
| **18** | Storage upload fails | Upload timeout / network drop | Resumable upload session recovers at exact byte offset when online. |
|
||
| **19** | GitHub OK, Firebase failed | Partial provider success | `SyncManager` tracks provider statuses independently per ride. |
|
||
| **20** | Firebase OK, GitHub failed | Partial provider success | Firebase marked `SYNCED`; GitHub remains queued for retry. |
|
||
|
||
---
|
||
|
||
## 22. Historical Ride Migration Strategy
|
||
|
||
* **Zero Forced Migrations**: Existing local rides created prior to Phase 2 are tagged as `LOCAL_ONLY`.
|
||
* **User-Initiated Cloud Backup**:
|
||
* In `Settings > Account`, riders are presented with:
|
||
> *"Found 12 offline rides. Back up to MotorCloud?"*
|
||
* User can tap `Back Up All Historical Rides` or select individual rides from History.
|
||
* Uploads are throttled in background isolates to avoid bandwidth saturation.
|
||
|
||
---
|
||
|
||
## 23. UI / UX Recommendations
|
||
|
||
```
|
||
Settings
|
||
└── Account & Cloud Sync
|
||
├── Account Profile Header (Email, Display Name, UID, Provider)
|
||
├── Sync Status Badge (● Synced 2m ago | ○ 3 rides pending | ⚠ Offline)
|
||
├── [Sync Now] Manual Trigger Button
|
||
├── Network Preferences:
|
||
│ ├── [Toggle] Sync over Cellular Data (Default: OFF)
|
||
│ └── [Toggle] Sync on Wi-Fi Only (Default: ON)
|
||
├── Cloud Storage Management:
|
||
│ ├── Usage Meter: 14.2 MB / 10 GB
|
||
│ └── [Button] Back Up Historical Offline Rides
|
||
└── Privacy & Data:
|
||
└── [Selector] Private Backup vs Local Only
|
||
```
|
||
|
||
* **Cockpit Safety Rule**: During an active ride (`RideState.riding`), all sync notifications and prompts are completely suppressed.
|
||
|
||
---
|
||
|
||
## 24. Observability & Diagnostics
|
||
|
||
The existing Diagnostics screen (`/settings/diagnostics`) will expose a new diagnostic section:
|
||
|
||
```
|
||
[CLOUD SYNCHRONIZATION DIAGNOSTICS]
|
||
• Engine State: IDLE (Connected to MotorCloud)
|
||
• Active Account: rider@motorsignal.app (UID: aB3kL99zX...)
|
||
• Outbox Queue Length: 0 pending
|
||
• Total Cloud Synced Rides: 34
|
||
• Last Successful Sync: 2026-08-31 01:25:12
|
||
• Last Sync Latency: 1.42s (Isolate: 240ms, Doc: 380ms, Storage: 800ms)
|
||
• Network Policy: Wi-Fi Only (Current: Wi-Fi connected)
|
||
• GitHub Sync Provider: ACTIVE (bponders12-cell/ride_logs)
|
||
```
|
||
|
||
---
|
||
|
||
## 25. Database Schema Impact
|
||
|
||
### Drift Schema 5 Remains 100% Frozen
|
||
* **No Schema Changes in Phase 2**: All outbox queues, retry timestamps, and remote document IDs are managed outside Drift tables (via `sync_outbox_manifest.json` and `SharedPreferences`).
|
||
* **Future Schema 6 (Deferred)**: If future bidirectional multi-device sync warrants a dedicated relational table, it will be introduced with an explicit, fully tested Drift migration without altering raw telemetry schemas.
|
||
|
||
---
|
||
|
||
## 26. Architectural Risks & Mitigations
|
||
|
||
1. **Risk: Main Thread Isolate Lag during ZIP Serialization**
|
||
* *Mitigation*: Strictly execute `FieldTestExportService` compression inside `compute()` worker isolates.
|
||
2. **Risk: High Cloud Storage Costs from Raw Telemetry**
|
||
* *Mitigation*: Store raw 20 Hz points exclusively in compressed GZIP/ZIP archives in Cloud Storage; store only summary metrics and events in Firestore.
|
||
3. **Risk: Unintentional Coupling to `RideStateMachine`**
|
||
* *Mitigation*: Decouple `RideStateMachine` from sync by listening to state machine events externally via Riverpod listeners rather than adding sync calls inside `finishRide()`.
|
||
|
||
---
|
||
|
||
## 27. Recommended Phase 2 Implementation Plan
|
||
|
||
```
|
||
Phase 2.1: Sync Foundation & Outbox Engine
|
||
├── Implement SyncItem outbox model & manifest persistence
|
||
├── Implement background worker isolate for ZIP packaging
|
||
└── Build SyncManager with exponential backoff & network monitoring
|
||
|
||
Phase 2.2: Firebase Cloud Provider Implementation
|
||
├── Implement FirebaseSyncProvider (Firestore doc + Storage bundle)
|
||
├── Configure Firestore security rules & Cloud Storage security rules
|
||
└── Refactor GitHubSyncService into GitHubSyncProvider
|
||
|
||
Phase 2.3: Riverpod State & UI Integration
|
||
├── Expose syncStateProvider, syncOutboxProvider, pendingRidesProvider
|
||
├── Add Sync Management card to AccountScreen and Settings
|
||
└── Integrate sync diagnostics into DiagnosticsScreen
|
||
|
||
Phase 2.4: Multi-Device Sync & Testing
|
||
├── Implement remote ride header fetch & on-demand bundle download
|
||
├── Write 15+ comprehensive unit & mock sync failure tests
|
||
└── Verify 100% isolation of 20 Hz TelemetryService and Schema 5
|
||
```
|
||
|
||
---
|
||
|
||
## 28. Explicit "DO NOT MODIFY" Boundaries
|
||
|
||
The following files and subsystems remain **STRICTLY FROZEN**:
|
||
|
||
1. `lib/services/telemetry_service.dart` (20 Hz acquisition cadence, 50 ms monotonic clock)
|
||
2. `lib/services/sensor_fusion_service.dart` (Complementary filtering, vehicle coordinates)
|
||
3. `lib/services/location_service.dart` (GPS filtering & accuracy gating)
|
||
4. `lib/services/compass_service.dart` (Heading acquisition)
|
||
5. `lib/services/validation/telemetry_validator.dart` (Data quality rules)
|
||
6. `lib/features/rides/ride_state_machine.dart` (3.0 MPH gate, 3s DB batching, crash recovery)
|
||
7. `lib/database/database.dart` & `tables.dart` (Drift Schema 5)
|
||
8. `lib/services/field_test/field_test_export_service.dart` (Forensic export schemas)
|
||
|
||
---
|
||
|
||
## 29. Verification Checklist
|
||
|
||
- [x] All 29 discovery sections comprehensively analyzed and structured.
|
||
- [x] Authoritative 20 Hz telemetry pipeline verified as 100% independent from cloud services.
|
||
- [x] Drift Schema 5 confirmed intact with zero schema migrations required.
|
||
- [x] Existing GitHub sync architecture analyzed and positioned for clean provider abstraction.
|
||
- [x] Phase 1 authentication integration mapped cleanly without telemetry coupling.
|
||
- [x] All 20 failure scenarios analyzed with concrete recovery behaviors.
|
||
- [x] Repository state verified via `git status --short`, `git diff --stat`, and `git diff --name-only`.
|
||
|
||
---
|
||
|
||
## RECOMMENDED NEXT IMPLEMENTATION
|
||
|
||
When Phase 2 execution begins, work should be sequenced into three strict tiers:
|
||
|
||
### MUST HAVE (Phase 2 Core)
|
||
1. **`SyncManager` & Outbox Engine**: Non-blocking outbox queue persisting sync status in `sync_outbox_manifest.json` outside Drift Schema 5.
|
||
2. **`FirebaseSyncProvider`**: Dual-destination upload sending structured summary/events to Firestore and compressed raw telemetry bundles to Cloud Storage.
|
||
3. **Background Worker Isolate**: Offloading bundle compression and JSON/CSV serialization to `compute()`.
|
||
4. **Decoupled Riverpod Observer**: Moving sync triggers out of `RideStateMachine` into an external `RideCompletionSyncObserver`.
|
||
5. **Account Screen Sync Controls**: Manual "Sync Now", Wi-Fi only toggle, and sync status indicator in `AccountScreen`.
|
||
|
||
### SHOULD HAVE (Phase 2 Refinement)
|
||
1. **`GitHubSyncProvider` Abstraction**: Unifying the existing GitHub sync behind the `SyncProvider` interface.
|
||
2. **Historical Rides Backup Action**: UI button allowing users to back up existing `LOCAL_ONLY` rides.
|
||
3. **Sync Diagnostics Panel**: Adding sync metrics to the Diagnostics screen.
|
||
|
||
### DEFER (Phase 3+)
|
||
1. **Two-Way Multi-Device Editing**: Conflict resolution for edited ride metadata across multiple active devices.
|
||
2. **Camera / Video Media Sync**: Resumable multi-gigabyte video chunk synchronization.
|
||
3. **Public Community Leaderboards & Social Feeds**: Public discovery and shared ride feeds.
|
||
4. **Drift Schema 6 Migration**: Introducing dedicated relational tables for sync outbox records.
|