You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
219 lines
8.1 KiB
Markdown
219 lines
8.1 KiB
Markdown
# Fast Handoff And Stale Lock Design
|
|
|
|
## Goal
|
|
|
|
Reduce the delay between a valid new YOLO detection and the moment when:
|
|
- `target_id` switches to the new target track,
|
|
- `locked_box_eff` stops following stale KLT/Kalman state,
|
|
- the blue point and yellow guidance line stop pointing at the old scene region.
|
|
|
|
The fix must improve turning behavior without materially increasing false switches to terrain, OSD, or random high-contrast background features.
|
|
|
|
## Current Problem
|
|
|
|
The current pipeline has three separate lag sources that compound during sharp turns:
|
|
|
|
1. `main.py` keeps the old target anchor alive through prediction hold and KLT, even when YOLO has already found a better candidate elsewhere.
|
|
2. The target switch policy requires multiple confirming frames before accepting the new candidate, especially in chase/close regimes.
|
|
3. `guidance.py` smooths both aim point and command state, so even after a switch the yellow line continues to drift toward the new target instead of snapping quickly.
|
|
|
|
Observed failure mode:
|
|
- YOLO draws fresh boxes around the real target in a new image region.
|
|
- `target_id`, `locked_box_eff`, and `pred_box_eff` still describe the stale region.
|
|
- KLT may still look "valid" because it is tracking texture on the wrong patch.
|
|
- Guidance uses the stale lock, so the blue point and yellow line remain pointed away from the true target.
|
|
|
|
## Recommended Approach
|
|
|
|
Implement a three-part fix:
|
|
|
|
1. Add stale-lock detection in `main.py`.
|
|
2. Add a fast-handoff switch path for valid replacement tracks.
|
|
3. Reset guidance smoothing state on stale-break and on confirmed target switch.
|
|
|
|
This is intentionally an incremental fix that preserves the existing architecture:
|
|
- YOLO
|
|
- ByteTrack
|
|
- SafeKalman
|
|
- Hybrid KLT tracker
|
|
- guidance overlay/export
|
|
|
|
## Design
|
|
|
|
### 1. Stale-Lock Detection
|
|
|
|
Add a new state evaluation step in `main.py` after fresh YOLO tracks are available and before final target selection/hold behavior is applied.
|
|
|
|
The stale-lock detector should mark the current lock as stale when all of the following are true:
|
|
|
|
- The system is in `confirmed` state.
|
|
- There is at least one fresh YOLO candidate track.
|
|
- The current `target_id` either has no fresh update or is significantly less plausible than another candidate.
|
|
- The best candidate is far from the current `pred_ref` / `locked_box_eff`.
|
|
- The current anchor is weak by at least one anchor-quality signal.
|
|
|
|
Anchor-quality signals:
|
|
- `klt_valid` is false.
|
|
- `klt_quality` is below a configurable stale threshold.
|
|
- `klt.box` exists but has poor IoU with `locked_box_eff`.
|
|
- `miss_streak` is already elevated.
|
|
- A fresh candidate is motion-consistent while the current target is not.
|
|
|
|
Distance signal:
|
|
- Compute distance from candidate center to `pred_ref` center.
|
|
- Compare against a dedicated stale-break distance threshold based on target diagonal, not just the existing reacquisition threshold.
|
|
|
|
Important behavior:
|
|
- Marking the old lock as stale does not immediately force a blind jump to a new candidate.
|
|
- It only disables continued trust in the stale anchor for guidance and prediction-hold purposes.
|
|
|
|
### 2. Fast Handoff Switch Policy
|
|
|
|
Keep the current conservative switch logic as the default path.
|
|
|
|
Add a second switch path named fast handoff, enabled only when the stale-lock detector fires.
|
|
|
|
Fast handoff should accept a replacement candidate after only `1-2` confirming frames if the candidate passes strict gating:
|
|
|
|
- fresh ByteTrack update (`time_since_update == 0`)
|
|
- score above switch floor
|
|
- residual/motion gate passes when available
|
|
- not inside blocked OSD zones unless near the expected path
|
|
- distance is large enough to prove it is not the stale lock, but not absurdly far
|
|
- optional trajectory agreement if reliable target velocity exists
|
|
|
|
Fast handoff should bypass or reduce:
|
|
- `TARGET_SWITCH_CONFIRM_HITS`
|
|
- `APPROACH_*_SWITCH_EXTRA_HITS`
|
|
- `SWITCH_FAST_MANEUVER_EXTRA_HITS`
|
|
|
|
Fast handoff should also reduce or bypass old-lock waiting:
|
|
- `TARGET_SWITCH_MISS_FRAMES`
|
|
- `APPROACH_*_SWITCH_EXTRA_MISS`
|
|
- `SWITCH_FAST_MANEUVER_EXTRA_MISS`
|
|
|
|
If stale-lock is active and no candidate survives gates:
|
|
- do not keep driving guidance from old stale lock indefinitely,
|
|
- prefer degrading to neutral/low-confidence hold rather than continuing to steer at the wrong patch.
|
|
|
|
### 3. Prediction Hold Restrictions
|
|
|
|
The current `pred_box_eff`/Kalman hold is useful when YOLO drops out briefly, but harmful once a stale-lock condition is detected.
|
|
|
|
Modify hold behavior so that:
|
|
- normal prediction hold remains available during benign short misses,
|
|
- stale-lock condition disables or sharply shortens prediction hold,
|
|
- KLT-only continuation is not allowed to dominate when fresh YOLO is contradicting it.
|
|
|
|
This change should affect the branch that currently sets:
|
|
- `locked_box_eff = pred_box_eff`
|
|
- increments `miss_streak`
|
|
- keeps `confirmed = True`
|
|
|
|
Expected result:
|
|
- the system stops visually "dragging" the old lock across the frame for up to `KALMAN_HOLD_MAX` frames while the real target is already detected somewhere else.
|
|
|
|
### 4. Guidance Reset On Switch
|
|
|
|
Update `guidance.py` so that guidance smoothing state is reset when:
|
|
- a stale-lock break occurs, or
|
|
- `target_id` changes to a new confirmed target.
|
|
|
|
Required behavior:
|
|
- `smoothed_point` should be reinitialized to the new aim point.
|
|
- `smoothed_cmd` should be zeroed or strongly damped.
|
|
- command continuity should favor fast visual alignment over preserving momentum from the old target.
|
|
|
|
This reset must be explicit and event-driven.
|
|
|
|
Without it, even a correct target switch still leaves the yellow line lagging for multiple frames because:
|
|
- aim point is EMA-smoothed,
|
|
- command is EMA-smoothed.
|
|
|
|
### 5. Logging And Diagnostics
|
|
|
|
Extend diagnostics so the fix can be evaluated from logs rather than by eye only.
|
|
|
|
Add fields/events such as:
|
|
- `stale_lock_active`
|
|
- `stale_lock_reason`
|
|
- `fast_handoff_candidate_id`
|
|
- `fast_handoff_active`
|
|
- `guidance_reset_event`
|
|
|
|
Add event tags such as:
|
|
- `stale_lock_break`
|
|
- `fast_handoff_switch`
|
|
- `guidance_reset`
|
|
|
|
This allows measuring:
|
|
- frames between first valid new candidate and switch
|
|
- frames where YOLO sees the new target while lock still follows the stale region
|
|
- false switch counts before/after the fix
|
|
|
|
## Implementation Boundaries
|
|
|
|
Files expected to change:
|
|
- `main.py`
|
|
- `guidance.py`
|
|
- `decision_logger.py`
|
|
- possibly `config.py` for new thresholds
|
|
|
|
Files not in scope for the first pass:
|
|
- `range_estimation.py`
|
|
- `proportional_navigation.py`
|
|
- `autopilot_bridge.py`
|
|
- major ByteTrack internals rewrite
|
|
|
|
## Suggested New Config Controls
|
|
|
|
Add narrowly scoped tuning parameters, for example:
|
|
|
|
- stale-lock enable flag
|
|
- stale-lock distance threshold by target diagonal
|
|
- stale-lock KLT quality floor
|
|
- stale-lock max miss before break
|
|
- fast handoff enable flag
|
|
- fast handoff confirm hits
|
|
- fast handoff max switch distance
|
|
- guidance reset damping factor
|
|
|
|
The first implementation should keep defaults conservative but clearly more responsive than the current chase preset.
|
|
|
|
## Success Criteria
|
|
|
|
The fix is successful if:
|
|
|
|
- During sharp turns, once YOLO consistently detects the target in a new region, the stale lock is abandoned within `1-3` frames.
|
|
- `target_id` switches to the correct candidate substantially earlier than today.
|
|
- The blue point stops following the stale region almost immediately after stale-break.
|
|
- The yellow line reorients quickly after switch instead of easing over many frames.
|
|
- False switches to terrain/OSD do not materially increase.
|
|
|
|
## Verification Plan
|
|
|
|
Use the most recent problem video and compare before/after on the known failure segment around `231.5s - 232.3s`.
|
|
|
|
Measure:
|
|
- first frame where new YOLO candidate is valid
|
|
- frame where stale lock is broken
|
|
- frame where `target_switch` occurs
|
|
- frame where guidance visually realigns
|
|
|
|
Also review aggregate log metrics:
|
|
- target switch count
|
|
- lock lost count
|
|
- time spent in stale-lock-active state
|
|
- rate of false handoffs
|
|
|
|
## Risks
|
|
|
|
- If stale-lock conditions are too loose, the tracker may become jumpy.
|
|
- If fast handoff ignores too many gates, terrain switches may increase.
|
|
- If guidance reset is too aggressive, command output may become visually jerky.
|
|
|
|
The design therefore uses:
|
|
- explicit stale-lock conditions,
|
|
- gated fast handoff,
|
|
- targeted guidance reset only on real switch/break events.
|