Tighten fallback latency + alert UI on degraded states
Submit-first ordering. Patch 0004 now calls generator_submitblock
BEFORE writing the pending-block hex to disk. The happy path adds zero
disk I/O — we only dump when the primary returned false. The same
patch bounds generator_submitblock's "no live current_si" spin to
~3s instead of the original infinite loop, so a permanently-down
primary doesn't pin the stratifier; the bounded spin lets the caller
return false and lets local_block_submit dump for kamado-api to take
over.
Default grace lowered from 30s to 3s. With ckpool's bounded spin and
sub-second sweep cadence, the fallback now reacts within ~4s of a
failed primary submit — fast enough that the work is still relevant
for the current chain tip. The submitter's sweep poll dropped to 1s
to match.
UI HealthBanners. New top-of-page strip surfaces:
* Fallback used (red banner, 24h after most recent event):
"primary bitcoind didn't accept; backup X took over Y ago"
* Submit gap (orange banner, only when no recent fallback):
"N blocks attempted but unconfirmed — configure backups"
* ZMQ stale (orange banner): no hashblock frame in 30+ minutes
Operators see degraded-but-not-fatal states without checking logs.
Startup readiness gate. main now waits up to 8s on agg.Ready() before
starting the HTTP server so the very first /api/snapshot doesn't show
all-zero state during the aggregator's first refresh. Capped so a
permanently-down bitcoind can't block startup; /healthz is honest
about the degraded state once we do start serving.
This commit is contained in:
@@ -185,6 +185,12 @@ type Aggregator struct {
|
||||
// escalate to WARN.
|
||||
ckFailStreak int
|
||||
|
||||
// readyOnce + ready closes the Ready() channel exactly once after
|
||||
// the first refresh completes. main blocks briefly on this so the
|
||||
// HTTP server doesn't serve a never-refreshed (all-zeros) snapshot.
|
||||
readyOnce sync.Once
|
||||
ready chan struct{}
|
||||
|
||||
// Submit-attempt vs confirmed counters. Persisted in kv so the
|
||||
// running gap survives restarts. Both are monotonic.
|
||||
blockSubmitAttempts int64
|
||||
@@ -210,9 +216,21 @@ func New(ck *ckpool.Client, rpc *bitcoind.RPC, interval time.Duration, log *slog
|
||||
RPC: rpc,
|
||||
Interval: interval,
|
||||
Log: log,
|
||||
ready: make(chan struct{}),
|
||||
}
|
||||
}
|
||||
|
||||
// Ready returns a channel that's closed once the aggregator has
|
||||
// completed its first refresh — used by main to delay HTTP serving
|
||||
// until /api/snapshot reflects real state instead of zeros.
|
||||
func (a *Aggregator) Ready() <-chan struct{} {
|
||||
return a.ready
|
||||
}
|
||||
|
||||
func (a *Aggregator) markReady() {
|
||||
a.readyOnce.Do(func() { close(a.ready) })
|
||||
}
|
||||
|
||||
// Run blocks until ctx is cancelled, refreshing the snapshot every Interval.
|
||||
// It runs one immediate refresh at startup so readers don't see an empty
|
||||
// snapshot after ctx launches the goroutine. Persisted block history is
|
||||
@@ -486,6 +504,7 @@ func (a *Aggregator) refresh(ctx context.Context) {
|
||||
if cb != nil {
|
||||
cb(pushed)
|
||||
}
|
||||
a.markReady()
|
||||
}
|
||||
|
||||
// loadPersistedState restores cumulative work and hashrate history from
|
||||
|
||||
Reference in New Issue
Block a user