Action Cable Stream Lifecycle Ownership - Plan

Goal Capsule

Keep Hive's application-level StatusChannel lifecycle fence as the active owner of the Action Cable subscription race, make its full lifecycle contract executable against real channels and both adapters, and define a fail-closed handoff to a future released Rails implementation. The current task does not remove the fence or change the Rails dependency.

Overview

Hive currently runs Rails and Action Cable 8.1.3.1. The framework implementation records a handler before its deferred adapter subscription and does not fence late registration or confirmation against channel teardown. StatusChannel therefore owns a small coordinated override around stream start and stop while retaining Rails' handler, callback, coder, and confirmation behavior.

The work first separates application-specific deterministic race tests from a reusable, mechanism-neutral adapter contract. It then makes the smallest application-level lifecycle repair needed to satisfy both, exercises real Async and Solid Cable adapters plus the browser boundary, and documents current rollback and a future framework handoff.

The removal gate is closed at plan time. The pinned Rails source and Rails main still show the unfenced registration shape; no exact upstream pull request or merge commit for the required lifecycle fix has been established. Removal may be planned only after that provenance and its first containing released gem are known.

Evidence Baseline

• web/Gemfile and web/Gemfile.lock resolve Rails/Action Cable 8.1.3.1; production uses Solid Cable 4.0.0.
• web/app/channels/status_channel.rb owns the local stream and broadcaster-lease lifecycle.
• web/test/channels/status_channel_test.rb and web/test/system/kanban_board_test.rb contain existing race, reconnect, rejection, delayed-confirmation, and cleanup coverage.
• Originating architecture issue: Hive issue #1143.
• Framework comparison points: Rails 8.1.3.1 stream source and Rails main stream source.

Product Contract

Summary and Problem Frame

Harden and verify the existing application-owned lifecycle fence without turning it into a framework fork. Action Cable registration crosses the channel event loop, worker pool, and pub/sub adapter. Teardown may occur before deferred work, during registration, or after registration but before confirmation. One synchronized owner must prevent confirmation of a dead channel, handler leaks, duplicate lease release, and ineffective or duplicate unsubscribe calls.

Preserve a reusable behavior contract so a future released Rails implementation can replace the application fence only after proving identical behavior in Hive.

Actor

• A1 — Hive Web maintainers: maintain Rails dependencies and the StatusChannel lifecycle contract, evaluate a released upstream candidate, and execute or roll back ownership changes.

Requirements

• R1 — Current ownership: Retain the application-local StatusChannel stream lifecycle fence while Hive uses a released Rails gem without the authoritative fix. It may narrowly override stream_from, stop_stream_from, and stop_all_streams because pending cancellation and cleanup ownership span those public methods.
• R2 — Dependency boundary: Use released Rails gems only. Do not pin a Git commit or prerelease, vendor or patch Rails files, or carry a broader framework backport.
• R3 — Conditional handoff: Remove the fence only in a future change that identifies the exact upstream fix and first released Rails gem containing it, upgrades Hive to that release, and verifies the framework-owned implementation against the unchanged mechanism-neutral contract.
• R4 — Teardown before subscription: If teardown wins before adapter registration, cancel the pending attempt, do not register or retain a handler, do not call raw adapter unsubscribe, do not confirm, and close the broadcaster lease exactly once.
• R5 — Teardown during or after registration: If teardown races or follows successful registration, unsubscribe each actually registered handler exactly once, send no confirmation after teardown, and leave no stream, pending-attempt, or lease state.
• R6 — Targeted stop: stop_stream_from must cancel a named pending attempt without raw unsubscribe; if it is already registered, it must unsubscribe exactly once. stop_all_streams must apply the same rule to every attempt. Repeated or concurrent stops are idempotent.
• R7 — Adapter failure: If subscription raises or rejects, leave no registered handler, stream entry, pending attempt, or lease state; send no confirmation; request one reconnect; and allow later recovery.
• R8 — Successful-stream compatibility: Preserve Rails' default transmit path, explicit callback/block behavior, coder behavior, live confirmation timing, inherited public stop semantics, and existing catch-up behavior.
• R9 — Acceptance evidence: Require deterministic negative controls, mechanism-neutral real-channel coverage, real Async and Solid Cable coverage, one-time bounded stress evidence, browser/system coverage, and broad Web checks. Tie future removal evidence to exact released provenance.
• R10 — Fail-closed rollback: Define recovery both for the lifecycle coordination shipped now and for a future framework handoff.

Key Decisions

• KD1 — Released gems only: keep the application fence until a qualifying release exists; reject Git/prerelease pins, vendoring, monkey patches, and broad backports.
• KD2 — Conditional framework handoff: removal is a future evidence-gated operation.
• KD3 — Full race contract: both teardown windows, targeted pending stop, exact cleanup, confirmation suppression, state absence, adapter failure, callback/coder compatibility, and both adapters are mandatory.
• KD4 — Pending-attempt ownership: do not place a handler in Rails' streams registry until adapter registration succeeds. Track queued/registering attempts in an application-owned registry. Narrowly override stop_stream_from and stop_all_streams to cancel that pending registry or delegate registered cleanup, preventing inherited stop methods from issuing an ineffective early unsubscribe.
• KD5 — Fail closed: any required verification failure rejects a future candidate; production trouble in the current coordination triggers the documented application rollback.

Success Criteria

• Each deterministic barrier test fails against an explicit unfenced negative-control channel and passes against StatusChannel.
• Mechanism-neutral assertions instantiate a real channel and connection without ActionCable::Channel::TestCase::ChannelStub; application-specific barriers may use controlled event-loop/pubsub doubles.
• Tests separately count pending cancellation, effective registration, raw unsubscribe, lease release, confirmations, payloads, reconnects, and residual state.
• Async and Solid Cable execute the same real-channel contract without leaked listeners, threads, rows, or temporary databases.
• Browser coverage sees no confirmation or catch-up payload after teardown and still recovers from subscription failure.
• Both race windows complete 100 bounded stress iterations with count and seed recorded as one-time acceptance evidence.
• Documentation records the dependency baseline, private-API drift guard, current rollback, and precise future handoff.

Requirements Trace

Scope Boundaries

In Scope

• Deterministic application-specific ordering tests and an unfenced negative control.
• A reusable contract using real channel/connection objects and real adapters.
• Application-owned coordination across stream start and public stop methods.
• Isolated Solid Cable test database setup, browser evidence, stress mode, documentation, and drift guards.

Out of Scope

• Removing the fence under the current lock; Git/prerelease Rails pins; framework vendoring or monkey-patching; a broader backport.
• Reworking status_stream_source.js, protocol, or lease design unless required by an unchanged contract.
• Publishing, deployment, release selection, or upstream Rails contribution.

Deferred Follow-Up

Identify exact merged upstream provenance and its first released gem. In one future change, raise Hive's Rails floor, update all Rails components and lockfile, remove only obsolete fence code, and run the applicable unchanged verification matrix. Restore dependency declaration, lockfile, and fence atomically on failure.

Planning Contract

Assumptions

• AS1 — Cleanup accounting: queued work has zero registrations and zero raw unsubscribes. Every actually registered handler has exactly one raw unsubscribe. Pending cancellation and broadcaster lease release are separately asserted exactly once.
• AS2 — Registry split: pending attempts live only in the application registry; successfully registered handlers move atomically into Rails' streams. The overridden stop methods consult both, cancel pending work locally, and unsubscribe only registered handlers.
• AS3 — Adapter matrix: coverage means real Async plus real Solid Cable with an isolated temporary cable database and explicit listener shutdown.
• AS4 — Stress: STATUS_CHANNEL_STRESS_ITERATIONS defaults to 0; STATUS_CHANNEL_STRESS_SEED defaults to a generated value printed before execution. Stress is one-time task acceptance evidence, not a default CI regression gate.
• AS5 — Provenance: an issue, unreleased main, or adjacent fix is insufficient; future handoff requires an exact merge and official released gem containing it.

Key Technical Decisions

• KTD1 — Two complementary suites: application-specific barrier regressions own deferred scheduling details; a reusable mechanism-neutral contract observes behavior only and is valid for a synchronous or asynchronous future owner.
• KTD2 — Explicit per-attempt ownership: synchronize queued, registering, registered, failed, closing, and closed transitions. Only an actually registered handler has an unsubscribe owner.
• KTD3 — Non-reentrant callbacks: post completion and late cleanup to the channel event loop so unsubscribe cannot re-enter Solid Cable subscriber locks.
• KTD4 — Real objects for adapter proof: never use the Rails channel-test ChannelStub in the adapter-backed suite because its stream_from only records names.
• KTD5 — Atomic future handoff: dependency floor, lockfile, fence removal, provenance, and evidence form one reversible future change.

High-Level Design

StatusChannel maintains pending attempts separately from registered Rails streams. A targeted or global stop marks pending attempts closed before deferred work can register. Registration completion either atomically publishes the handler into streams and confirms a live channel, or observes closure and schedules exactly one late unsubscribe outside the adapter callback. Registered stops use the normal stream registry semantics through the narrow override/delegation path.

Compatibility Constraints

• Use worker_pool_stream_handler as the canonical callback/coder wrapper.
• Preserve the public behavior of stop_stream_from and stop_all_streams, including pending attempts.
• Keep lease state independent from adapter registration state.
• Confirm only after successful registration while live.
• Never unsubscribe inline from an adapter success callback.
• Keep the override's dependency on private Rails APIs explicit and enforced by tests.

Implementation Units

U1 — Encode Deterministic Regressions and the Reusable Contract

Goal: Make every race, failure, stop, and success behavior observable without a false-green channel stub.

Requirements: R3-R9.

Files:

• web/test/channels/status_channel_test.rb
• web/test/support/status_channel_stream_lifecycle_contract.rb (new)

Approach:

1. Keep application-specific deterministic tests in status_channel_test.rb. Use bounded queues/latches around a controlled event loop and pubsub fixture to force teardown before deferred start and during registration completion. Record registration, raw unsubscribe, pending/stream state, callbacks, frames, reconnects, and lease release independently.
2. Define an unfenced negative-control channel that invokes the superclass ActionCable::Channel::Streams#stream_from via instance_method(...).bind_call(...); do not patch Rails. Require each barrier regression to demonstrate its forbidden late-registration/confirmation or cleanup outcome against this control before asserting the fence passes.
3. Add targeted stop_stream_from during queued and registering states, plus stop_all_streams, repeated stop, post-registration stop, teardown, and failure races. Assert zero raw unsubscribe before real registration and exactly one afterward.
4. Extract mechanism-neutral assertions that instantiate a real StatusChannel (and later a framework-owned candidate) with a real ActionCable::Connection::Base test connection. Do not use ActionCable::Channel::TestCase subscriptions or ChannelStub. Do not require a deferred-start barrier in this reusable layer.
5. In the real-object contract, cover successful registration/confirmation, payload delivery, targeted/global stop, teardown, subscribe failure and recovery, default transmit, callback/block, coder, and empty final state. Let adapters register synchronously or asynchronously.
6. Add a private-API drift guard asserting the required methods exist (worker_pool_stream_handler, defer_subscription_confirmation!, ensure_confirmation_sent, streams) and that pubsub.subscribe supports the three-argument success-callback call shape used by the fence. Make failures name the changed API.
7. Bound every asynchronous wait and emit lifecycle state on timeout; use no timing sleeps.

Verification:

• Run cd web && bin/rails test test/channels/status_channel_test.rb.
• Record that each deterministic negative control fails for the expected invariant while StatusChannel passes; the test itself should treat the expected negative-control observation as success.
• Confirm the reusable module neither names the application override nor depends on deferred scheduling or ChannelStub.

U2 — Make StatusChannel the Single Start/Stop Cleanup Owner

Goal: Satisfy the lifecycle contract with an application-local coordination change while preserving successful behavior.

Requirements: R1, R4-R8.

Dependencies: U1.

Files:

• web/app/channels/status_channel.rb
• web/test/channels/status_channel_test.rb

Approach:

1. Add a synchronized application registry for queued/registering attempts. Do not insert them into Rails' streams until registration succeeds.
2. Narrowly override stop_stream_from and stop_all_streams alongside stream_from. Under the same lifecycle synchronization, cancel named/all pending attempts locally; for registered entries, remove/delegate exactly once using Rails-compatible semantics.
3. If teardown or targeted stop wins before deferred work starts, prevent registration, confirmation, reconnect, and raw unsubscribe while closing the lease once.
4. If stop wins while registration is completing, let completion observe closure, suppress confirmation/payloads, and post exactly one unsubscribe for the actual late registration. Clear both registries and lease state.
5. On failure, clear pending/stream state before releasing the lease and emitting one reconnect. Make concurrent stop/failure idempotent.
6. Preserve worker-pool handler construction, callback/coder behavior, stream names, protocol, cursor, and catch-up semantics. Keep late cleanup on the event loop outside adapter callbacks.
7. Comment why all three overrides form one narrow ownership boundary and link the documented removal gate without predicting a Rails version.

Verification:

• Run the focused U1 file until all cases pass.
• Confirm pending stop_stream_from, repeated/global stop, both teardown orderings, and concurrent failure leave zero state with AS1 counts.
• Review the diff for application locality and unchanged client/lease contracts.

U3 — Prove Real Adapters, Browser Behavior, and Repetition

Goal: Run the mechanism-neutral contract through real Async and Solid Cable and retain browser and stress evidence.

Requirements: R4-R10.

Dependencies: U1, U2.

Files:

• web/test/channels/status_channel_test.rb
• web/test/channels/status_channel_solid_cable_test.rb (new, required)
• web/test/system/kanban_board_test.rb
• web/config/database.yml
• web/config/cable.yml (read-only configuration input unless isolated test wiring demonstrably requires a non-default named configuration; do not repoint test:)
• web/db/cable_schema.rb
• .github/workflows/ci.yml (inspection only; stress remains outside CI)

Approach:

1. Run the reusable contract with a real connection/channel and the real Async adapter, including event-loop scheduling, delivery, targeted/global stop, and shutdown.
2. Always run the same contract in status_channel_solid_cable_test.rb. From the test, build an isolated Solid Cable adapter/configuration and temporary cable database using web/db/cable_schema.rb; add only the test database structure needed in database.yml. Do not change cable.yml's existing test: async entry, because system tests require in-process delivery.
3. Ensure listener/thread shutdown, connection clearing, rows, and temporary database cleanup execute in teardown/ensure paths after both pass and injected failure.
4. Extend existing browser cases only where distinct transport evidence is absent: observe frames around disposed subscriptions and assert no late confirmation or catch-up payload. Preserve retry, bounded never-confirmed cleanup, rejection, reconnect, and catch-up cases.
5. Implement stress in the focused channel test with STATUS_CHANNEL_STRESS_ITERATIONS defaulting to 0 and STATUS_CHANNEL_STRESS_SEED controlling reproducible order. When enabled, print adapter/window, iteration count, and seed at start and on failure. Execute both deterministic windows for the configured count, with per-iteration bounds and cleanup assertions.
6. Keep stress as retained one-time acceptance evidence outside the default bin/rails test/CI path; deterministic tests and both real-adapter suites remain the regression gate.

Verification:

• Async: cd web && bin/rails test test/channels/status_channel_test.rb.
• Solid Cable, unconditionally: cd web && bin/rails test test/channels/status_channel_solid_cable_test.rb.
• Focused browser: cd web && bin/rails test test/system/kanban_board_test.rb.
• Stress acceptance: cd web && STATUS_CHANNEL_STRESS_ITERATIONS=100 STATUS_CHANNEL_STRESS_SEED=<recorded-integer> bin/rails test test/channels/status_channel_test.rb; retain emitted count/seed and verify 100 iterations for each window.
• Inspect listener/thread/connection/temp-database cleanup after a pass and an injected failure.

U4 — Document Ownership, Drift, Rollback, and Future Handoff

Goal: Make current operations and future Rails ownership precise and fail closed.

Requirements: R1-R3, R9-R10.

Dependencies: U1-U3.

Files:

• wiki/dependencies.md
• wiki/state-model.md
• wiki/testing.md
• wiki/commands/web.md
• wiki/gaps.md
• wiki/log.d/<timestamp>-action-cable-stream-lifecycle.md (new)
• web/Gemfile and web/Gemfile.lock (future handoff inputs only; unchanged now)

Approach:

1. Record Rails/Action Cable 8.1.3.1, Solid Cable 4.0.0, issue #1143, application ownership, and absence of a qualifying release.
2. Document pending versus registered state, effective/raw cleanup accounting, targeted/global stop behavior, lease independence, and all focused/adapter/browser/stress commands.
3. In wiki/dependencies.md, list the private APIs and three-argument subscribe callback shape relied on by the fence and name U1's guard as the automatic Rails-update drift trigger.
4. Define the future provenance packet: upstream PR, merge commit, official release/tag, gem version, lock resolution, and released-source comparison.
5. Define the future atomic change: raise the Rails minimum, update all Rails components and lockfile, remove only obsolete fence/helpers, and keep the reusable contract unchanged.
6. Add a current-task rollback runbook: if the new U2 coordination deadlocks, leaks, or suppresses legitimate confirmation after merge, revert U2's production lifecycle changes while retaining U1's owner-neutral/negative-control tests and documentation evidence. Watch the StatusBroadcaster subscriber lease count for drift and the never-confirmed cleanup/browser path in kanban_board_test.rb for boards that never become live; rerun focused Async, Solid Cable, and the affected system case before redeploying a repaired change.
7. Define future rollback as atomic restoration of the previous released Rails declaration/lock plus the application fence. Do not claim framework ownership until every applicable gate passes.

Verification:

• Cross-check current versions with the lockfile and future versions with official artifacts.
• Dry-review both rollback paths and ensure each failure has an explicit stop/restore action.
• Confirm current diff leaves web/Gemfile and web/Gemfile.lock unchanged.
• Confirm the log links only evidence actually produced.

Verification Contract

Evidence Quality

• Application race tests use barriers, not sleeps; reusable tests are mechanism-neutral.
• Adapter suites use real connection/channel objects and never ChannelStub.
• Every wait is bounded and reports lifecycle state.
• Pending cancellation, registration, raw unsubscribe, stream/pending registry, lease, reconnect, confirmation, and payload are observed separately.
• Stress reports its seed/count and is not silently run by the normal suite.
• Both adapters clean up on pass and failure.
• Future provenance comes from official Rails repository/release artifacts.

Rollback Acceptance

• Current task: operator-visible failure signals are a drifting StatusBroadcaster subscriber lease count and the never-confirmed cleanup path/board failing to become live. Revert U2 production coordination while retaining U1 contracts, then pass focused Async, Solid Cable, and affected browser coverage before shipping a repair.
• Future handoff: restore the last verified released Rails declaration and lockfile together with the application fence, rerun focused lifecycle plus the failing layer, and keep Rails ownership rejected until a later candidate passes all applicable rows.

Risks

Definition of Done

Current Task

• [ ] The application fence across stream_from, stop_stream_from, and stop_all_streams remains the documented lifecycle owner.
• [ ] Each deterministic race test demonstrates failure against the explicit unfenced negative control and success against StatusChannel.
• [ ] Pending targeted/global stop, both teardown windows, exact raw unsubscribe counts, no late confirmation/payload, empty state, and exactly-once lease release are proven.
• [ ] Failure tests prove clean state, one reconnect, no confirmation, and later recovery.
• [ ] Default transmit, callback/block, coder, stop API, catch-up, and live confirmation semantics remain unchanged.
• [ ] Real-channel Async and unconditional Solid Cable suites pass without ChannelStub and clean all resources.
• [ ] Both race windows pass 100 explicit stress iterations with retained count/seed; normal tests run zero stress iterations.
• [ ] Focused browser, full Web/system, golden path, and lint checks pass.
• [ ] Private Rails API drift guard passes and is documented as the dependency-update trigger.
• [ ] Documentation records current baseline/owner, both rollback paths, removal provenance, and future atomic gate.
• [ ] Current Rails declaration and lockfile remain unchanged; no framework patch, vendoring, prerelease, or Git pin is introduced.

Future Framework Handoff

• [ ] Exact upstream PR/merge and first official released gem containing it are verified from released source.
• [ ] Rails floor and all Rails lockfile components move atomically with fence removal.
• [ ] The unchanged mechanism-neutral, adapter, stress, system, broad-suite, golden-path, and lint rows pass without the fence.
• [ ] The current-task-only no-dependency-change row is excluded; provenance and atomic dependency checks pass.
• [ ] Any failure restores the prior released dependency set and application fence before deployment; only a complete pass transfers ownership.