Questions and decisions · Track skill and workflow improvements

Round 1

Q1. Which write flow should be authoritative for a proposal record?

Choose the closest model: (a) the proposing agent or operator opens the record and a named evaluator finalizes it, (b) Hive opens and advances the record automatically from task and evaluation events, or (c) an operator records the proposal only after the decision. If roles differ for skill and workflow proposals, describe the split.

A1.

Choose (b): Hive opens and advances records from task/evaluation events, while named evaluators and operators author decisions. Use the same flow for skills and workflows.

Q2. Where should v1 stop between preserving evaluation knowledge and changing active behavior?

Choose one boundary: (a) tracking only—the record and compiled knowledge view never edit an active skill or workflow, (b) accepted decisions may activate a referenced change while rejection is record-only, or (c) the record is the source from which active skills and workflows are compiled. State whether rollback must also perform a revert or merely document one performed elsewhere.

A2.

Choose (a) tracking only for v1. Records and compiled views never mutate active behavior; rollback documents and links the externally performed revert. Activation becomes a separate approval-gated integration.

Q3. What constitutes one proposal identity when the same idea is revised, retried, or evaluated more than once?

For example, should every proposed revision receive a new stable ID linked by supersedes or retries, while multiple evaluations attach to that ID, or should an ID follow the idea across revisions? Also state whether IDs must be reproducible from content or only permanent once minted.

A3.

Mint a new permanent opaque ID for each candidate revision, linked with supersedes/retries; multiple evaluations attach to that revision ID. IDs need not be content-derived but never change after minting.

Q4. What lifecycle and history semantics should be canonical?

Choose whether records are (a) immutable proposal data plus append-only evaluation and decision events, (b) mutable snapshots with an audit trail, or (c) another model. Define the allowed path to accepted, rejected, superseded, and rolled_back, including whether a rolled-back acceptance remains historically accepted and links to a later rollback event.

A4.

Use immutable proposal data plus append-only evaluation and decision events. One authoritative terminal decision moves to accepted/rejected; superseded links to successor; rolled_back records a later compensating event while historical acceptance remains true.

Q5. What should refresh or compile produce, and which artifact is the source of truth?

For example: validate individual canonical records, then deterministically regenerate a machine-readable index and human wiki summary without touching active skills. Name the outputs that users or agents must be able to consume and whether generated files are committed or rebuilt on demand.

A5.

Canonical individual records/events are source of truth. Validate them and deterministically generate a machine-readable index plus human wiki summary, committed when normal llm-wiki artifacts are committed; active skills remain untouched.

Q6. How should concurrent or contradictory evaluations be resolved?

Choose the expected behavior when two writers evaluate or decide the same proposal from the same base: (a) retain both evaluations but require one explicit authoritative decision, rejecting conflicting terminal decisions, (b) deterministic last-writer-wins, or (c) another rule. Identify who or what may issue the authoritative decision.

A6.

Retain all evaluations but require one explicit authoritative decision from the configured human/operator or policy authority. Reject conflicting terminal decisions through compare-and-swap; never use last-writer-wins.

Q7. Which fields are mandatory at proposal creation versus at each terminal outcome, and what should happen to incomplete or malformed records?

Please confirm or amend this baseline: subject kind and revision, proposed change, motivation/evidence, author provenance, and stable ID at creation; evaluator provenance, method or benchmark identity, result, decision rationale, and task/commit links at decision; rollback reason and reverted revision for rolled_back; successor ID for superseded. State whether valid drafts are allowed and whether malformed records fail the whole refresh, are skipped with diagnostics, or are quarantined.

A7.

Adopt the baseline fields. Allow schema-valid drafts, exclude them from terminal summaries, and quarantine malformed records with diagnostics without aborting compilation of valid records.

Q8. How must retained evidence be surfaced so rejected and rolled-back lessons actually influence future work?

Choose the minimum useful access path—direct wiki browsing, CLI list/show/filter commands, automatic context supplied during future skill or workflow proposals, or some combination—and state any privacy or retention limits on evaluator notes, benchmark artifacts, and external links.

A8.

Provide direct wiki browsing plus CLI list/show/filter and automatically supply a bounded relevant set to future proposal workflows. Redact secrets/private notes, store digests for restricted artifacts, and make retention/visibility explicit.

Q9. What concrete accepted and rejected examples should anchor the documentation and acceptance tests?

Name one realistic Hive skill or workflow proposal for each outcome, including the evidence that justified the decision. If historical backfill is in scope, also identify which existing records or tasks should be imported; otherwise confirm that v1 covers new proposals only.

A9.

Accepted example: a review prompt change that improves held-out review recall with no protected regressions. Rejected example: a broader planning prompt that raises target score but increases false positives/cost beyond threshold. V1 covers new proposals only; no historical backfill.

Requirements

Actors and authority: Hive automatically opens and advances skill and workflow proposal records from task and evaluation events. Proposers supply the candidate and evidence; named evaluators add evaluations; only the configured human/operator or policy authority may issue the authoritative decision.
V1 boundary: This is tracking-only. Records, indexes, summaries, and supplied context never mutate an active skill or workflow. Acceptance activation is a separate approval-gated integration, and rollback records and links an externally performed revert.
Identity and provenance: Mint a permanent opaque proposal ID for every candidate revision. A revision may link to another with supersedes or retries; multiple evaluations attach to the same revision. Proposal data is immutable, while evaluations, decisions, supersession, and rollback are append-only events with actor and source provenance.
Required data: Creation requires subject kind and revision, proposed change, motivation/evidence, author provenance, and stable ID. A decision requires evaluator provenance, evaluation method or benchmark identity, result, rationale, and relevant task/commit links. Supersession requires a successor ID; rollback requires its reason and reverted revision.
Lifecycle: Schema-valid drafts are allowed. One authoritative terminal decision establishes accepted or rejected; superseded links the candidate to its successor. A later rolled_back compensating event preserves that the proposal was historically accepted while recording why and where it was reverted.
Evaluation and concurrency: Preserve every evaluation, including negative and contradictory evidence. Conflicting terminal decisions from the same base must fail compare-and-swap validation; never resolve them with last-writer-wins.
Canonical and compiled views: Canonical individual records and events are the source of truth. Refresh validates them and deterministically regenerates a machine-readable index and a human-readable llm-wiki summary without touching active behavior. Generated views are committed when normal llm-wiki artifacts are committed.
Invalid input handling: Exclude valid drafts from terminal summaries. Quarantine malformed or incomplete records with actionable diagnostics while continuing to compile all valid records.
Discovery and retention: Support direct wiki browsing, CLI list/show/filter access, and bounded relevance-based context for future skill or workflow proposal runs. Redact secrets and private notes, retain digests for restricted artifacts, and record visibility and retention policy explicitly.
Scope: V1 applies to new proposals only; historical backfill is out of scope.
Acceptance example — accepted: Given a review-prompt candidate whose held-out benchmark improves review recall with no protected regression, an authorized acceptance preserves its evidence and appears identically in the canonical record, machine index, and wiki summary without activating the prompt.
Acceptance example — rejected: Given a broader planning-prompt candidate that improves its target score but exceeds false-positive or cost thresholds, rejection preserves the benchmark and rationale, leaves the active prompt unchanged, and makes the lesson discoverable through wiki, CLI, and future bounded context.
Acceptance example — superseded and rolled back: A revised candidate receives a new ID linked from its predecessor; a previously accepted candidate can later record a rollback that preserves the original acceptance and links the compensating revert and reason.
Acceptance example — deterministic failure isolation: Concurrent conflicting decisions permit at most one authoritative terminal event, and repeated compilation over the same valid inputs is byte-stable; malformed records are quarantined with diagnostics and do not prevent valid outputs.