Skip to main content
dupcanon is currently human-operated, but it is being shaped for workflow-native automation in GitHub Actions.

What this is

Primary objective

Reduce duplicate noise in high-cadence repos (for example, OpenClaw) by selecting stable canonicals and closing true duplicates safely.

Why it exists

Naive automation creates duplicate chains and unstable targets; this system adds durable state + deterministic gates.

Operating constraints

Cheap enough to run continuously, accurate enough for guarded production use, and fully auditable.

Core strategy

Embeddings for retrieval, LLM for semantic judgment, deterministic policy for acceptance and close safety.

Current vs target mode

  • Human-operated CLI pipeline
  • Online detect-new in shadow/suggest mode
  • Close actions gated through reviewed plan-close + explicit apply-close --yes

Core approach

Two paths share one data model and one safety model.
  1. Online entrypoint: detect-new for newly opened issues/PRs
  2. sync / refresh: ingest issues + PRs into Postgres
  3. analyze-intent: extract intent cards for intent-first modeling (default)
  4. embed: intent-card embeddings in pgvector (raw title/body when --source raw)
  5. candidates: persisted nearest-neighbor candidate sets
  6. judge: LLM chooses duplicate target inside the candidate set
  7. deterministic gates veto risky decisions
  8. canonicalize: compute canonical representatives
  9. plan-close -> reviewed apply-close --yes
If you prefer raw embeddings, skip analyze-intent and pass --source raw to embed, candidates, judge, and detect-new.
All key artifacts are persisted for replay, audit, and threshold tuning.

Deterministic gates (with actual thresholds)

  • strict JSON parse
  • target must be in candidate set
  • confidence threshold: model_confidence >= 0.85 (min_edge default)
  • target must be open
  • candidate gap gate: selected_score - best_alternative_score >= 0.015
  • mismatch vetoes (uncertain/overlap/root-cause/scope class failures)
  • one accepted outgoing edge per source unless explicit rejudge
  • maintainer author protection
  • maintainer assignee protection
  • default canonical-only: direct accepted edge to canonical required
  • optional direct-fallback: use source item’s direct accepted target when canonical edge is missing
  • close threshold: selected direct edge confidence >= 0.90 (min_close default)
  • default thresholds: maybe=0.85, duplicate=0.92
  • strict duplicate downgrade if structural guardrails fail
  • duplicate class also requires strong retrieval support (current floor: top match score >= 0.90)
  • candidate gap gate also applies on strict duplicate path (>= 0.015)
Confidence vs score (important):
  • confidence = model’s self-reported duplicate confidence in [0,1]
  • score = retrieval similarity score from vector search
  • gap = selected_candidate_score - best_alternative_score
High confidence ≠ accepted edge.A judgment can have very high model confidence and still be rejected when deterministic gates fail.

Judge gate examples

Actual judge system prompt (current)

Cost and accuracy stance

Goal is practical operations, not perfect AI: keep cost low enough for continuous runs, keep precision high enough for controlled close actions.

Current status

Implemented commands:
  • init, sync, refresh, analyze-intent, embed
  • candidates, judge, judge-audit, report-audit
  • detect-new, search, llm
  • canonicalize, maintainers, plan-close, apply-close

What’s missing next

  • first-class evaluation command + reporting workflow for production gate decisions
  • programmatic orchestration command/workflow for unattended DB freshness updates
  • richer action surface in future (for example, label taxonomy / tree-editing operations)

Stack

  • Python + Typer + Pydantic + Rich
  • Supabase Postgres + pgvector
  • providers: OpenAI, Gemini, OpenRouter, and openai-codex via pi RPC
DB can be moved to self-hosted Postgres with minimal architecture changes.

Internal docs

Deep design and runbook docs are in docs/internal/.