Aveloxis

Getting Started

  • Installation
    • Requirements
      • Verify prerequisites
    • Install via go install (recommended)
    • Install via go build (local binary)
    • PATH troubleshooting
      • Find the Go bin directory
      • Add it to your PATH
      • Verify
    • Installing optional tools
    • Verifying installation
    • Updating
    • Next steps
  • Configuration
    • Creating the config file
    • Full config reference
      • Database
      • GitHub
      • GitLab
      • Collection
        • Scancode worker (v0.21.0)
        • DistributionWorker (v0.24.0)
        • Mailing-list worker (v0.25.7)
      • Web (OAuth + GUI)
      • Monitor (dashboard, v0.23.0)
      • Mail (Gmail SMTP, optional)
      • Logging
    • API key sources
    • API key rotation behavior
      • Throughput math
    • Clone directory
    • Public analytics API (api)
      • Reaching the API from another host
      • Response caching (advanced, off by default)
    • Email (Gmail SMTP, optional)
      • Setup
      • Validation at startup
      • Verifying the setup with aveloxis test-mail
      • Transport details
      • Common failure modes
      • Disabling
    • v0.25.x distribution-tracking knobs
      • The settings in this group
      • Why they exist
      • Lifecycle and deprecation horizon
      • What operators on fresh installs should do
      • What operators upgrading through v0.25.x should do
    • Next steps
    • Operator vulnerability digest (v0.27.12)
  • Quick Start
    • Prerequisites
    • Step 1: Create a config file
    • Step 2: Create the database schema
    • Step 3: Store your API keys
    • Step 4: Add repos to the collection queue
      • Add a single repo
      • Add multiple repos
      • Add all repos from a GitHub organization
    • Step 5: Start the processes
    • Check the monitoring dashboard
    • Verify data in the database
    • What happens next
    • Next steps
  • Production Deployment
    • The three processes
    • 1. Requirements
    • 2. Install
    • 3. PostgreSQL
    • 4. Configuration
    • 5. Schema
    • 6. API keys
    • 7. OAuth applications and the first admin
    • 8. Running
    • 9. Reverse proxy
    • 10. Verify
    • 11. Upgrading
  • Augur Migration
    • Overview
    • Step 1: Point at the existing Augur database
    • Step 2: Create the Aveloxis schemas
    • Step 3: Import API keys
    • Step 4: Import repos
    • Schema coexistence
    • Contributor ID compatibility
    • Running both systems side-by-side
      • Gradual migration
      • Parallel collection for validation
      • Resource considerations
    • Differences from Augur
    • Cleanup (optional)
    • Next steps
  • Upgrading
    • The four steps
    • The two halves of an upgrade
    • The standard ladder
    • What to watch for in the migrate output
    • Configuration compatibility
    • Operator-run heals, with the release that introduced each
      • The v0.26.3 event-cohort SQL
      • Mailing lists on an existing catalog
    • Recurring maintenance (not tied to a release)
    • Where the per-release detail lives

User Guide

  • Commands Reference
    • aveloxis serve
      • Flags
      • Behavior
      • Periodic tasks
      • Scancode worker tuning
      • Examples
    • aveloxis web
      • Configuration
      • OAuth app setup
      • Behavior
      • Examples
    • aveloxis api
    • aveloxis scancode-worker
    • aveloxis test-mail
    • aveloxis collect
      • Flags
      • Examples
    • aveloxis add-repo
      • Flags
      • URL formats
      • Platform detection
    • aveloxis add-key
      • Flags
      • Examples
    • aveloxis prioritize
    • aveloxis recollect
      • Automatic triggering (v0.18.24+)
    • aveloxis migrate
      • The completed-backfill ledger (v0.28.4)
    • aveloxis backfill-identities
    • aveloxis dedup-repos
      • Flags
      • What each merge does
      • Precondition
      • After the run
    • aveloxis migrate-cntrb-ids
    • aveloxis merge-cntrb-collisions
    • aveloxis heal-collection-gaps
    • aveloxis refresh-views
    • aveloxis sbom
    • aveloxis install-tools
    • aveloxis upgrade-tools
    • aveloxis start
    • aveloxis stop
      • How a backend is placed on a host
    • aveloxis data-test
      • Flags
      • Behavior
      • Examples
      • Column-fill diff (v0.26.1)
    • aveloxis shadow-diff
    • aveloxis reconcile-repos
    • aveloxis generate-showcase
    • aveloxis adopt-forge-id
    • aveloxis backfill-repo-metadata
    • aveloxis rewalk-whitespace
    • aveloxis data-verify
    • aveloxis heal-messages
    • aveloxis heal-libyear
    • aveloxis heal-commit-daily
    • aveloxis heal-vulnerabilities
    • aveloxis mark-gone-repos
    • aveloxis run-scorecard
    • aveloxis staging-stats
    • aveloxis distribution-stats
    • Mailing-list commands
      • aveloxis load-foundation-core-repos
      • aveloxis load-foundation-orgs
      • aveloxis load-apache-lists
      • aveloxis register-mailing-list
      • aveloxis backfill-issue-external-keys
      • aveloxis mailing-list-stats
      • aveloxis verify-mailing-list
    • aveloxis deploy-checklist
    • aveloxis ack-deploy
    • aveloxis register-jira-projects
    • aveloxis backfill-jira-identities
    • aveloxis strip-quoted-history
    • aveloxis resolve-email-identities
    • aveloxis backfill-mailing-list-projection
    • aveloxis load-numfocus-projects
    • aveloxis load-numfocus-orgs
    • aveloxis version
    • Global behavior
      • Config file
      • Exit codes
      • Signal handling
  • Web GUI
    • Prerequisites
      • Creating a GitHub OAuth App
      • Creating a GitLab OAuth App
    • Configuration
    • Starting the Web GUI
    • Login Flow
    • Creating Groups
    • Adding Individual Repos to a Group
    • Adding an Entire GitHub Org or GitLab Group
    • Navigation and Breadcrumbs
    • Comparing Repositories
    • Searching and Pagination
      • Search
      • Pagination
    • How Org Tracking Works
    • How Repos Get Queued for Collection
    • Session Management
    • Running Alongside aveloxis serve
    • Security Considerations
  • REST API
    • Endpoints
      • Health Check
      • Public fleet stats
      • Repository Statistics
      • Forge-ID changes (admin, v0.29.63)
      • Batch Statistics
      • Time Series
      • Dependency Licenses
      • Repository Search
      • SBOM Download
      • Contributor identities in a window
      • Affiliation breakdown for the same window
      • Tweaks you can make on the client side
      • Top contributors, ranked (v0.27.61)
      • Where else are these contributors active? (v0.27.64)
      • One contributor’s cross-repo activity (v0.27.64)
      • Knowing whether your coverage is complete
      • When to use which endpoint
      • Mailing-list collection coverage
    • Scancode results
    • Augur-compatible metric endpoints
      • License drill-down on /deps (v0.28.1)
    • CORS
    • Deployment
    • Comparison analytics (v0.27.2)
    • Authentication: getting an API token
    • Vulnerabilities and home tab (v0.27.4)
    • Supply-chain package view (v0.29.60)
    • Portal and admin endpoints (v0.27.3)
    • Collections (v0.27.63)
    • Window-boundary semantics (v0.27.39)
  • Visualizations
    • Design Principles
    • Requirements
    • Repository Detail Page
      • Summary Cards
      • Weekly Time Series Charts
      • Dependency License Table
      • Source Code Licenses (ScanCode)
      • SBOM Downloads
    • Comparison Page
      • Selecting Repositories
      • Comparison Modes
      • Charts
      • Sharing
    • Performance
    • Technical Implementation
  • Aveloxis Metrics — Improvements on CHAOSS metrics
    • Contributors {#contributors}
    • Change Requests {#change_requests}
    • Change Requests Merged {#change_requests_merged}
    • Issues {#issues}
    • Issues Closed {#issues_closed}
    • Code Change Commits {#code_change_commits}
    • Committers {#committers}
    • Burstiness {#burstiness}
    • Project Velocity {#project_velocity}
    • Contributor Retention (Drive-by vs Repeat) {#contributor_retention}
    • Labor Investment {#labor_investment}
    • Upstream Code Dependencies {#upstream_dependencies}
    • License Coverage {#license_coverage}
  • Collection Pipeline
    • Overview
      • Collection order
      • Parallel collection
    • Prelim phase
      • Redirect detection
      • Dead repo sidelining
      • Duplicate checking
    • Phase 1: Staging (staged pipeline only)
      • Staging order
      • Envelope types
      • Batch flushing
    • Phase 2: Processing (staged pipeline only)
      • Processing order
      • Contributor resolution
      • Envelope processing
      • Error isolation
    • Phase 3: Facade (git)
      • Bare clone
      • Git log parsing
      • Affiliation resolution
      • Facade aggregates
    • Phase 5: Commit author resolution
      • Resolution strategy (cheapest first)
      • For each resolved author
    • Periodic: Contributor enrichment and canonical emails
    • Phase 4: Analysis
      • Dependency scanning
      • Libyear calculation
      • Code complexity (scc)
      • ScanCode Toolkit (license and copyright detection)
      • OpenSSF Scorecard (remote-first, local fallback)
    • Periodic tasks
    • Next steps
  • Monitoring
    • Dashboard
      • URL and configuration
      • Queue statistics
      • Repo status table
      • Search and pagination (v0.18.6)
      • Boost button
    • REST API endpoints
      • GET /api/queue
      • GET /api/stats
      • POST /api/prioritize/{repoID}
    • Log output
      • INFO level (default)
      • DEBUG level
      • Where logs go
    • Checking status via psql
    • Subsystem health (aveloxis_status)
    • Next steps
  • Running Aveloxis as a Service
    • Pick one manager
    • Unit files
    • Enable and manage
    • Upgrading under systemd
    • The PATH trap
    • Logs
    • Automatic restarts, unattended upgrades, and needrestart
    • Dedicated scancode host
    • What about macOS?
  • Dedicated Scancode Host
    • Why
    • 1. PostgreSQL remote access
    • 2. Install aveloxis + the scancode toolchain
    • 3. Minimal config template
    • 4. Turn scancode OFF on the primary server
    • 5. Run it
      • What NOT to run on this host
      • systemd unit example
    • 6. One-time libmagic version lock
    • Monitoring
  • CI/CD Pipelines
    • Workflows
      • Tests (test.yml) — unit tier
      • Integration (integration.yml) — integration tier (v0.21.1)
      • Local dev — running the integration tier
      • Lint (lint.yml)
      • Docs (docs.yml)
      • CodeQL (codeql.yml)
      • Container Build (container-build.yml)
      • Docker Publish (docker-publish.yml)
    • Status Badges
    • Dockerfile
    • Running in Docker
    • Schema change verification
      • When the gate applies
      • Running the gate
      • Interpreting results
      • CI integration (optional)
    • Fuzzing (cifuzz.yml)
    • Network canary (network-canary.yml)
  • Schema-change verification with aveloxis data-test
    • When to use it
    • What it actually verifies
    • Prerequisites
    • Running the harness
    • Interpreting the report
      • When all rows are PASS
      • When some rows are FLAG
      • When any rows are FAIL
    • Phase-by-phase walkthrough
    • Why full collection (--full)
    • Design decisions worth knowing
    • CI integration
    • Limitations
    • Related
    • Column-fill diff (v0.26.1)
    • Expected event-table asymmetry vs pre-0.26.2 tags
    • --diff-only: re-running the diff against kept databases (v0.27.129)
    • Schema shape drift in the report (v0.27.129)
  • Scaling
    • Worker count recommendations
      • Rule of thumb
      • Too many workers
      • Too few workers
    • Rate limit math
      • Estimating collection time
      • Background sweeps and the foreground reserve
    • Horizontal scaling
      • Setup
      • What is shared
      • Considerations
    • Database connection pool
      • PostgreSQL configuration
    • PostgreSQL server tuning
      • Recommended settings
      • Why work_mem is the dangerous one
      • Reference configuration
      • Operating-system settings
      • Verifying what is actually live
      • Write-heavy hosts: WAL, checkpoints and temporary files
      • Diagnostic settings
      • Drift is the failure mode
    • Clone directory sizing
      • Sizing estimates
      • Recommendations
    • Queue behavior
      • Many repos, few workers
      • Few repos, many workers
      • Priority override
    • Sizing summary
    • Routine performance snapshot
    • Next steps
  • Troubleshooting
    • Commits not collected, but Issues and Pull Requests are collected
    • Monitor dashboard renders slowly on a large fleet
    • Search keystrokes freeze the dashboard at 100K repos
    • /api/queue endpoint slow or returns huge JSON
    • Token invalidation (401 vs 403)
      • HTTP 401 Bad Credentials
      • HTTP 403 Forbidden / Rate Limited
      • rate limit exhausted repeats for one key while other keys have budget
      • Observation lines added in 0.29.70
    • FK constraint violations
    • “No API keys configured” / Startup failure
    • “Commit resolution FAILED”
    • “No data collected”
    • unsupported Unicode escape sequence (SQLSTATE 22P05)
    • “Pull requests / contributors / events: not found” or “forbidden”
    • Gap-filled historical issues/PRs have no comments
    • Gap-fill failure on large repos persists across multiple cycles
    • Metadata shows issues/PRs but gathered count stays at 0
    • Repeated “unexpected status 301” retries on moved/renamed repos
    • HTTP 410 Gone on individual issues / PRs
    • GitLab repo_info.commit_count is 0 but facade found commits
    • Release collection “not found” errors
    • Git clone exit status 128
    • Garbage timestamps (year 0001 BC)
    • Schema version mismatch: web, api and the scancode worker refuse to start
    • Null byte errors in text fields
    • Scancode produces no results / scanned-repo count barely advances (corrupt libmagic)
    • PostgreSQL restarts nightly — aveloxis.log floods with “database system is shutting down”
    • process stalled — the whole collector stops for seconds at a time
    • Brief “database unavailable” blips that aren’t real outages (connect/SASL-auth timeouts)
    • Restart procedure
    • Checking queue status
      • Via the dashboard
      • Via psql
      • Via the REST API
    • Checking status of jobs when you think things are stuck
      • Step 1 — count what’s actually claimable right now
      • Step 2 — understand the priority direction
      • Step 3 — understand the monitor’s display order
      • Step 4 — find the rows you’re worried about
      • Step 5 — check whether locked_by points at a dead worker
        • Compare to the currently-running scheduler
        • Check locked_at freshness
        • Check pg_stat_activity for live aveloxis backends
      • Step 6 — recover stuck locks
      • Step 7 — make specific repos top-of-queue
    • Changed days_until_recollect is being ignored
    • Checking collection status
    • Re-running a failed repo
    • GraphQL PR batch errors on large repos
    • Force-recollect a single repository
    • Dead repo sidelining and un-sidelining
      • How sidelining works
      • Un-sidelining a repo
      • List all sidelined repos
    • Gateway errors (502/503/504)
      • The same pulls/comments page fails every cycle on a large repository
    • Charts or pages time out (504 / 502 in the browser)
    • Deadlock errors
    • prepared statement "stmtcache_..." does not exist (SQLSTATE 26000)
    • Restart appears to take days before collection resumes
    • All API tokens exhausted within minutes of restart
    • Repeated duplicate key value violates unique constraint "contributors_pkey" in Postgres logs
    • Case-variant duplicate repositories
    • Orphaned postgres backend after aveloxis stop serve
    • serve crashed with fatal error: out of memory allocating heap arena metadata
    • A repository’s daily commit counts need rebuilding
    • A one-shot migration backfill needs to re-run
    • Next steps
    • Issues or PRs missing after a second collection round
    • GitLab-only deployment: “GitHub key pool has no usable key — GitHub-only background tasks stay idle”

Architecture

  • Architecture Overview
    • System diagram
    • Four schemas
      • aveloxis_data (103 tables + 22 materialized views)
      • aveloxis_scan (4 tables)
      • aveloxis_ops (42 tables)
      • aveloxis_augur_data (6 views)
    • Collection flow
    • Key design decisions vs Augur
      • Postgres queue instead of Celery/Redis/RabbitMQ
      • JSONB staging instead of direct writes
      • Deterministic contributor IDs
      • Bare clones instead of full clones
      • Built-in monitoring
      • Platform abstraction layer
        • Known GitLab API limitations
    • Project structure
    • Scheduler internals
      • Job dispatch
      • Phase execution within a job
      • Periodic background tasks
      • Graceful shutdown
      • Startup recovery
    • Next steps
  • Staged Pipeline Architecture
    • Why staging?
      • The contention problem
      • The staging solution
    • Staging writer
      • Batch flushing
      • Callers MUST call Flush(ctx) before handing control to the processor
      • The staging table
    • Envelope types
      • stagedIssue
      • stagedPR
      • Why envelopes?
    • Processing order
    • Error isolation
    • Restart and resume
      • On shutdown
      • On startup
      • Idempotent processing
    • Performance characteristics
      • Staging phase (Phase 1)
      • Processing phase (Phase 2)
      • Contributor cache
    • Comparison with the direct pipeline
    • Next steps
  • Contributor Resolution
    • What a contributor record represents
    • Where contributor data comes from
    • Contract rules
      • R1: Identity-key determinism
      • R2: Identity-key immutability
      • R3: cntrb_login partial uniqueness
      • R4: Identity rows are denormalized truth
      • R5: Email aliasing
      • R6: Enrichment is best-effort, cooldown-bounded
      • R7: Cached resolution on the hot path
      • R8: Bulk operations dedup before flushing
      • R9: Idempotent re-collection
      • R10: Foreign-key integrity
      • R11: Search-resolve never modifies identity
      • R12: Commit author resolution chain order
      • R13: Documentation as a first-class deliverable
    • Resolution flow
      • Layer 1: API-phase resolution (during collection)
      • Layer 2: Git-phase resolution (after facade)
      • Layer 3: Background tasks
      • Shared email→identity resolution
    • GithubUUID / PlatformUUID
    • Rename handling: which columns track renames, which don’t
      • Three tables, three rename behaviors
        • aveloxis_data.contributors (one row per person)
        • aveloxis_data.contributor_identities (one row per (platform_id, platform_user_id))
        • aveloxis_data.commits (one row per file per commit)
      • Operational consequences
      • Login history table (v0.23.0)
      • Why the two write paths exist (v0.22.12 vs v0.22.13)
    • Data-quality FAQ
      • Why do I see two rows for the same person?
      • Why is cntrb_canonical empty?
      • What does gh_user_id IS NULL mean?
      • Why is cntrb_id random instead of the deterministic UUID?
      • Why does cmt_author_platform_username differ from cntrb_login for the same author?
      • Why does my query show contributors with no commits?
    • Diagnostic queries
    • Intentional limitations
      • The rename edge case (largely closed by v0.20.2)
      • Random-UUID rows for email-only contributors
      • GitLab parity gaps
      • Cooldown windows are 30 days, not configurable
      • Search-resolve is GitHub-only, low-yield
      • The cache is per-job, not fleet-wide (today)
      • Activity classification is GitHub-only (v0.27.57)
    • GitLab vs GitHub: column-by-column parity matrix (v0.20.3)
      • Querying contributor data on mixed-platform fleets
      • Closable gaps that aren’t planned
    • Related code
    • Next steps
  • Human provenance — where identity lives, column by column
    • The four identity chains
    • Issues — the multi-provider entity
    • Messages — one body table, five origins
    • jira_identities — the banked usernames
    • contributors_aliases — the email bridge
    • Which heal advances what
    • Validated invariants (Part G, 2026-09-01)
  • Facade Commits
    • Bare clone vs full clone
      • Bare clones
      • Full clones (temporary)
      • Disk usage
    • Git log parsing
      • Empty repositories and git failures (v0.29.58)
      • Format string
      • Parsing logic
    • Per-file commit rows
      • Columns populated from git log
      • Upsert behavior
    • Commit parents
    • Commit messages
    • Affiliation resolution
      • Resolution logic
      • Populated columns
      • Adding affiliations
    • Daily commit counts
    • Facade aggregates
      • Aggregate tables
      • Aggregate columns
      • Refresh SQL
    • Resilience
      • Fetch failure recovery
      • Incremental collection
      • GitLab repo_info.commit_count backfill (v0.16.9+)
    • Next steps
  • Analysis
    • On-demand full clone design
      • Workflow
      • Key points
    • Dependency scanning
      • Supported ecosystems
      • Output
      • Multiple manifests
    • Libyear calculation
      • What is libyear?
      • Supported registries
      • Version cleaning
      • Output
      • Rate limiting
    • Code complexity via scc
      • Installation
      • Execution
      • Output
      • Snapshot rotation and history (v0.27.7)
      • If scc is not installed
      • Materialized view
    • Disk usage summary
    • Error handling
    • Next steps
  • ScanCode Worker (v0.21.0+)
    • 1. What scancode does (and doesn’t)
    • 2. Why the worker was decoupled (the 2026-05-14 incident)
    • 3. Architecture
      • 3.1 Components
      • 3.2 The lifecycle of a single scan
      • 3.3 Dispatcher pacing: a start gap per worker (v0.21.3, v0.29.70)
    • 4. Cadence rationale (180 days default)
    • 5. Crash recovery — the four-state table
    • 6. Graceful shutdown
    • 7. Force-rerun cookbook
    • 8. Configuration reference
    • 9. UX: the “last run” signal
    • 10. Observability — what to grep in aveloxis.log
    • 11. Code map
    • 12. Regression guards
    • 13. Startup health preflight + aveloxis_ops.aveloxis_status
      • The preflight
      • The status table
      • Per-repo failure capture is bounded (v0.25.28)
  • Vulnerability Scanning and SBOM Generation
    • The short answer
    • Where this runs
    • Vulnerability scanning
      • Data source: OSV.dev
      • Advisory sources reach us through OSV
      • Severity
      • Direct vs transitive
      • Lifecycle
      • Caching
      • The package-centred views
    • SBOM generation
      • Real dependency graphs (v0.27.134, C2)
      • Vulnerability-annotated SBOMs
    • What feeds the dependency graph
      • Manifest parsing — 14 ecosystems
      • Lockfile parsing — 18 formats
      • Dependency edges + parent-chain attribution (v0.27.133, C2)
      • Libyear registries — 12
    • External binaries
    • What is not part of this pipeline
    • Known limitations
      • Closed in v0.27.23
    • Configuration
    • Operator commands
    • API endpoints
    • Related documentation
  • Distribution Worker (v0.24.0+)
    • 1. The question this subsystem answers
    • 2. Why a dedicated worker (the v0.24.0 design call)
    • 3. The five evidence sources
      • 3.1 deps.dev (GET /v3/projects/github.com%2F{owner}%2F{repo}:packageversions)
      • 3.2 ecosyste.ms (GET /api/v1/packages/lookup?repository_url=...)
      • 3.3 GitHub release-asset extension heuristic (GET /repos/{o}/{r}/releases)
      • 3.4 GitHub Packages (GET /users/{o}/packages → fallback /orgs/{o}/packages)
      • 3.5 In-repo manifest walk (GET /repos/{o}/{r}/contents/)
    • 4. The headline analysis query
    • 5. Schema
      • 5.1 Repo-level columns
      • 5.2 The four distribution tables
      • 5.3 Rotation semantics
      • 5.4 The v0.25.0 distribution_scan_complete column
    • 6. Worker architecture
      • 6.1 Components
      • 6.2 The lifecycle of a single scan
      • 6.3 Per-call vs source-level circuit breakers (v0.25.0)
      • 6.4 Failure backoff (v0.21.4 pattern)
    • 7. GitHub-only for v1
    • 8. Operator CLI
    • 9. Config knobs
    • 10. What the subsystem does NOT do
    • 11. Cross-references
    • 12. v0.25.x escape hatches (ephemeral)
      • What’s in the group
      • Why they exist
      • Lifecycle and planned deprecation (target: 2027)
      • What operators on fresh installs should do
      • What operators upgrading through v0.25.x should do
  • Mailing-List Ingestion (v0.25.7+)
    • 1. The question this subsystem answers
      • 1.1 Two orthogonal axes
    • 2. Why a dedicated worker
    • 3. email_message — the first-class entity
    • 4. The two archive backends
      • 4.1 Apache Pony Mail (apache_ponymail)
      • 4.2 lore.kernel.org public-inbox (lore_public_inbox)
    • 5. Classification
    • 6. Sender and signaled-repo resolution
      • 6.1 Sender identity (§5d)
      • 6.2 Signaled repo — two columns, never block
    • 7. Schema (v0.25.7)
    • 8. Worker architecture
      • 8.1 Staging split (v0.25.x)
    • 9. Mirror handling
      • How a mirror message is linked (v0.28.20)
    • 10. Operator CLI
    • 11. Config knobs
    • 11c. Layer 2 projection — mailing-list → canonical entities (Phase 3)
    • 11d. Forge-less PR-equivalents — the special case (Phase C)
    • 12. Verification (Phase 4) and the collection-ordering caveat
    • 13. What the subsystem does NOT do
    • 14. Cross-references
  • Materialized Views
    • Refresh schedule
    • Quick reference
    • Bulk counters
      • api_get_all_repos_commits
      • api_get_all_repos_issues
      • api_get_all_repo_prs
    • Catalog and navigation
      • explorer_entry_list
      • explorer_user_repos
    • Contributor activity
      • explorer_contributor_actions (the parent view)
      • augur_new_contributors (VIEW alias for explorer_contributor_actions as of v0.25.6)
      • explorer_contributor_recent_actions
      • explorer_new_contributors
      • explorer_cntrb_per_file
      • issue_reporter_created_at
    • Commits over time
      • explorer_commits_and_committers_daily_count
    • Pull requests
      • explorer_pr_response_times
      • explorer_pr_response
      • explorer_pr_assignments
      • explorer_pr_files
    • Issues
      • explorer_issue_assignments
    • Files and code
      • explorer_repo_files
      • explorer_repo_languages
    • Dependencies
      • explorer_libyear_summary
      • explorer_libyear_all (alias view, v0.25.5+)
      • explorer_libyear_detail
    • Monitoring refresh progress
    • Column stability across releases
      • The contract
      • Example: v0.25.6 commit-branch rewrite of explorer_new_contributors
      • Example: v0.25.6 augur_new_contributors restoration
      • Verifying column stability
    • Troubleshooting
      • “could not refresh materialized view concurrently”
      • Slow refresh
      • Views out of date
    • Cross-references
    • Supply-chain package views (Aveloxis-owned)
      • What is new here, and what is only regrouped
      • explorer_package_exposure
      • explorer_package_advisory
      • Live cohorts, and a deployment without the views
      • Lifecycle
      • Dropping the 8Knot set on a deployment that does not use it
  • Column Name Mapping (Augur to Aveloxis)
    • Design Philosophy
    • Pull Requests
    • Pull Request Meta
    • Pull Request Reviews
    • Pull Request Labels
    • Pull Request Assignees / Reviewers
    • Issues
    • Issue Events
    • Issue Labels / Assignees
    • Messages
    • Repo Info
    • Repo Clones
    • Table Names
    • Schema Names
    • Libyear Compatibility Note
    • Writing Queries
    • pull_request_meta.meta_label — why it is sometimes empty
    • Known-empty columns (the 2026-08-19 fill-audit contract, v0.27.105)
  • Platform Abstraction Layer
    • Interface hierarchy
    • HTTP client (HTTPClient)
    • Key pool (KeyPool)
      • Key pool contract
    • GraphQL transport (the GitHub default)
      • GraphQL errors that arrive with HTTP 200
    • Pagination
      • Page-size step-down (v0.29.72)
    • URL parsing (RepoURL)
    • Adding a new platform
    • Design notes
    • GitHub vs GitLab data gaps
  • Database Package (internal/db)
    • Package layout
    • Deadlock retry
    • Batch operations
    • Text sanitization
    • Queue system
    • Group ownership verification

Contributing

  • Contributing to Aveloxis
    • What Aveloxis is
    • How the codebase is laid out
    • Chapters
      • Foundation (read before doing anything)
      • Extending Aveloxis
    • What to read for a given task
    • The single-most-important rule
  • Local development setup
    • Prerequisites
    • 1. Clone and build
    • 2. Run the unit test suite
    • 3. Set up a local PostgreSQL
      • macOS (Homebrew)
      • Linux (Debian/Ubuntu)
      • Verify
    • 4. Configure aveloxis.json
    • 5. Run the migration
    • 6. (Optional) Install scc + scorecard
    • 7. (Optional) Add API keys
    • 8. Run the binary
    • 9. Run integration tests
    • 10. Set up your editor
    • Common problems
      • relation "aveloxis_data.repos" does not exist
      • column "X" does not exist
      • failed to connect to ... server is starting up
      • no API keys configured for any platform
      • Tests pass locally but fail in CI
    • Next steps
  • Code conventions
    • SPDX headers (mandatory)
    • Embedded SPDX license data
    • Package layout
    • Imports
    • Naming
    • Error handling
      • The contract
      • Everything that errors is logged (mandatory)
      • Idioms
    • Logging (slog)
    • Version bumping (mandatory)
      • When to bump
      • When NOT to bump
    • Commit messages
    • The release notes are the public record
    • Don’t add features beyond what the task requires
    • Don’t write comments that restate the code
    • Don’t write to files in ways that surprise
    • CLI command style
    • What NOT to do
  • Testing
    • The three tiers
      • Tier 1 — Unit tests
      • Tier 2 — Integration tests
      • Tier 3 — data-test harness (cross-version regression detection)
    • Static analysis — the three blocking linters (v0.27.36)
    • The standing gate list (definition of done)
    • TDD discipline
    • The source-contract pattern
      • When to use a source-contract test
      • When NOT to use a source-contract test
      • The v0.21.1 lesson
      • Shared helpers: internal/srctest (v0.27.118)
      • SQL write scanning: internal/srctest/sqlscan (v0.27.124)
      • Column write-policy registry (v0.27.126)
      • Standing-rules registry (v0.27.127)
      • Convergence-contracts registry (v0.27.146 — SR-19 mechanized)
      • Analyzer graduation criteria (deliberately deferred)
      • SQL parsing: tested micro-parsers now; pg_query_go pilot flagged
    • Where expected values come from
      • REGRESSION-PIN labeling
      • The review question
    • The behavioral test pattern
    • The integration test pattern
    • What to test
      • Always
      • Often valuable
      • Skip
    • Naming
    • Run patterns
    • What good test failures look like
    • Writing tests for code that depends on time
    • Writing tests for goroutines
    • Writing tests for HTTP clients
    • CI
  • Review lenses — the adversarial pass every diff gets before it ships
    • L1 — Error arms on probes (SR-5 / SR-16, the most-recurrent class)
    • L2 — Key granularity (SR-17)
    • L3 — One shared normalizer (SR-17)
    • L4 — Who enforces the invariant? (SR-18)
    • L5 — Convergence (SR-19)
    • L6 — Check-then-act and lock lifecycles
    • L7 — Boundary equality
    • L8 — External authority vs local convention
    • L9 — Diagnostics tell the truth (SR-10’s logging half)
    • L10 — The fix itself is new code
    • L11 — Class sweep
    • L12 — Stale prose
    • L13 — Probe/tripwire parity
    • L14 — Keep-going passes and exits tell the truth
    • L15 — The forge’s cache and session semantics
    • L16 — Widening a tripwire is a contract change
    • L17 — What the PR #193 rounds kept finding (the four blind spots)
    • Running the pass
  • The fresh-context review pass — brief, protocol, checklist
    • When
    • The brief
    • The protocol on findings
    • Pre-launch checklist (what shortened the tail)
    • Where the results go
  • Schema migrations
    • The two sources of truth
    • The fail-closed contract (v0.19.4)
    • Adding a column
      • Source-contract test
      • Integration test (the v0.21.1 lesson)
    • Adding an index
    • Adding a backfill
      • Idempotency is mandatory
      • Source-contract test
      • Integration test
    • Adding a foreign key
    • Migration ordering
    • Materialized views
    • The aveloxis migrate --skip-views flag
    • Version-bump checklist for a schema-touching change
    • What v0.21.5 made explicit: who can run migrations
    • When NOT to write a migration
    • The migration ledger (v0.28.4)
  • Adding a new platform
    • What “a platform” means in Aveloxis
    • Bugzilla — the worked example
    • API choice: REST vs XML-RPC
    • The end-to-end inventory: what you’ll touch
      • Mandatory
      • Likely
      • Optional
    • The capability design call
    • Step-by-step walkthrough
      • Step 1 — declare the platform
      • Step 2 — seed the database
      • Step 3 — package skeleton
      • Step 4 — Client struct + interface satisfaction
      • Step 5 — auth style
      • Step 6 — URL parsing
      • Step 7 — implement the meaningful methods
        • FetchRepoInfo — Bugzilla product metadata
        • ListIssues — bugs as issues
        • ListIssuesAndPRs — the unified phase-2 enumerator
        • ListIssueComments — Bugzilla bug comments
        • ListContributors — Bugzilla users
        • SearchUserByEmail — Bugzilla user lookup
      • Step 8 — implement the no-op methods
      • Step 9 — wire into the scheduler
      • Step 10 — wire into main.go
      • Step 11 — config
      • Step 12 — tests
      • Step 13 — version bump, release-note entry
    • What you’ll discover during implementation
    • Don’ts
    • A reasonable order of operations
    • Final note
  • Adding a REST endpoint
    • Where the API lives
    • Existing endpoints
    • Walkthrough: add GET /api/v1/repos/{repoID}/contributors
      • Step 1 — write the failing test
      • Step 2 — add the store method
      • Step 3 — add the handler
      • Step 4 — register the route
      • Step 5 — run the tests
      • Step 6 — bump version, document
    • Patterns to follow
      • Path parameters via r.PathValue("name")
      • Query params via r.URL.Query().Get
      • Errors
      • Response shape
      • CORS
      • Caching
      • Logging
    • What NOT to do
    • Wiring to the web GUI
  • Adding a collection phase
    • The pipeline shape
    • When to add a phase vs extend an existing one
    • Patterns by phase type
      • “Per-repo, inline, runs every cycle”
      • “Per-repo, runs after facade (needs git clone)”
      • “Background, runs on a periodic ticker, fleet-wide”
      • “Decoupled worker pool, not in the main pipeline”
    • The staging writer contract
    • Error handling discipline
    • Testing phases
    • Order-of-operations checklist
    • A real example to study
  • Adding a visualization
    • How visualizations work today
    • Walkthrough: add a “contributor activity” chart
      • Step 1 — make sure the data is available
      • Step 2 — find the right template
      • Step 3 — add the canvas + the fetch
      • Step 4 — handle the data-shape mismatch
    • Patterns to follow
      • Color palette
      • Chart options
      • Comparison page differences
      • Time zones
      • Empty data
    • What NOT to do
    • Testing visualizations
    • Updating the comparison page
    • Documentation

Reference

  • Aveloxis Schema Documentation
    • Overview
    • Metadata Columns
    • aveloxis_data Schema
      • Core Tables
        • platforms
        • repo_groups
        • repos
        • repo_groups_list_serve
        • email_message
        • email_message_ref
      • Contributors
        • contributors
        • contributor_identities
        • contributors_aliases
        • jira_identities
        • contributor_affiliations
        • contributor_repo
        • unresolved_commit_emails
      • Issues
        • issues
        • issue_labels
        • issue_assignees
        • issue_events
        • issue_message_ref
      • Pull Requests
        • pull_requests
        • pull_request_labels
        • pull_request_assignees
        • pull_request_reviewers
        • pull_request_reviews
        • pull_request_meta
        • pull_request_commits
        • pull_request_files
        • pull_request_events
        • pull_request_repo
        • pull_request_review_message_ref
        • pull_request_teams
        • pull_request_analysis
      • Messages and Comments
        • messages
        • pull_request_message_ref
        • review_comments
      • Commits (Git/Facade)
        • commits
        • commit_parents
        • commit_messages
        • commit_comment_ref
      • Releases
        • releases
      • Repository Metadata
        • repo_info
        • repo_clones
        • repo_badging
        • dei_badging
        • repo_insights
        • repo_insights_records
        • repo_group_insights
      • Dependencies and SBOM
        • repo_dependencies
        • repo_deps_libyear
        • repo_deps_scorecard
        • repo_sbom_scans
      • Libraries
        • libraries
        • library_dependencies
        • library_version
      • Facade Aggregates (Data Mart)
        • dm_repo_annual
        • dm_repo_monthly
        • repo_commit_daily
        • dm_repo_weekly
        • dm_repo_group_annual
        • dm_repo_group_monthly
        • dm_repo_group_weekly
      • Code Complexity
        • repo_labor
        • repo_labor_history
        • repo_meta
        • repo_stats
        • repo_test_coverage
      • CHAOSS Metrics
        • chaoss_metric_status
        • chaoss_user
      • Analysis and ML (schema parity, not yet populated)
        • message_analysis
        • message_analysis_summary
        • message_sentiment
        • message_sentiment_summary
        • discourse_insights
        • lstm_anomaly_models
        • lstm_anomaly_results
      • Topic Modeling (schema parity, not yet populated)
        • topic_model_meta
        • topic_model_event
        • topic_words
        • repo_cluster_messages
        • repo_topic
      • Network Analysis (schema parity, not yet populated)
        • network_beyond_augur
        • network_beyond_augur_dependencies
      • Miscellaneous
        • exclude
        • historical_repo_urls
        • repos_fetch_log
        • settings
        • unknown_cache
        • utility_log
        • working_commits
    • aveloxis_ops Schema
      • Jira collection state (v0.29.0)
        • jira_project_serve
        • jira_staging
      • Subsystem health
        • aveloxis_status
      • Collection Pipeline
        • staging
        • collection_queue
        • collection_status
      • API Credentials
        • worker_oauth
      • Users and Auth
        • users
        • user_groups
        • user_repos
        • client_applications
        • user_session_tokens
        • refresh_tokens
        • subscription_types
        • subscriptions
      • Configuration
        • augur_settings
        • config
      • GitHub Users (Affiliation Data)
        • github_users
      • Network Weighted Tables
        • network_weighted_commits
        • network_weighted_issues
        • network_weighted_pr_reviews
        • network_weighted_prs
      • Worker Management
        • worker_history
        • worker_job
        • worker_settings_facade
      • Fetch Log and Working Commits (ops)
        • repos_fetch_log (aveloxis_ops)
        • working_commits (aveloxis_ops)
    • Indexes
      • aveloxis_data indexes
      • aveloxis_ops indexes
Aveloxis
  • Search

© Copyright 2026, Sean Goggins, University of Missouri, Derek Howard.

Built with Sphinx using a theme provided by Read the Docs.