# Vulnerability Scanning and SBOM Generation

How Aveloxis determines what a repository depends on, whether those
dependencies have known vulnerabilities, and how it emits a Software Bill of
Materials.

This page describes **what the system does today**, including its known
limitations. It is the reference to hand someone who asks "what tooling are you
using for vulnerabilities and SBOMs?"

---

## The short answer

Aveloxis does not wrap an existing scanner. There is no Syft, no Trivy, no
Grype, no Dependency-Track, and no vendored SBOM library. The pipeline is
first-party Go that reads dependency manifests directly, queries **OSV.dev** for
advisories, and serialises SBOMs by hand.

The dependency surface is deliberately small. `go.mod` declares six direct
requirements — `google/uuid`, `jackc/pgx/v5`, `spf13/cobra`, `golang.org/x/net`,
`golang.org/x/oauth2`, `gopkg.in/yaml.v3` — and **none of them are security or
SBOM libraries**. `google/uuid` is the only third-party package that touches the
SBOM path at all, and only to mint document identifiers.

Three external binaries are shelled out to (ScanCode, OpenSSF Scorecard, scc),
plus `git` and `curl`. Everything else is ours.

---

## Where this runs

Vulnerability scanning and SBOM generation are the last two phases of the
per-repository collection pipeline. They depend on earlier phases having
populated the dependency tables:

| Phase | Step | Produces |
|---|---|---|
| 3 | Facade — bare clone + `git log` | commit history, working tree for analysis |
| 4 | Analysis — manifest + lockfile parsing, libyear | `repo_dependencies`, `repo_deps_libyear`, `repo_lockfile_packages` |
| 4b | OpenSSF Scorecard | `repo_deps_scorecard` |
| — | ScanCode (decoupled worker, own cadence) | `aveloxis_scan.scancode_file_results` |
| 6 | **SBOM generation** | `repo_sbom_scans` (CycloneDX + SPDX) |
| 7 | **Vulnerability scan** | `repo_deps_vulnerabilities` |

Analysis must run after facade because it needs the clone. SBOM and vulnerability
scanning both read from the dependency tables, so they run last.

---

## Vulnerability scanning

### Data source: OSV.dev

**OSV.dev is the only vulnerability data source.** No other advisory feed is
queried.

| | |
|---|---|
| Batch query | `POST https://api.osv.dev/v1/querybatch` |
| Detail fetch | `GET https://api.osv.dev/v1/vulns/{id}` |
| Query format | Package URL (purl) — e.g. `pkg:npm/lodash@4.17.20` |
| Authentication | None. OSV.dev is a free public service. |
| Stored `source` value | `osv.dev` |

The scan is deliberately **two-phase**. `querybatch` returns only ID stubs
(`{id, modified}`) — it does *not* return severity, summary, affected ranges,
aliases, or references. Those come from a second pass of per-ID `GET
/v1/vulns/{id}` calls, run with a concurrency limit of 6.

This matters historically: the original implementation parsed detail fields out
of the `querybatch` response, which meant every field except `vuln_id` was empty
on every row ever stored. Fixed in v0.27.4; existing rows self-heal on each
repository's next scan.

### Advisory sources reach us *through* OSV

OSV.dev is an aggregator. The advisories that surface in
`repo_deps_vulnerabilities` originate from:

- **NVD** — CVE identifiers
- **GitHub Advisory Database** — `GHSA-*`
- **PyPI Advisory Database** — `PYSEC-*`
- **RustSec** — `RUSTSEC-*`
- **Go Vulnerability Database** — `GO-*`

CVE identifiers are extracted from OSV's `aliases` array, so a GHSA record with
a CVE alias yields both.

We do **not** query NIST NVD directly. Mapping a purl to a CPE — NVD's
identifier scheme — is unreliable, and a bad mapping produces false positives
that are worse than the coverage gained. OSV's per-ecosystem version-range
matching is authoritative in a way CPE matching is not.

### Severity

Severity **labels** (`CRITICAL` / `HIGH` / `MEDIUM` / `LOW`) are read from OSV's
`database_specific.severity` where present — GHSA always populates it — with
`MODERATE` folded to `MEDIUM`. These are trustworthy.

Severity **scores** are computed from the advisory's CVSS vector with the real
CVSS v3.0/v3.1 base-score formula (`internal/collector/cvss.go`, verified
against published specification vectors; CVSS v2 is implemented for old
records). A vector we cannot score — CVSS v4-only, or malformed — yields **no
score** (stored as 0), never a guessed value. Scores stored before v0.27.23
came from a six-bucket approximation; they heal on each repository's next scan
or via `aveloxis heal-vulnerabilities --rescore-only`. See
[Known limitations](#known-limitations).

### Direct vs transitive

Direct dependencies — those declared in a manifest — are always scanned.

Transitive dependencies (the full lockfile closure) are scanned only when
`collection.vuln_scan_transitive` is enabled. It defaults to **off**. When on,
every finding carries `dependency_kind` of `direct` or `transitive`, and the API
splits counts so a transitive-heavy total can never headline as though it were
direct exposure.

Findings also carry `version_resolution`, recording how confidently we know the
installed version: `locked` (from a lockfile), `exact` (pinned `==`),
`bounded-range`, `range-floor` (`>=X`, floor scanned), or `unpinned`. A `>=`
requirement is **not** resolved to the latest satisfying version — actual
resolution depends on the whole dependency graph, so the floor is scanned and
labelled as such.

### Lifecycle

Findings are never deleted. Each carries `first_detected_at`, `last_seen_at`,
and `resolved_at`. After a complete successful scan, findings absent from the
current result set are stamped `resolved_at` rather than removed, preserving the
historical record. A reappearing vulnerability clears `resolved_at` and becomes
current again. Dashboards count unresolved findings only.

### Caching

An in-process cache (`osv_cache.go`) holds purl → vulnerability-ID results and
ID → detail records with a 24-hour TTL. Negative results are cached
deliberately — most packages have no advisories, and caching the absence is
where most of the traffic reduction comes from. Failed fetches are never cached.

This is politeness infrastructure for a free public service, not just a
performance optimisation.

---

## SBOM generation

Both formats are generated per repository and stored in `repo_sbom_scans`. Generation
failure is non-fatal to the collection job.

| Format | Spec version |
|---|---|
| CycloneDX | **1.5** |
| SPDX | **2.3** |

Both are **hand-written Go structs serialised with `encoding/json`**. No
CycloneDX or SPDX library is vendored.

**CycloneDX 1.5 specifics.** Uses the 1.5 `metadata.tools` object form rather
than the deprecated bare array. Development dependencies are marked `scope:
"excluded"`, runtime dependencies `"required"`. ScanCode-detected licenses and
copyrights are placed in `evidence.licenses` and `evidence.copyright`, kept
deliberately separate from registry-*declared* licenses so consumers can tell
asserted metadata from observed evidence.

**SPDX 2.3 specifics.** Package identifiers are derived from
`sha256(name@version)`, so they are stable across regenerations. Package URLs
are emitted in `externalRefs` with `referenceCategory: PACKAGE-MANAGER` and
`referenceType: purl`. Per-dependency `licenseConcluded` is always
`NOASSERTION` — ScanCode runs at repository root, not per dependency, so we have
no basis to conclude a license for an individual package.

The `documentNamespace` is an `https://aveloxis.io/spdx/...` URI. Per the SPDX
specification this is an identifier, not a resolvable endpoint; nothing is
served there.

### Vulnerability-annotated SBOMs

Requesting the CycloneDX SBOM with `?vulns=1` appends a native CycloneDX 1.5
`vulnerabilities` array covering unresolved findings, with `affects.ref` pointing
at the component's purl. SPDX has no equivalent native representation, so the
parameter returns 400 for that format.

---

## What feeds the dependency graph

### Manifest parsing — 14 ecosystems

Parsers are hand-written (regex, JSON, line scanning). There is no TOML or XML
library — `pom.xml` and `Directory.Packages.props` are parsed by hand.

| Ecosystem | Manifests |
|---|---|
| JavaScript | `package.json`, `yarn.lock` |
| Python | `requirements.txt`, `setup.py`, `setup.cfg`, `pyproject.toml`, `Pipfile`, `poetry.lock` |
| Go | `go.mod`, `go.sum` |
| Rust | `Cargo.toml`, `Cargo.lock` |
| Ruby | `Gemfile`, `Gemfile.lock` |
| Java / Kotlin | `pom.xml`, `build.gradle`, `build.gradle.kts` |
| PHP | `composer.json` |
| Elixir | `mix.exs` |
| Swift | `Package.swift` |
| Dart | `pubspec.yaml` |
| Scala | `build.sbt` |
| .NET | `packages.config`, `Directory.Packages.props` |
| Haskell | `package.yaml`, `stack.yaml` |
| C / C++ | `Makefile`, `CMakeLists.txt` |

C/C++ manifests are recognised for **language detection only** — there is no
parse implementation, so they yield no dependency rows. There is no widely
adopted C/C++ dependency manifest to target.

### Lockfile parsing — 18 formats

Lockfiles are the disambiguator: they turn "some version satisfying `>=2.0`"
into "2.5.0 is installed", which is what makes a vulnerability match meaningful.

| Ecosystem | Lockfiles |
|---|---|
| npm | `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`, `bun.lock`, `bun.lockb` |
| PyPI | `poetry.lock`, `Pipfile.lock`, `uv.lock`, `pdm.lock` |
| Cargo | `Cargo.lock` |
| RubyGems | `Gemfile.lock` |
| Packagist | `composer.lock` |
| Hex | `mix.lock` |
| Pub | `pubspec.lock` |
| SwiftPM | `Package.resolved` |
| NuGet | `packages.lock.json` |
| Maven | `gradle.lockfile` |
| Hackage | `stack.yaml.lock` |

`bun.lockb` is binary and is detected but not parsed. Parsing is best-effort
throughout: a malformed lockfile logs a warning and is skipped, never failing
the analysis run.

Two deliberate exclusions: **`requirements.txt` is never treated as a
lockfile**, even when fully hash-pinned, because it typically carries the same
version ambiguity as any other manifest. And Go needs no lockfile — `go.mod`
under Minimal Version Selection is already exact.

### Libyear registries — 12

Dependency freshness ("how many years behind is this dependency?") is resolved
against upstream registries. All are queried anonymously with a single
identifying User-Agent (`aveloxis/{version}`).

npm · PyPI · Go module proxy · crates.io · RubyGems · Maven Central ·
Packagist · Hex.pm · NuGet · pub.dev · SwiftPM (resolved via GitHub releases,
as Swift has no central registry) · Hackage

The User-Agent is load-bearing, not courtesy: crates.io returns HTTP 403 to
default client user-agents under its crawler policy. That silently zeroed every
Rust dependency until v0.27.19.

---

## External binaries

| Binary | Project | Installed by | Purpose |
|---|---|---|---|
| **ScanCode Toolkit** (mini) | nexB / AboutCode | `aveloxis install-tools` (pipx) | per-file license + copyright detection → SBOM evidence fields |
| **OpenSSF Scorecard** | `ossf/scorecard` | `aveloxis install-tools` (GitHub release tarball) | supply-chain security posture → `repo_deps_scorecard` |
| **scc** | `boyter/scc` | `aveloxis install-tools` (`go install`) | code size and complexity → `repo_labor` |
| `git` | — | system prerequisite | cloning, log walking, tree listing |
| `curl` | — | system prerequisite | libyear registry lookups |

ScanCode is optional — if it is not installed, scanning is skipped and SBOMs are
emitted without evidence fields.

---

## What is *not* part of this pipeline

**deps.dev and ecosyste.ms** are queried by Aveloxis, but by the *distribution
tracking* subsystem, which answers "where is this repository published?" — not
"what does it depend on?" and not "is it vulnerable?" Nothing from either
service flows into `repo_deps_vulnerabilities` or into purl construction.
See [Distribution tracking](distribution.md).

**OpenCVE** appears as `app.opencve.io` links in the web interface. These are
constructed URLs for human navigation. OpenCVE is never called as an API.

---

## Known limitations

Documented deliberately so consumers can calibrate. Remediation history is in
`summary/16-vuln-sbom-gap-closure-plan.md`; several original findings were
closed in v0.27.23 and are recorded below so readers of older SBOMs and
findings know what changed.

1. **Transitive scanning is off by default.** Vulnerability counts reflect
   directly declared dependencies unless `vuln_scan_transitive` is enabled.

2. **Per-dependency licenses are `NOASSERTION` in SPDX.** ScanCode runs at
   repository root only.

3. **C/C++ yields no dependency rows.** Recognised for language detection only.

4. **Single advisory source.** An OSV.dev outage means no vulnerability data for
   its duration. There is no secondary feed and no fallback.

5. **CVSS v4 vectors are not scored.** v4 scoring is a MacroVector lookup, not
   a formula; a v4-only advisory keeps its vector for display but gets no
   numeric score. Its severity label still arrives from the advisory database.

6. **External tool versions are unpinned** (installed at `@latest`,
   auto-updated monthly). Since v0.27.23 each SBOM *records* the ScanCode
   version that produced its evidence, so documents are attributable — but two
   runs months apart may still legitimately differ.

### Closed in v0.27.23

- **CVSS scores are now computed, not approximated.** Scores before v0.27.23
  came from a six-value substring lookup (and were not even monotonic in
  severity). They are now computed with the CVSS v3.0/v3.1 base-score formula
  (v2 for old records), verified against published specification vectors. An
  unscoreable vector yields *no* score, never a guess. Stored rows heal on each
  repository's next scan, or fleet-wide in one pass via
  `aveloxis heal-vulnerabilities --rescore-only` (no OSV traffic).
- **The SPDX license identifier list is now the official one** (733
  identifiers, embedded at build time) instead of a hand-maintained ~70-entry
  allowlist. Valid identifiers that were previously demoted to `license.name`
  now emit as `license.id`. Refresh procedure lives in the header of
  `internal/collector/spdx_license_ids.txt`.
- **The OSV client is hardened**: 30-second timeout (was unbounded) and an
  identifying `aveloxis/<version>` User-Agent (was anonymous).

---

## Configuration

| Key | Default | Effect |
|---|---|---|
| `collection.vuln_scan_transitive` | `false` | Scan the full lockfile closure, not just declared dependencies |
| `mail.vuln_digest_include_transitive` | `false` | Include transitive findings in operator digest email |
| `mail.vuln_digest_min_severity` | `HIGH` | Severity floor for the operator digest |
| `mail.vuln_digest_interval_hours` | `24` | Digest cadence |

## Operator commands

```bash
aveloxis install-tools        # install scc, scorecard, scancode
aveloxis sbom <repo-id>       # generate SBOMs for one repository
aveloxis heal-vulnerabilities # re-fetch detail for findings stored before v0.27.4
aveloxis heal-vulnerabilities --rescore-only
                              # recompute cvss_score from stored vectors
                              # (no OSV traffic) — heals pre-v0.27.23 scores
```

## API endpoints

```
GET /api/v1/repos/{id}/vulnerabilities        # findings, most critical first
GET /api/v1/repos/{id}/sbom?format=cyclonedx  # CycloneDX 1.5
GET /api/v1/repos/{id}/sbom?format=spdx       # SPDX 2.3
GET /api/v1/repos/{id}/sbom?format=cyclonedx&vulns=1  # with vulnerabilities array
GET /api/v1/repos/{id}/licenses               # dependency license breakdown
GET /api/v1/repos/{id}/scorecard              # OpenSSF Scorecard results
```

---

## Related documentation

- [Dependency analysis and libyear](analysis.md)
- [ScanCode worker](scancode.md)
- [Distribution tracking](distribution.md)
- [Collection pipeline](../guide/collection-pipeline.md)