Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .github/workflows/rust.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,6 @@ jobs:
# Test comprehensive feature combinations to ensure optional dependencies work correctly
# Each module has its own feature flag that gates both the module and its dependencies

- name: Run cargo-readme check
run: cargo install cargo-readme && cargo readme > TMP_README.md && diff -b TMP_README.md README.md

- name: Build with no features
run: cargo build --no-default-features

Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,6 @@ as2org.jsonl

.env
.env.*

# Claude Code
CLAUDE.md
14 changes: 13 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,18 @@

All notable changes to this project will be documented in this file.

## Unreleased

### Bug fixes

* Fixed typo in `RpkiViewsCollector` enum variant: `SoborostNet` renamed to `SobornostNet`

### Documentation

* Rewrote `README.md` independently from `lib.rs` docstrings; README now includes a Mermaid architecture diagram and module summary table
* Removed `cargo-readme` workflow step and `readme.tpl` template
* Added `CLAUDE.md` to `.gitignore`

## v0.10.2 - 2026-02-25

### Bug fixes
Expand Down Expand Up @@ -33,7 +45,7 @@ All notable changes to this project will be documented in this file.
### RPKIviews Historical Data Support

* **Added RPKIviews as a historical RPKI data source**: Users can now load historical RPKI data from RPKIviews collectors in addition to RIPE NCC archives
- New `RpkiViewsCollector` enum with four collectors: SoborostNet (default), MassarsNet, AttnJp, and KerfuffleNet
- New `RpkiViewsCollector` enum with four collectors: SobornostNet (default), MassarsNet, AttnJp, and KerfuffleNet
- Added `RpkiTrie::from_rpkiviews(collector, date)` method for loading from a specific collector
- Added `RpkiTrie::from_rpkiviews_file(url, date)` and `from_rpkiviews_files(urls, date)` for loading from specific archive URLs
- Added `list_rpkiviews_files(collector, date)` function to discover available archives for a given date
Expand Down
222 changes: 82 additions & 140 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,131 +1,89 @@
# BGPKIT Commons

*This readme is generated from the library's doc comments using [cargo-readme](https://github.com/livioribeiro/cargo-readme). Please refer to the Rust docs website for the full documentation*

[![Crates.io](https://img.shields.io/crates/v/bgpkit-commons)](https://crates.io/crates/bgpkit-commons)
[![Docs.rs](https://docs.rs/bgpkit-commons/badge.svg)](https://docs.rs/bgpkit-commons)
[![License](https://img.shields.io/crates/l/bgpkit-commons)](https://raw.githubusercontent.com/bgpkit/bgpkit-commons/main/LICENSE)
[![Discord](https://img.shields.io/discord/919618842613927977?label=Discord&style=plastic)](https://discord.gg/XDaAtZsz6b)

## Overview

BGPKIT-Commons is a library for common BGP-related data and functions with a lazy-loading architecture.
Each module can be independently enabled via feature flags, allowing for minimal builds.

### Available Modules

#### [`asinfo`] - Autonomous System Information (requires `asinfo` feature)
**Load Methods**: `load_asinfo(as2org, population, hegemony, peeringdb)`, `load_asinfo_cached()`, or `load_asinfo_with(builder)`
**Access Methods**: `asinfo_get(asn)`, `asinfo_all()`, `asinfo_are_siblings(asn1, asn2)`
**Data Sources**: RIPE NCC, CAIDA as2org, APNIC population, IIJ IHR hegemony, PeeringDB
**Functionality**: AS name resolution, country mapping, organization data, population statistics, hegemony scores, sibling detection

**New in v0.10.0**: The `as2org-rs` and `peeringdb-rs` crates have been consolidated into this module.
Use [`AsInfoBuilder`](asinfo::AsInfoBuilder) for ergonomic configuration of data sources.

#### [`as2rel`] - AS Relationship Data (requires `as2rel` feature)
**Load Method**: `load_as2rel()`
**Access Methods**: `as2rel_lookup(asn1, asn2)`
**Data Sources**: BGPKIT AS relationship inference
**Functionality**: Provider-customer, peer-to-peer, and sibling relationships between ASes

#### [`bogons`] - Bogon Detection (requires `bogons` feature)
**Load Method**: `load_bogons()`
**Access Methods**: `bogons_match(input)`, `bogons_match_prefix(prefix)`, `bogons_match_asn(asn)`, `get_bogon_prefixes()`, `get_bogon_asns()`
**Data Sources**: IANA special registries (IPv4, IPv6, ASN)
**Functionality**: Detect invalid/reserved IP prefixes and ASNs that shouldn't appear in routing

#### [`countries`] - Country Information (requires `countries` feature)
**Load Method**: `load_countries()`
**Access Methods**: `country_by_code(code)`, `country_by_code3(code)`, `country_by_name(name)`, `country_all()`
**Data Sources**: GeoNames geographical database
**Functionality**: ISO country code to name mapping and geographical information
`bgpkit-commons` is a library for common BGP-related data and functions. It provides a unified
interface to multiple BGP data sources through a lazy-loading architecture — modules are
independently enabled via feature flags and data is only fetched when explicitly requested.

## Architecture

```mermaid
graph TD
B[BgpkitCommons]

B -->|load_asinfo| M1[asinfo]
B -->|load_as2rel| M2[as2rel]
B -->|load_bogons| M3[bogons]
B -->|load_countries| M4[countries]
B -->|load_mrt_collectors| M5[mrt_collectors]
B -->|load_rpki| M6[rpki]

M1 -->|asinfo_get, asinfo_all| A1[RIPE NCC / CAIDA / APNIC / PeeringDB]
M2 -->|as2rel_lookup| A2[BGPKIT inference]
M3 -->|bogons_match| A3[IANA registries]
M4 -->|country_by_code| A4[GeoNames]
M5 -->|mrt_collectors_all| A5[RouteViews / RIPE RIS]
M6 -->|rpki_validate| A6[Cloudflare / RIPE NCC / RPKIviews]
```

#### [`mrt_collectors`] - MRT Collector Metadata (requires `mrt_collectors` feature)
**Load Methods**: `load_mrt_collectors()`, `load_mrt_collector_peers()`
**Access Methods**: `mrt_collectors_all()`, `mrt_collectors_by_name(name)`, `mrt_collectors_by_country(country)`, `mrt_collector_peers_all()`, `mrt_collector_peers_full_feed()`
**Data Sources**: RouteViews and RIPE RIS official APIs
**Functionality**: BGP collector information, peer details, full-feed vs partial-feed classification
Each module is gated by a feature flag. The `all` feature (default) enables everything.
Data is fetched on the first `load_xxx()` call and kept in memory until `reload()` is called.

#### [`rpki`] - RPKI Validation (requires `rpki` feature)
**Load Methods**: `load_rpki(optional_date)`, `load_rpki_historical(date, source)`, `load_rpki_from_files(urls, source, date)`
**Access Methods**: `rpki_validate(asn, prefix)`, `rpki_validate_check_expiry(asn, prefix, timestamp)`, `rpki_lookup_by_prefix(prefix)`
**Data Sources**: Cloudflare real-time, RIPE NCC historical, RPKIviews historical
**Functionality**: Route Origin Authorization (ROA) and ASPA validation, supports multiple data sources
## Modules

**New in v0.10.0**: Added RPKIviews as a historical RPKI data source with multiple collectors.
New public types [`Roa`](rpki::Roa) and [`Aspa`](rpki::Aspa) provide stable API for RPKI objects.
| Module | Feature | Data Sources | Key Functions |
|--------|---------|--------------|---------------|
| [`asinfo`] | `asinfo` | RIPE NCC, CAIDA as2org, APNIC, IIJ IHR, PeeringDB | `asinfo_get`, `asinfo_all`, `asinfo_are_siblings` |
| [`as2rel`] | `as2rel` | BGPKIT AS relationship inference | `as2rel_lookup` |
| [`bogons`] | `bogons` | IANA special registries | `bogons_match`, `bogons_match_prefix`, `bogons_match_asn` |
| [`countries`] | `countries` | GeoNames | `country_by_code`, `country_by_code3`, `country_by_name` |
| [`mrt_collectors`] | `mrt_collectors` | RouteViews, RIPE RIS | `mrt_collectors_all`, `mrt_collector_peers_all` |
| [`rpki`] | `rpki` | Cloudflare, RIPE NCC, RPKIviews | `rpki_validate`, `rpki_validate_check_expiry`, `rpki_lookup_by_prefix` |

### Quick Start
## Quick Start

Add `bgpkit-commons` to your `Cargo.toml`:
```toml
[dependencies]
bgpkit-commons = "0.10"
```

#### Basic Usage Pattern

All modules follow the same lazy-loading pattern:
1. Create a mutable `BgpkitCommons` instance
2. Load the data you need by calling `load_xxx()` methods
3. Access the data using the corresponding `xxx_yyy()` methods

```rust
use bgpkit_commons::BgpkitCommons;

let mut commons = BgpkitCommons::new();

// Load bogon data
// Load the modules you need
commons.load_bogons().unwrap();
commons.load_asinfo(false, false, false, false).unwrap();

// Use the data
// Access the data
if let Ok(is_bogon) = commons.bogons_match("23456") {
println!("ASN 23456 is a bogon: {}", is_bogon);
println!("ASN 23456 is bogon: {}", is_bogon);
}
```

#### Working with Multiple Modules

```rust
use bgpkit_commons::BgpkitCommons;

let mut commons = BgpkitCommons::new();

// Load multiple data sources
commons.load_asinfo(false, false, false, false).unwrap();
commons.load_countries().unwrap();

// Use the data together
if let Ok(Some(asinfo)) = commons.asinfo_get(13335) {
println!("AS13335: {} ({})", asinfo.name, asinfo.country);
if let Ok(Some(info)) = commons.asinfo_get(13335) {
println!("AS13335: {} ({})", info.name, info.country);
}
```

#### Using the AsInfoBuilder (New in v0.10.0)
## Examples

The builder pattern provides a clearer API for loading AS information:
### RPKI Validation

```rust
use bgpkit_commons::BgpkitCommons;

let mut commons = BgpkitCommons::new();
commons.load_rpki(None).unwrap(); // None = real-time from Cloudflare

// Clear, self-documenting configuration
let builder = commons.asinfo_builder()
.with_as2org()
.with_peeringdb();
commons.load_asinfo_with(builder).unwrap();

// Check if two ASes are siblings (requires as2org data)
if let Ok(are_siblings) = commons.asinfo_are_siblings(13335, 132892) {
println!("AS13335 and AS132892 are siblings: {}", are_siblings);
}
let result = commons.rpki_validate(13335, "1.1.1.0/24").unwrap();
println!("Validation result: {:?}", result);
```

#### Loading Historical RPKI Data (New in v0.10.0)

Load RPKI data from multiple historical sources:
### Historical RPKI Data

```rust
use bgpkit_commons::BgpkitCommons;
Expand All @@ -135,79 +93,63 @@ use chrono::NaiveDate;
let mut commons = BgpkitCommons::new();
let date = NaiveDate::from_ymd_opt(2024, 1, 4).unwrap();

// Load from RIPE NCC historical archives
// From RIPE NCC historical archives
commons.load_rpki_historical(date, HistoricalRpkiSource::Ripe).unwrap();

// Or load from RPKIviews collectors
let source = HistoricalRpkiSource::RpkiViews(RpkiViewsCollector::KerfuffleNet);
// Or from an RPKIviews collector
let source = HistoricalRpkiSource::RpkiViews(RpkiViewsCollector::SobornostNet);
commons.load_rpki_historical(date, source).unwrap();

// List available files for a date
let files = commons.list_rpki_files(date, HistoricalRpkiSource::Ripe).unwrap();
```

### Feature Flags
Available RPKIviews collectors: `SobornostNet` (default), `MassarsNet`, `AttnJp`, `KerfuffleNet`.

#### Module Features
- `asinfo` - AS information with organization and population data (includes integrated as2org and peeringdb)
- `as2rel` - AS relationship data
- `bogons` - Bogon prefix and ASN detection
- `countries` - Country information lookup
- `mrt_collectors` - MRT collector metadata
- `rpki` - RPKI validation functionality (ROA and ASPA)
### AS Information with Builder

#### Convenience Features
- `all` (default) - Enables all modules for backwards compatibility
```rust
use bgpkit_commons::BgpkitCommons;

#### Minimal Build Example
```toml
[dependencies]
bgpkit-commons = { version = "0.10", default-features = false, features = ["bogons", "countries"] }
let mut commons = BgpkitCommons::new();
let builder = commons.asinfo_builder()
.with_as2org()
.with_peeringdb();
commons.load_asinfo_with(builder).unwrap();

if let Ok(are_siblings) = commons.asinfo_are_siblings(13335, 132892) {
println!("AS13335 and AS132892 are siblings: {}", are_siblings);
}
```

### Direct Module Access

All modules support both central access via `BgpkitCommons` and direct module access:
All modules can be used directly without `BgpkitCommons`:

```rust
// Via BgpkitCommons (recommended for most use cases)
use bgpkit_commons::BgpkitCommons;
let mut commons = BgpkitCommons::new();
commons.load_bogons().unwrap();

// Direct module access (useful for standalone usage)
use bgpkit_commons::bogons::Bogons;
use bgpkit_commons::rpki::RpkiTrie;

let bogons = Bogons::new().unwrap();
let trie = RpkiTrie::from_cloudflare().unwrap();
```

### Error Handling

All access methods return `Result<T>` and will return an error if the corresponding module
hasn't been loaded yet or if there are data validation issues. Error messages include guidance
on which `load_xxx()` method to call. Always call the appropriate `load_xxx()` method before accessing data.

### Data Persistence and Reloading
## Feature Flags

All loaded data is kept in memory for fast access. Use the `reload()` method to refresh
all currently loaded data sources:
| Feature | Description |
|---------|-------------|
| `asinfo` | AS information: names, countries, organizations, population, hegemony |
| `as2rel` | AS relationship data |
| `bogons` | Bogon prefix and ASN detection |
| `countries` | Country information lookup |
| `mrt_collectors` | MRT collector metadata |
| `rpki` | RPKI validation (ROA and ASPA) |
| `all` *(default)* | Enables all modules |

```rust
let mut commons = BgpkitCommons::new();
commons.load_bogons().unwrap();
For a minimal build:

// Later, reload all loaded data
commons.reload().unwrap();
```toml
[dependencies]
bgpkit-commons = { version = "0.10", default-features = false, features = ["bogons", "rpki"] }
```

### What's New in v0.10.0

- **RPKIviews Historical Data**: Load historical RPKI data from RPKIviews collectors (SoborostNet, MassarsNet, AttnJp, KerfuffleNet) in addition to RIPE NCC archives
- **Crate Consolidation**: `as2org-rs` and `peeringdb-rs` functionality integrated directly into the `asinfo` module
- **AsInfoBuilder**: New builder pattern for ergonomic configuration of AS information data sources
- **Public RPKI Types**: New stable [`Roa`](rpki::Roa) and [`Aspa`](rpki::Aspa) structs for RPKI objects
- **Streaming Optimization**: RPKIviews archives are streamed efficiently without downloading entire files
- **RIPE Historical JSON**: RIPE historical data now uses JSON format for richer data including expiry timestamps

## License

MIT
14 changes: 0 additions & 14 deletions README.tpl

This file was deleted.

Loading
Loading