vault_audit_tools/lib.rs
1//! # Vault Audit Tools
2//!
3//! High-performance command-line tools for analyzing `HashiCorp Vault` audit logs with
4//! automatic parallel processing and compressed file support.
5//!
6//! ## Overview
7//!
8//! This crate provides a suite of specialized tools for parsing and analyzing
9//! `HashiCorp Vault` audit logs. It's designed to handle large production logs
10//! (multi-gigabyte files) efficiently through streaming parsing, parallel processing,
11//! and minimal memory overhead.
12//!
13//! ## Performance
14//!
15//! - **3x faster** than equivalent Python implementations (single file)
16//! - **40% faster** with automatic parallel processing (multi-file workloads)
17//! - **10x less memory** usage through streaming parser
18//! - Processes 4M line logs in ~17 seconds (vs ~60s in Python)
19//! - Near-linear CPU scaling with available cores
20//!
21//! ### Real-world Benchmarks
22//!
23//! - KV Analysis: 141s -> 85s (40.1% faster with parallel processing)
24//! - Memory usage: ~77 MB with parallel workers (2x overhead)
25//! - Throughput: 233 MB/s (vs 140 MB/s sequential)
26//!
27//! ## Features
28//!
29//! - **Parallel Processing** - Automatically processes multiple files concurrently
30//! - **18 specialized analysis commands** for different use cases
31//! - **Compressed File Support** - Direct analysis of `.gz` and `.zst` files
32//! - **Streaming JSON parser** for memory-efficient processing
33//! - **Entity lifecycle tracking** across multiple days
34//! - **Token usage analysis** and abuse detection
35//! - **KV secrets engine analysis** (v1 and v2)
36//! - **Vault mount enumeration** (KV and auth mounts)
37//! - **Kubernetes auth analysis**
38//! - **Shell completion** for bash, zsh, fish, powershell, and elvish
39//!
40//! ## Architecture
41//!
42//! The crate is organized into several key modules:
43//!
44//! - [`audit`] - Core audit log parsing and data structures
45//! - [`commands`] - Individual analysis command implementations
46//! - [`utils`] - Shared utilities (parallel processing, progress, time parsing)
47//! - [`vault_api`] - Vault API client for entity enrichment
48//!
49//! ## Example Usage
50//!
51//! ```bash
52//! # System overview with automatic parallel processing
53//! vault-audit system-overview logs/*.log
54//!
55//! # Entity analysis (unified command with auto-preprocessing)
56//! vault-audit entity-analysis churn day1.log day2.log day3.log
57//! vault-audit entity-analysis gaps audit.log
58//!
59//! # Token analysis with abuse detection
60//! vault-audit token-analysis audit.log --abuse-threshold 5000
61//!
62//! # KV secrets analysis (40% faster with parallel processing)
63//! vault-audit kv-analysis analyze logs/*.log --output kv_usage.csv
64//!
65//! # Enumerate Vault mounts (requires VAULT_ADDR and VAULT_TOKEN)
66//! vault-audit kv-mounts
67//! vault-audit auth-mounts
68//!
69//! # Compressed files work seamlessly
70//! vault-audit path-hotspots audit.log.gz
71//! ```
72//!
73//! ## Command Categories
74//!
75//! ### System Analysis (Parallel Processing)
76//! - `system-overview` - High-level audit log statistics
77//! - `path-hotspots` - Identify most accessed paths
78//!
79//! ### Entity Analysis (Unified Commands)
80//! - `entity-analysis churn` - Multi-day entity lifecycle tracking
81//! - `entity-analysis creation` - Track when entities first appear
82//! - `entity-analysis gaps` - Find gaps in entity activity (parallel)
83//! - `entity-analysis timeline` - Individual entity activity timeline
84//! - `entity-analysis preprocess` - Extract entity mappings
85//!
86//! ### Token Analysis (Unified Command, Parallel Processing)
87//! - `token-analysis` - Token lifecycle operations with abuse detection
88//!
89//! ### KV Secrets Analysis (Unified Commands, Parallel Processing)
90//! - `kv-analysis analyze` - Analyze KV secret access patterns
91//! - `kv-analysis compare` - Compare KV usage across time periods
92//! - `kv-analysis summary` - Summarize KV usage by mount point
93//!
94//! ### Authentication Analysis (Parallel Processing)
95//! - `k8s-auth` - Analyze Kubernetes/OpenShift authentication patterns
96//!
97//! ### Vault Mount Enumeration (Live Vault API)
98//! - `kv-mounts` - Enumerate all KV secrets engines and their paths
99//! - `auth-mounts` - List all authentication methods and their configurations
100//!
101//! ### Application-Specific (Parallel Processing)
102//! - `airflow-polling` - Detect Airflow polling patterns
103//!
104//! ## Parallel Processing
105//!
106//! Commands automatically detect when multiple files are provided and process them
107//! concurrently using all available CPU cores. Single-file operations use sequential
108//! processing for optimal performance.
109//!
110//! Commands with parallel processing:
111//! - `system-overview`, `entity-analysis gaps`, `path-hotspots`
112//! - `k8s-auth`, `airflow-polling`, `token-analysis`, `kv-analysis analyze`
113//!
114//! ## Installation
115//!
116//! From crates.io:
117//! ```bash
118//! cargo install vault-audit-tools
119//! ```
120//!
121//! From source:
122//! ```bash
123//! git clone https://github.com/trenner1/hashicorp-vault-audit-analysis
124//! cd hashicorp-vault-audit-analysis
125//! cargo install --path .
126//! ```
127
128pub mod audit;
129pub mod commands;
130pub mod utils;
131pub mod vault_api;