vault_audit_tools/
lib.rs

1//! # Vault Audit Tools
2//!
3//! High-performance command-line tools for analyzing `HashiCorp Vault` audit logs with
4//! automatic parallel processing and compressed file support.
5//!
6//! ## Overview
7//!
8//! This crate provides a suite of specialized tools for parsing and analyzing
9//! `HashiCorp Vault` audit logs. It's designed to handle large production logs
10//! (multi-gigabyte files) efficiently through streaming parsing, parallel processing,
11//! and minimal memory overhead.
12//!
13//! ## Performance
14//!
15//! - **3x faster** than equivalent Python implementations (single file)
16//! - **40% faster** with automatic parallel processing (multi-file workloads)
17//! - **10x less memory** usage through streaming parser
18//! - Processes 4M line logs in ~17 seconds (vs ~60s in Python)
19//! - Near-linear CPU scaling with available cores
20//!
21//! ### Real-world Benchmarks
22//!
23//! - KV Analysis: 141s -> 85s (40.1% faster with parallel processing)
24//! - Memory usage: ~77 MB with parallel workers (2x overhead)
25//! - Throughput: 233 MB/s (vs 140 MB/s sequential)
26//!
27//! ## Features
28//!
29//! - **Parallel Processing** - Automatically processes multiple files concurrently
30//! - **16 specialized analysis commands** for different use cases
31//! - **Compressed File Support** - Direct analysis of `.gz` and `.zst` files
32//! - **Streaming JSON parser** for memory-efficient processing
33//! - **Entity lifecycle tracking** across multiple days
34//! - **Token usage analysis** and abuse detection
35//! - **KV secrets engine analysis** (v1 and v2)
36//! - **Kubernetes auth analysis**
37//! - **Shell completion** for bash, zsh, fish, powershell, and elvish
38//!
39//! ## Architecture
40//!
41//! The crate is organized into several key modules:
42//!
43//! - [`audit`] - Core audit log parsing and data structures
44//! - [`commands`] - Individual analysis command implementations
45//! - [`utils`] - Shared utilities (parallel processing, progress, time parsing)
46//! - [`vault_api`] - Vault API client for entity enrichment
47//!
48//! ## Example Usage
49//!
50//! ```bash
51//! # System overview with automatic parallel processing
52//! vault-audit system-overview logs/*.log
53//!
54//! # Entity analysis (unified command with auto-preprocessing)
55//! vault-audit entity-analysis churn day1.log day2.log day3.log
56//! vault-audit entity-analysis gaps audit.log
57//!
58//! # Token analysis with abuse detection
59//! vault-audit token-analysis audit.log --abuse-threshold 5000
60//!
61//! # KV secrets analysis (40% faster with parallel processing)
62//! vault-audit kv-analysis analyze logs/*.log --output kv_usage.csv
63//!
64//! # Compressed files work seamlessly
65//! vault-audit path-hotspots audit.log.gz
66//! ```
67//!
68//! ## Command Categories
69//!
70//! ### System Analysis (Parallel Processing)
71//! - `system-overview` - High-level audit log statistics
72//! - `path-hotspots` - Identify most accessed paths
73//!
74//! ### Entity Analysis (Unified Commands)
75//! - `entity-analysis churn` - Multi-day entity lifecycle tracking
76//! - `entity-analysis creation` - Track when entities first appear
77//! - `entity-analysis gaps` - Find gaps in entity activity (parallel)
78//! - `entity-analysis timeline` - Individual entity activity timeline
79//! - `entity-analysis preprocess` - Extract entity mappings
80//!
81//! ### Token Analysis (Unified Command, Parallel Processing)
82//! - `token-analysis` - Token lifecycle operations with abuse detection
83//!
84//! ### KV Secrets Analysis (Unified Commands, Parallel Processing)
85//! - `kv-analysis analyze` - Analyze KV secret access patterns
86//! - `kv-analysis compare` - Compare KV usage across time periods
87//! - `kv-analysis summary` - Summarize KV usage by mount point
88//!
89//! ### Authentication Analysis (Parallel Processing)
90//! - `k8s-auth` - Analyze Kubernetes/OpenShift authentication patterns
91//!
92//! ### Application-Specific (Parallel Processing)
93//! - `airflow-polling` - Detect Airflow polling patterns
94//!
95//! ## Parallel Processing
96//!
97//! Commands automatically detect when multiple files are provided and process them
98//! concurrently using all available CPU cores. Single-file operations use sequential
99//! processing for optimal performance.
100//!
101//! Commands with parallel processing:
102//! - `system-overview`, `entity-analysis gaps`, `path-hotspots`
103//! - `k8s-auth`, `airflow-polling`, `token-analysis`, `kv-analysis analyze`
104//!
105//! ## Installation
106//!
107//! From crates.io:
108//! ```bash
109//! cargo install vault-audit-tools
110//! ```
111//!
112//! From source:
113//! ```bash
114//! git clone https://github.com/trenner1/hashicorp-vault-audit-analysis
115//! cd hashicorp-vault-audit-analysis
116//! cargo install --path .
117//! ```
118
119pub mod audit;
120pub mod commands;
121pub mod utils;
122pub mod vault_api;
vault_audit_tools/lib.rs

vault_audit_tools/
lib.rs
