pub fn process_files_parallel<T, F, C, R>(
files: &[String],
processor: F,
combiner: C,
) -> Result<(R, usize)>Expand description
Process multiple files in parallel with memory-efficient streaming
This function processes files concurrently using a streaming approach that processes entries line-by-line without loading entire files into memory. This prevents memory allocation failures on large files.
§Arguments
files- List of file paths to processprocessor- Function that processes a single file with streaming callbackcombiner- Function that combines results from all files
§Returns
Combined result from all files plus total lines processed