1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275
/*
This file is part of larz.
larz is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
larz is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with larz. If not, see <https://www.gnu.org/licenses/>.
*/
//! # larz
//! A simple, fast, and efficient file archiver and compressor.
//! larz creates archives in the [GNU TAR](https://en.wikipedia.org/wiki/Tar_(computing)#File_format) format, and compresses them using [LZ4](https://en.wikipedia.org/wiki/LZ4_(compression_algorithm)).
//!
//! ## Usage
//!
//! ### Compression
//!
//! ```rust
//! use larz::compress_archive_memory;
//! use std::path::PathBuf;
//! use std::io::StdoutLock;
//!
//! let paths = vec![PathBuf::from("path/to/file"), PathBuf::from("path/to/directory")];
//! let output_path = PathBuf::from("path/to/output.larz");
//!
//! compress_archive_memory::<StdoutLock>(paths, output_path, None);
//! ```
//!
//! ### Decompression
//!
//! ```rust
//! use larz::extract_archive_memory;
//! use std::path::PathBuf;
//!
//! let paths = vec![PathBuf::from("path/to/archive.larz")];
//! let output_path = PathBuf::from("path/to/output");
//!
//! extract_archive_memory(paths, output_path);
//! ```
//!
//! ## Features
//! - `safe` - Ensures that compression and decompression are performed in a memory-safe manner. This is enabled by default.
//! - `streaming` - larz supports streaming compression and decompression using the LZ4 frame format. This means that larz can compress and decompress files with larger sizes, without having to load the entire file into memory. This is enabled by default.
//!
//! ## Installation
//! Run `cargo add larz` to add larz to your `Cargo.toml` file.
//! If you intend on using `larz` as a tool, run `cargo install larz`.
//!
//! ## License
//! larz is licensed under the [GNU Affero General Public License](https://www.gnu.org/licenses/agpl-3.0.en.html).
//!
//! ## Contributing
//! Contributions are welcome! Please see [`CONTRIBUTING.md`](https://github.com/Dirout/larz/blob/master/CONTRIBUTING.md) for more information.
//!
//! ## Authors
//! - [Emil Sayahi](https://github.com/emmyoh)
//!
//! ## Acknowledgements
//! - [`lz4_flex`](https://crates.io/crates/lz4_flex) - The LZ4 compression library used by larz.
//! - [`tar`](https://crates.io/crates/tar) - The TAR archiving library used by larz.
#![warn(missing_docs)]
use std::fs::File;
use std::io::BufReader;
use std::io::BufWriter;
use std::io::Write;
use std::path::PathBuf;
/// Archive & compress a file or set of files
///
/// # Arguments
///
/// * `paths` - A list of paths pointing to files or directories intended to be archived
///
/// * `output_path` - Path to write the archive to
///
/// * `optional_logger` - An optional `BufWriter` to log information to
///
/// # Panics
///
/// This function will panic if any of the input paths are invalid or cannot be read, if the output path is invalid, or if the archive cannot be written to.
///
/// # Examples
///
/// ```rust
/// use larz::compress_archive_streaming;
/// use std::path::PathBuf;
/// use std::io::StdoutLock;
///
/// let paths = vec![PathBuf::from("path/to/file"), PathBuf::from("path/to/directory")];
/// let output_path = PathBuf::from("path/to/output.larz");
///
/// compress_archive_streaming::<StdoutLock>(paths, output_path, None);
/// ```
#[cfg(feature = "streaming")]
pub fn compress_archive_streaming<W: Write>(
paths: Vec<PathBuf>,
output_path: PathBuf,
mut optional_logger: Option<&mut BufWriter<W>>,
) {
std::fs::create_dir_all(output_path.parent().unwrap()).unwrap();
let f = File::create(&output_path).expect("Unable to create file");
let buf = BufWriter::new(f);
let compressor = lz4_flex::frame::FrameEncoder::new(buf);
let mut tar = tar::Builder::new(compressor);
for fs_path in paths {
if let Some(ref mut logger) = optional_logger {
writeln!(logger, "Compressing '{}' … ", fs_path.to_string_lossy()).unwrap();
}
match fs_path.is_dir() {
true => {
tar.append_dir_all(".", fs_path)
.expect("Failed to write to archive");
}
false => {
tar.append_path(fs_path)
.expect("Failed to write to archive");
}
}
}
let tar_compressor = tar.into_inner().expect("Unable to finish writing archive");
tar_compressor
.finish()
.expect("Unable to finish with compression")
.flush()
.unwrap();
}
/// Extract & decompress an existing archive
///
/// # Arguments
///
/// * `paths` - A list of paths pointing to `larz` archives
///
/// * `output_path` - Path to write the extracted files to
///
/// # Panics
///
/// This function will panic if any of the input paths are invalid or cannot be read, or if the output path is invalid or cannot be written to.
///
/// # Examples
///
/// ```rust
/// use larz::extract_archive_streaming;
/// use std::path::PathBuf;
///
/// let paths = vec![PathBuf::from("path/to/archive.larz")];
/// let output_path = PathBuf::from("path/to/output");
///
/// extract_archive_streaming(paths, output_path);
/// ```
#[cfg(feature = "streaming")]
pub fn extract_archive_streaming(paths: Vec<PathBuf>, output_path: PathBuf) {
std::fs::create_dir_all(&output_path).unwrap();
for file_path in paths {
let f = std::fs::File::open(file_path).expect("Could not read archive file");
let buf = BufReader::new(f);
let extractor = lz4_flex::frame::FrameDecoder::new(buf);
let mut tar = tar::Archive::new(extractor);
tar.unpack(&output_path).expect("Could not extract archive");
}
}
/// Archive & compress a file or set of files, in memory
///
/// # Arguments
///
/// * `paths` - A list of paths pointing to files or directories intended to be archived
///
/// * `output_path` - Path to write the archive to
///
/// * `optional_logger` - An optional `BufWriter` to log information to
///
/// # Panics
///
/// This function will panic if any of the input paths are invalid or cannot be read, if the output path is invalid, or if the archive cannot be written to.
///
/// # Examples
///
/// ```rust
/// use larz::compress_archive_memory;
/// use std::path::PathBuf;
/// use std::io::StdoutLock;
///
/// let paths = vec![PathBuf::from("path/to/file"), PathBuf::from("path/to/directory")];
/// let output_path = PathBuf::from("path/to/output.larz");
///
/// compress_archive_memory::<StdoutLock>(paths, output_path, None);
/// ```
pub fn compress_archive_memory<W: Write>(
paths: Vec<PathBuf>,
output_path: PathBuf,
mut optional_logger: Option<&mut BufWriter<W>>,
) {
std::fs::create_dir_all(output_path.parent().unwrap()).unwrap();
let buf_tar: BufWriter<Vec<u8>> = BufWriter::new(Vec::new());
let mut tar = tar::Builder::new(buf_tar);
for fs_path in paths {
if let Some(ref mut logger) = optional_logger {
writeln!(logger, "Compressing '{}' … ", fs_path.to_string_lossy()).unwrap();
}
match fs_path.is_dir() {
true => {
tar.append_dir_all(".", fs_path)
.expect("Failed to write to archive");
}
false => {
tar.append_path(fs_path)
.expect("Failed to write to archive");
}
}
}
tar.finish().expect("Unable to finish with compression");
let mut buf_tar_again = tar.into_inner().unwrap();
buf_tar_again.flush().unwrap();
std::fs::create_dir_all(output_path.parent().unwrap()).unwrap();
let f = File::create(&output_path).expect("Unable to create file");
let mut buf = BufWriter::new(f);
buf.write_all(&lz4_flex::block::compress_prepend_size(
&buf_tar_again.into_inner().unwrap(),
))
.unwrap_or_else(|_| panic!("Could not write data to {}", output_path.to_string_lossy())); // Write data to file
buf.flush().unwrap();
}
/// Extract & decompress an existing archive, in memory
///
/// # Arguments
///
/// * `paths` - A list of paths pointing to `larz` archives
///
/// * `output_path` - Path to write the extracted files to
///
/// # Panics
///
/// This function will panic if any of the input paths are invalid or cannot be read, or if the output path is invalid or cannot be written to.
///
/// # Examples
///
/// ```rust
/// use larz::extract_archive_memory;
/// use std::path::PathBuf;
///
/// let paths = vec![PathBuf::from("path/to/archive.larz")];
/// let output_path = PathBuf::from("path/to/output");
///
/// extract_archive_memory(paths, output_path);
/// ```
pub fn extract_archive_memory(paths: Vec<PathBuf>, output_path: PathBuf) {
std::fs::create_dir_all(&output_path).unwrap();
for file_path in paths {
let compressed = std::fs::read(file_path).expect("Could not read archive file");
let archive = lz4_flex::decompress_size_prepended(&compressed)
.expect("Could not decompress archive file");
let archive_bytes = &archive[..];
let mut tar = tar::Archive::new(archive_bytes);
tar.unpack(&output_path).expect("Could not extract archive");
}
}