🕸 WebShooter
WebShooter is a high-performance archiving utility designed for Unix-like systems, leveraging multi-threading to optimize operations. Developed as part of my Computer Science final project, this tool significantly outperforms traditional archivers like tar and cpio through parallel processing.
Contents
🛠️ Technologies
Built with:
🚀 Getting Started
Prerequisites
Before running the application, ensure the following tools are installed:
- Unix-like OS (Linux, BSD, macOS, etc.)
- GCC (C Compiler)
- Make and CMake
- Git
Installation & Setup
# Clone the repository
git clone https://github.com/RiadOliveira/WebShooter.git
# Navigate to the project directory
cd WebShooter/release
# Build with CMake and Make
cmake -DCMAKE_BUILD_TYPE=Release ../
make
# Run the executable
./wst -h
CLI Usage
The CLI offers three main options: archiving (--web), unarchiving (--unweb), and help.
It was designed to be simple and intuitive for both users and developers.
| Option | Alias | Required | Description |
|---|
--web | -w | Yes | Archives the specified files/folders into a single output file. Requires the output archive path followed by one or more input paths. |
--unweb | -u | Yes | Extracts contents from a previously created archive. Requires the archive path and optionally a destination path (defaults to current directory). |
--help | -h | No | Displays the full help screen with usage examples and descriptions. |
Library Integration
WebShooter also exposes a simple C API, allowing developers to embed its core functionality into other projects. Below is the definition of the main interface:
typedef struct {
const char* archivePath; // Path to the archive file (input or output)
const char** contentPaths; // List of files/folders to process
size_t contentsQuantity; // Number of input paths
} WstParams;
/**
* Archives the specified contents into a single archive file.
*/
void webContentsIntoArchive(WstParams* params);
/**
* Extracts the contents from a given archive into the current or specified directory.
*/
void unwebArchiveIntoContents(WstParams* params);
📐 Architecture & Logic
Multithreaded Strategy
The archiver uses the classic Producer-Consumer pattern:
- Producer Thread: Reads input files/directories and pushes data into shared buffers.
- Consumer Thread: Writes buffer data into an output file.
- Synchronization is done with mutexes and condition variables from
pthread.
- Double-buffering prevents thread contention.
Each buffer has a fixed size of 1 MB and is managed via status flags:
unset: Ready for Producer.consumable: Ready for Consumer.finished: End of transmission.
Archiving Process
Producer Thread
- Reads files/directories recursively.
- Store metadata and file contents into buffers.
- Fills buffers with 1MB chunks.
- Signals consumer when buffers are ready.
Consumer Thread
- Writes buffer contents to output file.
- Signals producer when buffers are empty.
- Handles final archive structure.
Unarchiving Process
Producer Thread
- Reads archive file in 1MB chunks.
- Fills buffers sequentially
- Signals consumer when data is available.
Consumer Thread
- Reconstructs original directory structure.
- Creates files with original metadata.
- Handles special directory markers.
📂 Project Structure
WebShooter/
├── .vscode/ # VSCode tasks and debug settings
├── debug/ # Debug build output
├── release/ # Release build output
├── src/
│ ├── cli/ # CLI argument handling and dispatch
│ ├── lib/ # Archiving/unarchiving logic
│ ├── utils/ # Shared helpers (buffers, I/O)
└── CMakeLists.txt # CMake configuration
⚙️ Features
- Multi-threaded Architecture: Implements producer-consumer pattern with POSIX threads for parallel archiving/unarchiving.
- CLI Utility - User-friendly and flexible command-line interface with simple usage syntax.
- Library Support - Modular library for easy integration in other C applications.
- Efficient I/O Handling - Operates using shared memory buffers, mutexes, and condition variables for synchronization.
- Directory Preservation: Maintains complete directory structure and file metadata.
- Portable and Lightweight - Pure C codebase with POSIX compliance for wide system compatibility.
- Performance-Oriented - Designed to minimize bottlenecks, reduce context-switching delays, and take advantage of multi-core CPUs.
🔧 Usage Examples
📦 Web (Archiving) Examples
# Archive a single folder into output.wst
./wst -w output.wst folder/
# Archive multiple contents (files and directories)
./wst --web archive.wst file1.txt file2.txt folder1/
📤 Unweb (Unarchiving) Examples
# Unarchive into the current directory
./wst -u archive.wst
# Unarchive into a specific output folder
./wst --unweb archive.wst ./output-folder/
Benchmarked using hyperfine against standard Unix tools:
| Tool | Archiving Speed | Unarchiving Speed |
|---|
| This Project | 1546.89 MB/s | 1419.61 MB/s |
| GNU tar | 1106.21 MB/s | 918.55 MB/s |
| GNU cpio | 448.61 MB/s | 331.51 MB/s |
🚀 Achieved 40% faster archiving and 55% faster unarchiving than tar.
Tested on Dell G15 (Intel i5-10500H, 6 cores/12 threads, NVMe SSD)
📚 Academic Article
This project was developed as part of a Computer Science undergraduate thesis. For full methodology and benchmarking details:
📖 Read the full article (PDF, 17 pages - in Brazilian Portuguese).
📜 Certification
This software is officially registered with the Brazilian National Institute of Industrial Property (INPI) as a Computer Program:
- Certificate Type: Certificate of Registration of a Computer Program
- Registration Number: BR512025003549-7
- Issuance Date: August 2025
- Expiration Date: January 2076
- Certificate URL: WebShooter Certificate
- Issuing Authority: INPI Brazil
📝 License
This project is MIT Licensed. See
LICENSE file for more details.
👨💻 Author
Ríad Oliveira
🌐 Socials
