src/core/BPE.h

BPE [Class]

Description

The BPE (Byte Pair Encoding) class implements the BPE compression algorithm for reducing file sizes.

Public Methods

CompressedData compress(const std::string& input)
Compresses the input string using the BPE algorithm.
Parameters
  • input: The raw text to compress.
Returns
A CompressedData structure containing the dictionary and encoded text.
std::string decompress(const CompressedData& compressedData)
Decompresses the BPE-encoded data back to original text.
Parameters
  • compressedData: The compressed data structure.
Returns
The original decompressed string.
std::string to_string(const CompressedData& data)
Serializes compressed data to a string format for file storage.
CompressedData from_string(const std::string& input)
Deserializes a string back into a CompressedData structure.