Skip to content

add support for zipping entire folders #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 6 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,8 @@ make
> [!IMPORTANT]
> To run the example, you absolutely need to provide the following environment variables:
>
> - `INPUT_FILE`: The absolute or relative path to the file you want to convert into a video
> - `INPUT_PATH`: The absolute or relative path to the file or folder you want to encrypt or decrypt
> - `OUTPUT_DIR`: The absolute or relative path to the directory where you want to save the output video (defaults to `./`)
> - `MODE`: `0` for encryption and `1` for decryption (defaults to `0`)

Execute the program:
Expand Down Expand Up @@ -104,10 +105,10 @@ docker pull jadit19/yt-storage
Make sure you've got the file you want to encrypt / decrypt in the [files](./files) directory. To run the container, execute:

```bash
docker run -v $(pwd)/files:/files -e INPUT_FILE="<file_name>" -e MODE="<0/1>" jadit19/yt-storage
docker run -v $(pwd)/files:/files -e INPUT_PATH="<file_name>" -e OUTPUT_DIR="/files" -e MODE="<0/1>" jadit19/yt-storage
```

Remember that the `INPUT_FILE` environment variable should be the base name of the file you want to encrypt / decrypt, and the `MODE` environment variable should be `0` for encryption and `1` for decryption.
Remember that the `INPUT_PATH` environment variable should be the absolute or relative path to the file or folder you want to encrypt / decrypt, the `OUTPUT_DIR` environment variable should be the absolute or relative path to the directory where you want to save the output video, and the `MODE` environment variable should be `0` for encryption and `1` for decryption.

If you are like me and prefer using docker compose, you can use the [encrypt.compose.yaml](./docker/encrypt.compose.yaml) and [decrypt.compose.yaml](./docker/decrypt.compose.yaml) files to run the encryption and decryption processes respectively after specifying the required environment variables:

Expand All @@ -123,7 +124,7 @@ For testing purposes, I've provided a sample file that you can decrypt post down

## Internal working

Apart from the actual 0s and 1s, I need to store some (meta)data in the encrypted video file that can be used to decrypt it. For now, I am saving 3 things, namely, the name of the output file (post decryption), the total size of the original file (in bits) and the version of yt-storage used to encrypt the file (backwards compatibility is not guaranteed).
Apart from the actual 0s and 1s, I need to store some (meta)data in the encrypted video file that can be used to decrypt it. For now, I am saving 3 things, namely, the name of the output file (post decryption), the total size of the original or final zipped file (in bits) and the version of yt-storage used to encrypt the file (backwards compatibility is not guaranteed).

I am saving this metadata as a json string in the first few bits of the video. Now, to properly read this metadata, I need to know how many bits to read. This is where the `metadata_size` comes in. It is the number of bits required to store the metadata, and occupies the first 16 bits of the first frame of the video.

Expand All @@ -133,9 +134,7 @@ After doing, I uploaded an encrypted file to YouTube and then download it in the

## Future works

For added security, the read stream can be encoded using the OpenSSL library and require a key to decrypt. Moreover, encrypting entire folders (as a zip file) could be a potential.

Also, this thing could do with some optimization, which are two ways I can think of right now:
For added security, the read stream can be encoded using the OpenSSL library and require a key to decrypt. Also, this thing could do with some optimization, which are two ways I can think of right now:

1. Black and white pixels are not the only way to represent bits. I could use the RGB channels to represent 3 bits per pixel, which would reduce the size of the video by a third.

Expand Down
3 changes: 2 additions & 1 deletion docker/decrypt.compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,6 @@ services:
volumes:
- ../files:/files
environment:
INPUT_FILE: 'output.mp4'
MODE: '1'
INPUT_PATH: '/files/output.mp4'
OUTPUT_DIR: '/files'
5 changes: 3 additions & 2 deletions docker/encrypt.compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,10 @@ services:
volumes:
- ../files:/files
environment:
INPUT_FILE: 'important-video.mp4'
MODE: '0'
INPUT_PATH: '/files/important-video.mp4'
OUTPUT_DIR: '/files'
OUTPUT_FILE: 'output.mp4'
WIDTH: '1920'
HEIGHT: '1080'
FPS: '30'
OUTPUT_FILE: 'output.mp4'
5 changes: 3 additions & 2 deletions example/main.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@
int main(int argc, char **argv) {
yt::options options;
int mode = std::stoi(brewtils::env::get("MODE", "0"));
options.inputFile = brewtils::env::get("INPUT_FILE");
if (options.inputFile.empty()) {
options.inputPath = brewtils::env::get("INPUT_PATH");
options.outputDir = brewtils::env::get("OUTPUT_DIR", "./");
if (options.inputPath.empty()) {
logger::error("Input file not provided", "int main(int argc, char **argv)");
}

Expand Down
9 changes: 7 additions & 2 deletions include/yt-storage/encrypter.h
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
#pragma once

#include <zippuccino/zipper.h>

#include <yt-storage/options.h>

namespace yt {
Expand All @@ -12,11 +14,14 @@ class Encrypter {

yt::options options;

std::vector<bool> getJsonData();
std::string getJsonData();
void setMetadata();
void setFiledata();
void setFiledata(const std::string &filePath);
void setStringData(const std::string &data);
void setData(bool data);

zippuccino::Zipper *zipper;

public:
Encrypter();
~Encrypter();
Expand Down
3 changes: 2 additions & 1 deletion include/yt-storage/options.h
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@ typedef struct {
int width;
int height;
int fps;
std::string inputFile;
std::string inputPath;
std::string outputFile;
std::string outputDir;
} options;

} // namespace yt
17 changes: 9 additions & 8 deletions src/decrypter.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,14 @@ void yt::Decrypter::decrypt(yt::options options) {
this->options = options;
this->options.width = -1;
this->options.height = -1;
if (!brewtils::os::file::exists(this->options.inputFile)) {
logger::error("Input file " + this->options.inputFile + " does not exist",
if (!brewtils::os::file::exists(this->options.inputPath)) {
logger::error("Input file " + this->options.inputPath + " does not exist",
"void yt::Decrypter::decrypt(yt::options options)");
}

this->cap.open(this->options.inputFile);
this->cap.open(this->options.inputPath);
if (!this->cap.isOpened()) {
logger::error("Could not open file " + this->options.inputFile,
logger::error("Could not open file " + this->options.inputPath,
"void yt::Decrypter::decrypt(yt::options options)");
}

Expand Down Expand Up @@ -96,8 +96,7 @@ void yt::Decrypter::readMetadata() {
"void yt::Decrypter::readMetadata()");
}

this->options.outputFile =
"../files/" + static_cast<std::string>(data["name"]);
this->options.outputFile = static_cast<std::string>(data["name"]);
this->fileSize = data["size"];
} catch (const std::exception &e) {
logger::error("Invalid metadata detected",
Expand All @@ -111,9 +110,11 @@ void yt::Decrypter::readMetadata() {
}

void yt::Decrypter::readFiledata() {
std::ofstream file(this->options.outputFile, std::ios::binary);
const std::string outputFile =
brewtils::os::joinPath(this->options.outputDir, this->options.outputFile);
std::ofstream file(outputFile, std::ios::binary);
if (!file.is_open()) {
logger::error("Could not open file " + this->options.outputFile,
logger::error("Could not open file " + outputFile,
"void yt::Decrypter::readFiledata()");
}

Expand Down
79 changes: 50 additions & 29 deletions src/encrypter.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@
yt::Encrypter::Encrypter() : col(0), row(0) { return; }

yt::Encrypter::~Encrypter() {
if (this->zipper != nullptr) {
delete this->zipper;
}
if (this->writer.isOpened()) {
this->writer.release();
}
Expand All @@ -11,22 +14,31 @@ yt::Encrypter::~Encrypter() {

void yt::Encrypter::encrypt(yt::options options) {
this->options = options;
if (!brewtils::os::file::exists(this->options.inputFile)) {
logger::error("Input file " + this->options.inputFile + " does not exist",
if (!brewtils::os::file::exists(this->options.inputPath) &&
!brewtils::os::dir::exists(this->options.inputPath)) {
logger::error("Input path " + this->options.inputPath + " does not exist",
"void yt::Encrypter::encrypt(yt::options options)");
}

this->writer.open(this->options.outputFile,
cv::VideoWriter::fourcc('m', 'p', '4', 'v'),
this->options.fps,
cv::Size(this->options.width, this->options.height), false);
this->writer.open(
brewtils::os::joinPath(this->options.outputDir, this->options.outputFile),
cv::VideoWriter::fourcc('m', 'p', '4', 'v'), this->options.fps,
cv::Size(this->options.width, this->options.height), false);
if (!this->writer.isOpened()) {
logger::error("Could not open video writer",
"void yt::Encrypter::encrypt(yt::options options)");
}

this->setMetadata();
this->setFiledata();
if (this->zipper == nullptr) {
this->setFiledata(this->options.inputPath);
} else {
while (!this->zipper->isFinished()) {
this->setStringData(this->zipper->getHeader());
this->setFiledata(this->zipper->getCurrentFile());
}
this->setStringData(this->zipper->getFooter());
}

if (this->row != 0 || this->col != 0) {
this->writer.write(this->frame);
Expand All @@ -35,44 +47,44 @@ void yt::Encrypter::encrypt(yt::options options) {
return;
}

std::vector<bool> yt::Encrypter::getJsonData() {
std::string yt::Encrypter::getJsonData() {
std::vector<bool> response;
json::object data;
data["name"] = brewtils::os::basePath(this->options.inputFile);
data["size"] =
(long long int)(brewtils::os::file::size(this->options.inputFile) * 8);
data["version"] = yt::version;
const std::string stringifiedJson = data.dumps();
int asciiValue, i;
for (char ch : stringifiedJson) {
asciiValue = static_cast<int>(ch);
for (i = 7; i >= 0; --i) {
response.push_back((asciiValue >> i) & 1);
}
if (brewtils::os::file::exists(this->options.inputPath)) {
data["name"] = brewtils::os::basePath(this->options.inputPath);
data["size"] =
(long long int)(brewtils::os::file::size(this->options.inputPath) * 8);
} else if (brewtils::os::dir::exists(this->options.inputPath)) {
this->zipper = new zippuccino::Zipper();
zipper->add(this->options.inputPath);
data["name"] = brewtils::os::basePath(this->options.inputPath) + ".zip";
data["size"] = (long long int)(this->zipper->getSize() * 8);
} else {
logger::error("Input path " + this->options.inputPath +
" is neither a file nor a directory",
"std::vector<bool> yt::Encrypter::getJsonData()");
}
return response;
data["version"] = yt::version;
return data.dumps();
}

void yt::Encrypter::setMetadata() {
const std::vector<bool> jsonData = yt::Encrypter::getJsonData();
uint16_t jsonDataSize = jsonData.size();
const std::string jsonData = yt::Encrypter::getJsonData();
uint16_t jsonDataSize = jsonData.size() * 8;

for (int i = 0; i < 16; i++) {
this->setData(jsonDataSize % 2 != 0);
jsonDataSize /= 2;
}

for (const bool &bit : jsonData) {
this->setData(bit);
}

this->setStringData(jsonData);
return;
}

void yt::Encrypter::setFiledata() {
std::ifstream file(this->options.inputFile, std::ios::binary);
void yt::Encrypter::setFiledata(const std::string &filePath) {
std::ifstream file(filePath, std::ios::binary);
if (!file.is_open()) {
logger::error("Could not open file " + this->options.inputFile,
logger::error("Could not open file " + filePath,
"void yt::Encrypter::setFiledata()");
}

Expand All @@ -87,6 +99,15 @@ void yt::Encrypter::setFiledata() {
return;
}

void yt::Encrypter::setStringData(const std::string &data) {
for (char ch : data) {
for (int i = 7; i >= 0; --i) {
this->setData((ch >> i) & 1);
}
}
return;
}

void yt::Encrypter::setData(bool data) {
for (int i = 0; i < yt::REDUNDANCY; i++) {
if (this->row == 0 && this->col == 0) {
Expand Down