Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 56 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ mpqcli remove fth.txt wow-patch.mpq
Pretty simple, list files in an MPQ archive. Useful to "pipe" to other tools, such as `grep` (see below for examples).

```
mpqcli list wow-patch.mpq
$ mpqcli list wow-patch.mpq
BM_COKETENT01.BLP
Blizzard_CraftUI.xml
CreatureSoundData.dbc
Expand All @@ -188,22 +188,67 @@ realmlist.wtf
Similar to the `ls` command with the `-l` and `-a` options, additional detailed information can be included with the `list` subcommand. The `-a` option includes printing "special" files used in MPQ archives including: `(listfile)`, `(attributes)` and `(signature)`.

```
mpqcli list -d -a wow-patch.mpq
88604 enUS 2006-03-29 02:02:37 BM_COKETENT01.BLP
243 enUS 2006-04-04 21:28:14 Blizzard_CraftUI.xml
388 enUS 2006-03-29 19:32:46 CreatureSoundData.dbc
...
184 enUS 2006-04-04 21:28:14 Blizzard_CraftUI.lua
44900 enUS 2006-03-29 02:01:02 30ee7bd3959906e358eff01332cf045e.blp
68 enUS 2006-04-07 00:58:44 realmlist.wtf
$ mpqcli list -d -a wow-patch.mpq
88604 enUS 2006-03-29 02:02:37 BM_COKETENT01.BLP
243 enUS 2006-04-04 21:28:14 Blizzard_CraftUI.xml
388 enUS 2006-03-29 19:32:46 CreatureSoundData.dbc
...
184 enUS 2006-04-04 21:28:14 Blizzard_CraftUI.lua
44900 enUS 2006-03-29 02:01:02 30ee7bd3959906e358eff01332cf045e.blp
68 enUS 2006-04-07 00:58:44 realmlist.wtf
```

### List specific properties

The `list` subcommand supports listing the following properties:

- `hash-index` - Index in the hash table where the file entry is.
- `name-hash1` - The first hash of the file name.
- `name-hash2` - The second hash of the file name.
- `name-hash3` - 64-bit Jenkins hash of the file name, used for searching in the HET table.
- `locale` - Locale info of the file.
- `file-index` - Index in the file table of the file.
- `byte-offset` - Offset of the file in the MPQ, relative to the MPQ header.
- `file-time` - Timestamp of the file.
- `file-size` - Uncompressed file size of the file, in bytes.
- `compressed-size` - Compressed file size of the file, in bytes.
- `encryption-key` - Encryption key for the file.
- `encryption-key-raw` - Encryption key for the file.
- `flags` - File flags for the file within MPQ:
Comment on lines +205 to +217
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The names and descriptions are from here: http://zezula.net/en/mpq/stormlib/sfilegetfileinfo.html

* `i`: File is Imploded (By PKWARE Data Compression Library).
* `c`: File is Compressed (By any of multiple methods).
* `e`: File is Encrypted.
* `2`: File is Encrypted with key v2.
* `p`: File is a Patch file.
* `u`: File is stored as a single Unit, rather than split into sectors.
* `d`: File is a Deletion marker. Used in MPQ patches, indicating that the file no longer exists.
* `r`: File has Sector CRC checksums for each sector. This is ignored if the file is not compressed or imploded.
* `s`: Present on STANDARD.SNP\(signature).
* `x`: File exists; this is reset if the file is deleted.
* `m`: Mask for a file being compressed.
* `n`: Use default flags for internal files.
* `f`: Fix key; This is obsolete.
Comment on lines +218 to +230
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


You can use the `-p` or `--property` argument with the `list` subcommand to print the given properties. Many properties can be given, and they will be printed in the order given.

```
$ mpqcli list -d -a War2Patch.mpq -p hash-index -p locale -p flags
3028 enUS iexmn Rez\gluchat.bin
8926 enUS iexmn Rez\Gateways.txt
9078 enUS iexmn Rez\mltiplay.bin
10329 enUS ce2xmnf (listfile)
14213 enUS iexmn Rez\mltiplay_ita.bin
14472 enUS iexmn Rez\mltiplay_esp.bin
15278 enUS iexmn Rez\mltiplay_fra.bin
15731 enUS iexmn Rez\mltiplay_deu.bin
```

### List all files with an external listfile

Older MPQ archives do not contain (complete) file paths of their content. By providing an external listfile that lists the content of the MPQ archive, the listed files will have the correct paths. Listfiles can be downloaded on [Ladislav Zezula's site](http://www.zezula.net/en/mpq/download.html).
Older MPQ archives do not contain (complete) file paths of their content. By using the `-l` or `--listfile` argument, one can provide an external listfile that lists the content of the MPQ archive, so that the listed files will have the correct paths. Listfiles can be downloaded on [Ladislav Zezula's site](http://www.zezula.net/en/mpq/download.html).

```
mpqcli list -l /path/to/listfile StarDat.mpq
$ mpqcli list -l /path/to/listfile StarDat.mpq
```

### Extract all files from an MPQ archive
Expand Down
37 changes: 28 additions & 9 deletions src/main.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,9 @@ int main(int argc, char **argv) {
std::string baseOutput = "default"; // create, extract
std::string baseListfileName = "default"; // list, extract
// CLI: info
std::string infoProperty = "default";
std::string infoProperty = "default";
// CLI: list
std::vector<std::string> listProperties;
// CLI: extract
bool extractKeepFolderStructure = false;
// CLI: create
Expand All @@ -41,13 +43,28 @@ int main(int argc, char **argv) {
bool verifyPrintSignature = false;

std::set<std::string> validInfoProperties = {
"format-version",
"header-offset",
"header-size",
"archive-size",
"file-count",
"max-files",
"signature-type"
"format-version",
"header-offset",
"header-size",
"archive-size",
"file-count",
"max-files",
"signature-type",
};
std::set<std::string> validFileListProperties = {
"hash-index",
"name-hash1",
"name-hash2",
"name-hash3",
"locale",
"file-index",
"byte-offset",
"file-time",
"file-size",
"compressed-size",
"flags",
"encryption-key",
"encryption-key-raw",
};

// Subcommand: Version
Expand Down Expand Up @@ -101,6 +118,8 @@ int main(int argc, char **argv) {
->check(CLI::ExistingFile);
list->add_flag("-d,--detailed", listDetailed, "File listing with additional columns (default false)");
list->add_flag("-a,--all", listAll, "File listing including hidden files (default true)");
list->add_option("-p,--property", listProperties, "Prints only specific property values")
->check(CLI::IsMember(validFileListProperties));

// Subcommand: Extract
CLI::App *extract = app.add_subcommand("extract", "Extract files from the MPQ archive");
Expand Down Expand Up @@ -245,7 +264,7 @@ int main(int argc, char **argv) {
std::cerr << "[!] Failed to open MPQ archive." << std::endl;
return 1;
}
ListFiles(hArchive, baseListfileName, listAll, listDetailed);
ListFiles(hArchive, baseListfileName, listAll, listDetailed, listProperties);
}

// Handle subcommand: Extract
Expand Down
108 changes: 96 additions & 12 deletions src/mpq.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,27 @@ int RemoveFile(HANDLE hArchive, const std::string& archiveFilePath) {
return 0;
}

int ListFiles(HANDLE hArchive, const std::string& listfileName, bool listAll, bool listDetailed) {
std::string GetFlagString(uint32_t flags) {
std::string result;

if (flags & MPQ_FILE_IMPLODE) result += 'i';
if (flags & MPQ_FILE_COMPRESS) result += 'c';
if (flags & MPQ_FILE_ENCRYPTED) result += 'e';
if (flags & MPQ_FILE_KEY_V2) result += '2';
if (flags & MPQ_FILE_PATCH_FILE) result += 'p';
if (flags & MPQ_FILE_SINGLE_UNIT) result += 'u';
if (flags & MPQ_FILE_DELETE_MARKER) result += 'd';
if (flags & MPQ_FILE_SECTOR_CRC) result += 'r';
if (flags & MPQ_FILE_SIGNATURE) result += 's';
if (flags & MPQ_FILE_EXISTS) result += 'x';
if (flags & MPQ_FILE_COMPRESS_MASK) result += 'm';
if (flags & MPQ_FILE_DEFAULT_INTERNAL) result += 'n';
if (flags & MPQ_FILE_FIX_KEY) result += 'f';

return result;
}

int ListFiles(HANDLE hArchive, const std::string& listfileName, bool listAll, bool listDetailed, std::vector<std::string>& propertiesToPrint) {
// Check if the user provided a listfile input
const char *listfile = (listfileName == "default") ? NULL : listfileName.c_str();

Expand All @@ -253,6 +273,13 @@ int ListFiles(HANDLE hArchive, const std::string& listfileName, bool listAll, bo
return -1;
}

if (propertiesToPrint.empty()) {
propertiesToPrint = {
// Default properties, if the user didn't specify any
"file-size", "locale", "file-time",
};
}

// "Special" files are base files used by MPQ file format
// These are skipped, unless "-a" or "--all" are specified
std::vector<std::string> specialFiles = {
Expand All @@ -278,17 +305,74 @@ int ListFiles(HANDLE hArchive, const std::string& listfileName, bool listAll, bo
continue; // Skip to the next file
}

int32_t fileSize = GetFileInfo<int32_t>(hFile, SFileInfoFileSize);
int32_t fileLocale = GetFileInfo<int32_t>(hFile, SFileInfoLocale);
std::string fileLocaleStr = LocaleToLang(fileLocale);
int64_t fileTime = GetFileInfo<int64_t>(hFile, SFileInfoFileTime);
std::string fileTimeStr = FileTimeToLsTime(fileTime);

// Print the file details in a formatted way
std::cout << std::setw(11) << fileSize << " " // 4GB max size is 10 characters
<< std::setw(5) << fileLocaleStr << " " // Locale is max 4 characters
<< std::setw(19) << fileTimeStr << " " // File time is formatted as "YYYY-MM-DD HH:MM:SS"
<< findData.cFileName << std::endl;
std::vector<std::pair<std::string, std::function<void()>>> propertyActions = {
{"hash-index", [&]() {
std::cout << std::setw(5) << GetFileInfo<int32_t>(hFile, SFileInfoHashIndex) << " " ;
}},
{"name-hash1", [&]() {
std::cout << std::setfill('0') << std::hex << std::setw(8) <<
GetFileInfo<int32_t>(hFile, SFileInfoNameHash1) <<
std::setfill(' ') << std::dec << " ";
}},
{"name-hash2", [&]() {
std::cout << std::setfill('0') << std::hex << std::setw(8) <<
GetFileInfo<int32_t>(hFile, SFileInfoNameHash2) <<
std::setfill(' ') << std::dec << " ";
}},
{"name-hash3", [&]() {
std::cout << std::setfill('0') << std::hex << std::setw(16) <<
GetFileInfo<int64_t>(hFile, SFileInfoNameHash3) <<
std::setfill(' ') << std::dec << " ";
}},
{"locale", [&]() {
int32_t fileLocale = GetFileInfo<int32_t>(hFile, SFileInfoLocale);
std::string fileLocaleStr = LocaleToLang(fileLocale);
std::cout << std::setw(4) << fileLocaleStr << " ";
}},
{"file-index", [&]() {
std::cout << std::setw(5) << GetFileInfo<int32_t>(hFile, SFileInfoFileIndex) << " ";
}},
{"byte-offset", [&]() {
std::cout << std::hex << std::setw(8) <<
GetFileInfo<int64_t>(hFile, SFileInfoByteOffset) <<
std::dec << " ";
}},
{"file-time", [&]() {
int64_t fileTime = GetFileInfo<int64_t>(hFile, SFileInfoFileTime);
std::string fileTimeStr = FileTimeToLsTime(fileTime);
std::cout << std::setw(19) << fileTimeStr << " ";
}},
{"file-size", [&]() {
std::cout << std::setw(8) << GetFileInfo<int32_t>(hFile, SFileInfoFileSize) << " ";
}},
{"compressed-size", [&]() {
std::cout << std::setw(8) << GetFileInfo<int32_t>(hFile, SFileInfoCompressedSize) << " ";
}},
{"flags", [&]() {
int32_t flags = GetFileInfo<int32_t>(hFile, SFileInfoFlags);
std::cout << std::setw(8) << GetFlagString(flags) << " ";
}},
{"encryption-key", [&]() {
std::cout << std::setfill('0') << std::hex << std::setw(8) <<
GetFileInfo<int64_t>(hFile, SFileInfoEncryptionKey) <<
std::setfill(' ') << std::dec << " ";
}},
{"encryption-key-raw", [&]() {
std::cout << std::setfill('0') << std::hex << std::setw(8) <<
GetFileInfo<int64_t>(hFile, SFileInfoEncryptionKeyRaw) <<
std::setfill(' ') << std::dec << " ";
}},
};

for (const auto& prop : propertiesToPrint) {
for (const auto &[key, action]: propertyActions) {
if (prop == key) {
action(); // Print property
}
}
}

std::cout << " " << findData.cFileName << std::endl;

SFileCloseFile(hFile);
} else {
Expand Down
2 changes: 1 addition & 1 deletion src/mpq.h
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ HANDLE CreateMpqArchive(std::string outputArchiveName, int32_t fileCount, int32_
int AddFiles(HANDLE hArchive, const std::string& inputPath);
int AddFile(HANDLE hArchive, fs::path localFile, const std::string& archiveFilePath);
int RemoveFile(HANDLE hArchive, const std::string& archiveFilePath);
int ListFiles(HANDLE hHandle, const std::string &listfileName, bool listAll, bool listDetailed);
int ListFiles(HANDLE hHandle, const std::string &listfileName, bool listAll, bool listDetailed, std::vector<std::string>& propertiesToPrint);
char* ReadFile(HANDLE hArchive, const char *szFileName, unsigned int *fileSize);
void PrintMpqInfo(HANDLE hArchive, const std::string& infoProperty);
uint32_t VerifyMpqArchive(HANDLE hArchive);
Expand Down
Loading