Locale support - base PR

README.md

@@ -173,7 +173,7 @@ mpqcli remove fth.txt wow-patch.mpq
 Pretty simple, list files in an MPQ archive. Useful to "pipe" to other tools, such as `grep` (see below for examples).
 
 ```
-mpqcli list wow-patch.mpq
+$ mpqcli list wow-patch.mpq
 BM_COKETENT01.BLP
 Blizzard_CraftUI.xml
 CreatureSoundData.dbc
@@ -188,22 +188,67 @@ realmlist.wtf
 Similar to the `ls` command with the `-l` and `-a` options, additional detailed information can be included with the `list` subcommand. The `-a` option includes printing "special" files used in MPQ archives including: `(listfile)`, `(attributes)` and `(signature)`.
 
 ```
-mpqcli list -d -a wow-patch.mpq 
-      88604  enUS 2006-03-29 02:02:37  BM_COKETENT01.BLP
-        243  enUS 2006-04-04 21:28:14  Blizzard_CraftUI.xml
-        388  enUS 2006-03-29 19:32:46  CreatureSoundData.dbc
-        ...
-        184  enUS 2006-04-04 21:28:14  Blizzard_CraftUI.lua
-      44900  enUS 2006-03-29 02:01:02  30ee7bd3959906e358eff01332cf045e.blp
-         68  enUS 2006-04-07 00:58:44  realmlist.wtf
+$ mpqcli list -d -a wow-patch.mpq 
+   88604 enUS 2006-03-29 02:02:37  BM_COKETENT01.BLP
+     243 enUS 2006-04-04 21:28:14  Blizzard_CraftUI.xml
+     388 enUS 2006-03-29 19:32:46  CreatureSoundData.dbc
+     ...
+     184 enUS 2006-04-04 21:28:14  Blizzard_CraftUI.lua
+   44900 enUS 2006-03-29 02:01:02  30ee7bd3959906e358eff01332cf045e.blp
+      68 enUS 2006-04-07 00:58:44  realmlist.wtf
+```
+
+### List specific properties
+
+The `list` subcommand supports listing the following properties:
+
+- `hash-index` - Index in the hash table where the file entry is.
+- `name-hash1` - The first hash of the file name.
+- `name-hash2` - The second hash of the file name.
+- `name-hash3` - 64-bit Jenkins hash of the file name, used for searching in the HET table.
+- `locale` - Locale info of the file.
+- `file-index` - Index in the file table of the file.
+- `byte-offset` - Offset of the file in the MPQ, relative to the MPQ header.
+- `file-time` - Timestamp of the file.
+- `file-size` - Uncompressed file size of the file, in bytes.
+- `compressed-size` - Compressed file size of the file, in bytes.
+- `encryption-key` - Encryption key for the file.
+- `encryption-key-raw` - Encryption key for the file.
+- `flags` - File flags for the file within MPQ:
+  * `i`: File is Imploded (By PKWARE Data Compression Library).
+  * `c`: File is Compressed (By any of multiple methods).
+  * `e`: File is Encrypted.
+  * `2`: File is Encrypted with key v2.
+  * `p`: File is a Patch file.
+  * `u`: File is stored as a single Unit, rather than split into sectors.
+  * `d`: File is a Deletion marker. Used in MPQ patches, indicating that the file no longer exists.
+  * `r`: File has Sector CRC checksums for each sector. This is ignored if the file is not compressed or imploded.
+  * `s`: Present on STANDARD.SNP\(signature).
+  * `x`: File exists; this is reset if the file is deleted.
+  * `m`: Mask for a file being compressed.
+  * `n`: Use default flags for internal files.
+  * `f`: Fix key; This is obsolete.
+
+You can use the `-p` or `--property` argument with the `list` subcommand to print the given properties. Many properties can be given, and they will be printed in the order given.
+
+```
+$ mpqcli list -d -a War2Patch.mpq -p hash-index -p locale -p flags
+ 3028 enUS    iexmn  Rez\gluchat.bin
+ 8926 enUS    iexmn  Rez\Gateways.txt
+ 9078 enUS    iexmn  Rez\mltiplay.bin
+10329 enUS  ce2xmnf  (listfile)
+14213 enUS    iexmn  Rez\mltiplay_ita.bin
+14472 enUS    iexmn  Rez\mltiplay_esp.bin
+15278 enUS    iexmn  Rez\mltiplay_fra.bin
+15731 enUS    iexmn  Rez\mltiplay_deu.bin
 ```
 
 ### List all files with an external listfile
 
-Older MPQ archives do not contain (complete) file paths of their content. By providing an external listfile that lists the content of the MPQ archive, the listed files will have the correct paths. Listfiles can be downloaded on [Ladislav Zezula's site](http://www.zezula.net/en/mpq/download.html).
+Older MPQ archives do not contain (complete) file paths of their content. By using the `-l` or `--listfile` argument, one can provide an external listfile that lists the content of the MPQ archive, so that the listed files will have the correct paths. Listfiles can be downloaded on [Ladislav Zezula's site](http://www.zezula.net/en/mpq/download.html).
 
 ```
-mpqcli list -l /path/to/listfile StarDat.mpq
+$ mpqcli list -l /path/to/listfile StarDat.mpq
 ```
 
 ### Extract all files from an MPQ archive

src/CMakeLists.txt

@@ -15,6 +15,7 @@ add_executable(mpqcli
     main.cpp 
     mpq.cpp 
     helpers.cpp
+    locales.cpp
 )
 
 # Add dependencies

src/helpers.cpp

@@ -1,8 +1,6 @@
 #include <algorithm>
 #include <filesystem>
-#include <fstream>
 #include <iostream>
-#include <vector>
 #include <ctime>
 
 #ifdef _WIN32
@@ -33,24 +31,6 @@ std::string FileTimeToLsTime(int64_t fileTime) {
     return std::string(buf);
 }
 
-std::string LocaleToLang(uint16_t locale) {
-    switch (locale) {
-        case 0:  return "enUS";     // English (US/GB)
-        case 1:  return "koKR";     // Korean
-        case 2:  return "frFR";     // French
-        case 3:  return "deDE";     // German
-        case 4:  return "zhCN";     // Chinese (Simplified)
-        case 5:  return "zhTW";     // Chinese (Taiwan)
-        case 6:  return "esES";     // Spanish (Spain)
-        case 7:  return "esMX";     // Spanish (Mexico)
-        case 8:  return "ruRU";     // Russian
-        case 9:  return "jaJP";     // Japanese
-        case 10: return "ptPT";     // Portuguese (Portugal)
-        case 11: return "itIT";     // Italian
-        default: return "";         // Unknown locale
-    }
-}
-
 std::string NormalizeFilePath(const fs::path &path) {
     std::string filePath = path.u8string();
     #ifndef _WIN32

src/helpers.h

@@ -7,7 +7,6 @@
 namespace fs = std::filesystem;
 
 std::string FileTimeToLsTime(int64_t fileTime);
-std::string LocaleToLang(uint16_t locale);
 std::string NormalizeFilePath(const fs::path &path);
 std::string WindowsifyFilePath(const fs::path &path);
 int32_t CalculateMpqMaxFileValue(const std::string &directory);

src/locales.cpp

@@ -0,0 +1,71 @@
+#include <algorithm>
+#include <fstream>
+#include <vector>
+#include <map>
+
+#include "locales.h"
+
+namespace {
+    // Files in MPQs have locales with which they are associated.
+    // Multiple files can have the same file name if they have different locales.
+    // This function maps locales to language names.
+    //
+    // The mappings are from the Windows Language Code Identifier (LCID).
+    // They can be found, for example, here:
+    // https://winprotocoldoc.z19.web.core.windows.net/MS-LCID/%5bMS-LCID%5d.pdf
+
+    // Define a bidirectional map for locale-language mappings
+    const std::map<uint16_t, std::string> localeToLangMap = {
+            {0x000, "enUS"},  // Default - English (US)
+            {0x404, "zhTW"},  // Chinese (Taiwan)
+            {0x405, "csCZ"},  // Czech
+            {0x407, "deDE"},  // German
+            {0x409, "enUS"},  // English (US)
+            {0x40a, "esES"},  // Spanish (Spain)
+            {0x40c, "frFR"},  // French
+            {0x410, "itIT"},  // Italian
+            {0x411, "jaJP"},  // Japanese
+            {0x412, "koKR"},  // Korean
+            {0x413, "nlNL"},  // Dutch
+            {0x415, "plPL"},  // Polish
+            {0x416, "ptPT"},  // Portuguese (Portugal)
+            {0x419, "ruRU"},  // Russian
+            {0x804, "zhCN"},  // Chinese (Simplified)
+            {0x809, "enGB"},  // English (UK)
+            {0x80A, "esMX"}   // Spanish (Mexico)
+    };
+
+    // Create a reverse map for language to locale lookups
+    const std::map<std::string, uint16_t> langToLocaleMap = []() {
+        std::map<std::string, uint16_t> reverseMap;
+        for (const auto& [locale, lang] : localeToLangMap) {
+            if (locale != defaultLocale) { // Skip the default locale to avoid duplication
+                reverseMap[lang] = locale;
+            }
+        }
+        return reverseMap;
+    }();
+}
+
+std::string LocaleToLang(uint16_t locale) {
+    auto it = localeToLangMap.find(locale);
+    return it != localeToLangMap.end() ? it->second : "";
+}
+
+LCID LangToLocale(const std::string& lang) {
+    auto it = langToLocaleMap.find(lang);
+    return it != langToLocaleMap.end() ? it->second : defaultLocale;
+}
+
+
+std::vector<std::string> GetAllLocales() {
+    std::vector<std::string> locales;
+    for (const auto& [locale, lang] : localeToLangMap) {
+        if (locale != defaultLocale) { // Skip the default locale to avoid duplication
+            locales.push_back(lang);
+        }
+    }
+    // Sort the locales for consistent output
+    std::sort(locales.begin(), locales.end());
+    return locales;
+}

src/locales.h

@@ -0,0 +1,35 @@
+#ifndef LOCALES_H
+#define LOCALES_H
+
+#include <string>
+#include <filesystem>
+
+#include <StormLib.h>
+#include <CLI/CLI.hpp>
+
+const LCID defaultLocale = 0;
+
+std::string LocaleToLang(uint16_t locale);
+LCID LangToLocale(const std::string &lang);
+std::vector<std::string> GetAllLocales();
+
+// Validator for CLI11
+const inline auto LocaleValid = CLI::Validator(
+        [](const std::string &str) {
+            if (str == "default") return std::string();
+
+            LCID locale = LangToLocale(str);
+            if (locale == 0) {
+                std::string validLocales = "Locale must be nothing, or one of:";
+                for (const auto& l : GetAllLocales()) {
+                    validLocales += " " + l;
+                }
+                return validLocales;
+            }
+            return std::string();
+        },
+        "Validates locales and outputs valid locales",
+        "LocaleValidator"
+);
+
+#endif //LOCALES_H

src/main.cpp

@@ -7,6 +7,7 @@
 
 #include "mpq.h"
 #include "helpers.h"
+#include "locales.h"
 #include "mpqcli.h"
 
 namespace fs = std::filesystem;
@@ -16,7 +17,7 @@ int main(int argc, char **argv) {
         "A command line tool to create, add, remove, list, extract, read, and verify MPQ archives "
         "using the StormLib library"
     };
-    
+
     // Require at least one subcommand
     app.require_subcommand(1);
 
@@ -25,10 +26,13 @@ int main(int argc, char **argv) {
     std::string baseTarget = "default";  // all subcommands
     std::string baseFile = "default";  // add, remove, extract, read
     std::string basePath = "default"; // add
+    std::string baseLocale = "default"; // create, add, remove, extract, read
     std::string baseOutput = "default";  // create, extract
     std::string baseListfileName = "default";  // list, extract
     // CLI: info
-    std::string infoProperty = "default";    
+    std::string infoProperty = "default";
+    // CLI: list
+    std::vector<std::string> listProperties;
     // CLI: extract
     bool extractKeepFolderStructure = false;
     // CLI: create
@@ -41,13 +45,28 @@ int main(int argc, char **argv) {
     bool verifyPrintSignature = false;
 
     std::set<std::string> validInfoProperties = {
-        "format-version",
-        "header-offset",
-        "header-size",
-        "archive-size",
-        "file-count",
-        "max-files",
-        "signature-type"
+            "format-version",
+            "header-offset",
+            "header-size",
+            "archive-size",
+            "file-count",
+            "max-files",
+            "signature-type",
+    };
+    std::set<std::string> validFileListProperties = {
+            "hash-index",
+            "name-hash1",
+            "name-hash2",
+            "name-hash3",
+            "locale",
+            "file-index",
+            "byte-offset",
+            "file-time",
+            "file-size",
+            "compressed-size",
+            "flags",
+            "encryption-key",
+            "encryption-key-raw",
     };
 
     // Subcommand: Version
@@ -101,6 +120,8 @@ int main(int argc, char **argv) {
         ->check(CLI::ExistingFile);
     list->add_flag("-d,--detailed", listDetailed, "File listing with additional columns (default false)");
     list->add_flag("-a,--all", listAll, "File listing including hidden files (default true)");
+    list->add_option("-p,--property", listProperties, "Prints only specific property values")
+        ->check(CLI::IsMember(validFileListProperties));
 
     // Subcommand: Extract
     CLI::App *extract = app.add_subcommand("extract", "Extract files from the MPQ archive");
@@ -147,7 +168,7 @@ int main(int argc, char **argv) {
     }
 
     // Handle subcommand: About
-    if (app.got_subcommand(about)){
+    if (app.got_subcommand(about)) {
         std::cout << "Name: mpqcli" << std::endl;
         std::cout << "Version: " << MPQCLI_VERSION << "-" << GIT_COMMIT_HASH << std::endl;
         std::cout << "Author: Thomas Laurenson" << std::endl;
@@ -156,10 +177,10 @@ int main(int argc, char **argv) {
         std::cout << "Dependencies:" << std::endl;
         std::cout << " - StormLib (https://github.com/ladislav-zezula/StormLib)" << std::endl;
         std::cout << " - CLI11 (https://github.com/CLIUtils/CLI11)" << std::endl;
-    };
+    }
 
     // Handle subcommand: Info
-    if (app.got_subcommand(info)){
+    if (app.got_subcommand(info)) {
         HANDLE hArchive;
         if (!OpenMpqArchive(baseTarget, &hArchive, MPQ_OPEN_READ_ONLY)) {
             std::cerr << "[!] Failed to open MPQ archive." << std::endl;
@@ -178,12 +199,12 @@ int main(int argc, char **argv) {
             outputFilePath.replace_extension(".mpq");
         }
         std::string outputFile = outputFilePath.u8string();
-        
+
         std::cout << "[*] Output file: " << outputFile << std::endl;
 
-        // Determine number of files we are going to add
+        // Determine the number of files we are going to add
         int32_t fileCount = CalculateMpqMaxFileValue(baseTarget);
-    
+
         // Create the MPQ archive and add files
         HANDLE hArchive = CreateMpqArchive(outputFile, fileCount, createMpqVersion);
         if (hArchive) {
@@ -245,7 +266,7 @@ int main(int argc, char **argv) {
             std::cerr << "[!] Failed to open MPQ archive." << std::endl;
             return 1;
         }
-        ListFiles(hArchive, baseListfileName, listAll, listDetailed);
+        ListFiles(hArchive, baseListfileName, listAll, listDetailed, listProperties);
     }
 
     // Handle subcommand: Extract