brzeszczot

brzeszczot.git
git clone git://git.lenczewski.org/brzeszczot.git
Log | Files | Refs

README.txt (19312B)


      1 libriot
      2 ===============================================================================
      3 libriot is the support library to brzeszczot. It currently implements reading
      4 and writing of Riot's WAD and INIBIN formats, documented below.
      5 
      6 libriot: Primitive types
      7 ===============================================================================
      8 In the below format descriptions, the following primitive types are assumed:
      9 +-----------+---------------+-------------------------------------------------+
     10 | Type Tag  | Size (bytes)  | Description                                     |
     11 +-----------+---------------+-------------------------------------------------+
     12 | chr8      | 1 byte        | ASCII character                                 |
     13 | b8        | 1 byte        | Boolean value                                   |
     14 | f32       | 4 bytes       | 32-bit floating point value                     |
     15 | s8        | 1 byte        | Signed 8-bit integer                            |
     16 | s16       | 2 bytes       | Signed 16-bit integer                           |
     17 | s32       | 4 bytes       | Signed 32-bit integer                           |
     18 | s64       | 8 bytes       | Signed 64-bit integer                           |
     19 | u8        | 1 byte        | Unsigned 8-bit integer                          |
     20 | u16       | 2 bytes       | Unsigned 16-bit integer                         |
     21 | u32       | 4 bytes       | Unsigned 32-bit integer                         |
     22 | u64       | 8 bytes       | Unsigned 64-bit integer                         |
     23 | fnv1a_u32 | 4 bytes       | FNV1a hash value                                |
     24 | xxh64_u64 | 8 bytes       | XXH64 hash value                                |
     25 +-----------+---------------+-------------------------------------------------+
     26 
     27 Arrays (fixed length contiguous sequences of elements) are represented by the
     28 following convention:
     29 +-----------+-----------------------------------------------------------------+
     30 | Type Tag  | Description                                                     |
     31 +-----------+-----------------------------------------------------------------+
     32 | T[const]  | Homogenous array of type T, with constant length `const`        |
     33 | T[Var]    | Homogenous array of type T, with indirect length given by `Var` |
     34 +-----------+-----------------------------------------------------------------+
     35 
     36 Bitfields may also be declared by formats, and are represented by the following
     37 connvention:
     38 +-----------+-----------------------------------------------------------------+
     39 | Type Tag  | Description                                                     |
     40 +-----------+-----------------------------------------------------------------+
     41 | T[a:b]    | Bitfield across T's backing bits, spanning from bit a to bit b  |
     42 +-----------+-----------------------------------------------------------------+
     43 
     44 Note that in addition to the above primitives, formats might define additional
     45 primitive or compound types unique to said format.
     46 
     47 libriot: WAD intro
     48 ===============================================================================
     49 The WAD format (no relation to Doom's WAD) is used as an archive of INIBINs.
     50 My understanding of this format came from studying the excellent LeagueToolkit
     51 tool (https://github.com/LeagueToolkit/LeagueToolkit), without which this
     52 library wouldn't have been possible.
     53 
     54 libriot: WAD format
     55 ===============================================================================
     56 Riot's WAD format has multiple versions, and is structured as follows:
     57 +-----------------------------------------------------------------------------+
     58 | Header                                                                      |
     59 | +-------------------------------------------------------------------------+ |
     60 | | 'RW' Magic : chr[2]                                                     | |
     61 | | Major Version : u8                                                      | |
     62 | | Minor Version : u8                                                      | |
     63 | |                                                                         | |
     64 | | V2 Signature (if Major Version == 2)                                    | |
     65 | | +---------------------------------------------------------------------+ | |
     66 | | | ECDSA Length : u32                                                  | | |
     67 | | | ECDSA Signature : u8[80]                                            | | |
     68 | | | Checksum : u64                                                      | | |
     69 | | +---------------------------------------------------------------------+ | |
     70 | |                                                                         | |
     71 | | V3 Signature (if Major Version == 3)                                    | |
     72 | | +---------------------------------------------------------------------+ | |
     73 | | | ECDSA Signature : u8[256]                                           | | |
     74 | | | Checksum : u64                                                      | | |
     75 | | +---------------------------------------------------------------------+ | |
     76 | |                                                                         | |
     77 | | Table Of Contents (if Major Version <= 2)                               | |
     78 | | +---------------------------------------------------------------------+ | |
     79 | | | Start Offset : u16                                                  | | |
     80 | | | Entry Size : u16                                                    | | |
     81 | | +---------------------------------------------------------------------+ | |
     82 | +-------------------------------------------------------------------------+ |
     83 |                                                                             |
     84 | Entries                                                                     |
     85 | +-------------------------------------------------------------------------+ |
     86 | | Count : u32                                                             | |
     87 | | Chunks[Count]                                                           | |
     88 | | +---------------------------------------------------------------------+ | |
     89 | | | Path Hash : xxh64_u64                                               | | |
     90 | | | Data Offset : u32                                                   | | |
     91 | | | Compressed Size : u32                                               | | |
     92 | | | Decompressed Size : u32                                             | | |
     93 | | | Sub-Chunk Count : b8[0:3]                                           | | |
     94 | | | Compression : b8[4:7]                                               | | |
     95 | | | Duplicated : b8                                                     | | |
     96 | | | Start Sub-Chunk Index : u16                                         | | |
     97 | | | Checksum : u64 (if Major Version >= 2)                              | | |
     98 | | +---------------------------------------------------------------------+ | |
     99 | +-------------------------------------------------------------------------+ |
    100 +-----------------------------------------------------------------------------+
    101 
    102 The compression used in a given WAD entry is one of the following enum values:
    103 +-------+---------------------------------------------------------------------+
    104 | Value | Description                                                         |
    105 +-------+---------------------------------------------------------------------+
    106 | 0     | No compression                                                      |
    107 | 1     | GZip compression                                                    |
    108 | 2     | Satellite chunk; the chunk data is a string file redirect           |
    109 | 3     | Zstd compression                                                    |
    110 | 4     | Zstd compression using sub-chunks                                   |
    111 +-------+---------------------------------------------------------------------+
    112 
    113 libriot: INIBIN into
    114 ===============================================================================
    115 The INIBIN format is used to store character model and animation data. My
    116 understanding of the format came from studying existing tools written by
    117 yretenai (https://github.com/yretenai), and moonshadow565
    118 (https://github.com/moonshadow565). Without them, I likely wouldn't have been
    119 able to write this library.
    120 
    121 libriot: INIBIN format
    122 ===============================================================================
    123 Riot's INIBIN file format has multiple versions, and is structured as follows:
    124 +-----------------------------------------------------------------------------+
    125 | PTCH Header (v3+, optional)                                                 |
    126 | +-------------------------------------------------------------------------+ |
    127 | | 'PTCH' Magic : chr8[4]                                                  | |
    128 | | Unknown bytes : u64                                                     | |
    129 | +-------------------------------------------------------------------------+ |
    130 |                                                                             |
    131 | Header (v1+)                                                                |
    132 | +-------------------------------------------------------------------------+ |
    133 | | 'PROP' Magic : chr8[4]                                                  | |
    134 | | Version : u32                                                           | |
    135 | +-------------------------------------------------------------------------+ |
    136 |                                                                             |
    137 | Linked Files (v2+)                                                          |
    138 | +-------------------------------------------------------------------------+ |
    139 | | Count : u32                                                             | |
    140 | | File Names : riot_bin_str[Count]                                        | |
    141 | +-------------------------------------------------------------------------+ |
    142 |                                                                             |
    143 | Prop Entries (v1+)                                                          |
    144 | +-------------------------------------------------------------------------+ |
    145 | | Count : u32                                                             | |
    146 | | Entry Name Hashes : fnv1a_u32[Count]                                    | |
    147 | | Entries[Count]                                                          | |
    148 | | +---------------------------------------------------------------------+ | |
    149 | | | Length : u32                                                        | | |
    150 | | | Name Hash : fnv1a_u32                                               | | |
    151 | | | Count : u16                                                         | | |
    152 | | | Items[Count]                                                        | | |
    153 | | | +-----------------------------------------------------------------+ | | |
    154 | | | | Name Hash : fnv1a_u32                                           | | | |
    155 | | | | Type : u8                                                       | | | |
    156 | | | | Value : riot_bin_node<Type>                                     | | | |
    157 | | | +-----------------------------------------------------------------+ | | |
    158 | | +---------------------------------------------------------------------+ | |
    159 | +-------------------------------------------------------------------------+ |
    160 |                                                                             |
    161 | Patch Entries (v3+, if PTCH header is present)                              |
    162 | +-------------------------------------------------------------------------+ |
    163 | | Count : u32                                                             | |
    164 | | Entries[Count]                                                          | |
    165 | | +---------------------------------------------------------------------+ | |
    166 | | | Name Hash : fnv1a_u32                                               | | |
    167 | | | Length : u32                                                        | | |
    168 | | | Type : u8                                                           | | |
    169 | | | String : riot_bin_str                                               | | |
    170 | | | Value : riot_bin_node<Type>                                         | | |
    171 | | +---------------------------------------------------------------------+ | |
    172 | +-------------------------------------------------------------------------+ |
    173 +-----------------------------------------------------------------------------+
    174 
    175 In addition to the default primitives, Riot's INIBIN format declares the
    176 following primitive types:
    177 +-----------+---------------+-------------------------------------------------+
    178 | Type Tag  | Size (bytes)  | Description                                     |
    179 +-----------+---------------+-------------------------------------------------+
    180 | fvec2     | 8 bytes       | Vector containing 2 f32 values                  |
    181 | fvec3     | 12 bytes      | Vector containing 3 f32 values                  |
    182 | fvec4     | 16 bytes      | Vector containing 4 f32 values                  |
    183 | fmat4x4   | 64 bytes      | Row-major 4x4 matrix containing 16 f32 values   |
    184 | rgba      | 4 bytes       | RGBA32 colour value                             |
    185 | flag8     | 1 byte        | 8-bit flag value                                |
    186 +-----------+---------------+-------------------------------------------------+
    187 
    188 The following type aliases are declared:
    189 +-----------+---------------+-------------------------------------------------+
    190 | Type Tag  | Aliases Type  | Description                                     |
    191 +-----------+---------------+-------------------------------------------------+
    192 | hash      | fnv1a_u32     | ? (TODO)                                        |
    193 | link      | fnv1a_u32     | ? (TODO)                                        |
    194 | file      | xxh64_u64     | ? (TODO)                                        |
    195 +-----------+---------------+-------------------------------------------------+
    196 
    197 All of the above primitives can be read directly from the INIBIN file. However,
    198 the following compound types are also declared:
    199 +-----------------+-----------------------------------------------------------+
    200 | Type Tag        | Description                                               |
    201 +-----------------+-----------------------------------------------------------+
    202 | riot_bin_str    | 16-bit length prefixed ASCII string                       |
    203 | riot_bin_ptr    | List of riot_bin_field                                    |
    204 | riot_bin_embed  | List of riot_bin_field                                    |
    205 | riot_bin_opt    | Optional riot_bin_node<T> value                           |
    206 | riot_bin_list   | Homogenous list<T> of riot_bin_node<T>                    |
    207 | riot_bin_map    | Homogenous map<K, V> of riot_bin_pair<K, V>               |
    208 +-----------------+-----------------------------------------------------------+
    209 
    210 The above compound types have the following structures:
    211 +-----------------------------------------------------------------------------+
    212 | riot_bin_str                                                                |
    213 | +-------------------------------------------------------------------------+ |
    214 | | Count : u16                                                             | |
    215 | | Data : chr8[Count]                                                      | |
    216 | +-------------------------------------------------------------------------+ |
    217 |                                                                             |
    218 | riot_bin_field                                                              |
    219 | +-------------------------------------------------------------------------+ |
    220 | | Name Hash : fnv1a_u32                                                   | |
    221 | | Type : u8                                                               | |
    222 | | Value : riot_bin_node<Type>                                             | |
    223 | +-------------------------------------------------------------------------+ |
    224 |                                                                             |
    225 | riot_bin_ptr, riot_bin_embed                                                |
    226 | +-------------------------------------------------------------------------+ |
    227 | | Name Hash : fnv1a_u32                                                   | |
    228 | | Size : u32                                                              | |
    229 | | Count : u16                                                             | |
    230 | | Items : riot_bin_field[Count]                                           | |
    231 | + ------------------------------------------------------------------------+ |
    232 |                                                                             |
    233 | riot_bin_opt                                                                |
    234 | +-------------------------------------------------------------------------+ |
    235 | | Type : u8                                                               | |
    236 | | Exists : b8                                                             | |
    237 | | Value : riot_bin_node<Type> (if Exists == 1)                            | |
    238 | +-------------------------------------------------------------------------+ |
    239 |                                                                             |
    240 | riot_bin_list                                                               |
    241 | +-------------------------------------------------------------------------+ |
    242 | | Type : u8                                                               | |
    243 | | Size : u32                                                              | |
    244 | | Count : u32                                                             | |
    245 | | Items : riot_bin_node<Type>[Count]                                      | |
    246 | +-------------------------------------------------------------------------+ |
    247 |                                                                             |
    248 | riot_bin_pair<Key Type, Val Type>                                           |
    249 | +-------------------------------------------------------------------------+ |
    250 | | Key : riot_bin_node<Key Type>                                           | |
    251 | | Val : riot_bin_node<Val Type>                                           | |
    252 | +-------------------------------------------------------------------------+ |
    253 |                                                                             |
    254 | riot_bin_map                                                                |
    255 | +-------------------------------------------------------------------------+ |
    256 | | Key Type : u8                                                           | |
    257 | | Val Type : u8                                                           | |
    258 | | Size : u32                                                              | |
    259 | | Count : u32                                                             | |
    260 | | Items : riot_bin_pair<Key Type, Val Type>[Count]                        | |
    261 | +-------------------------------------------------------------------------+ |
    262 +-----------------------------------------------------------------------------+
    263 
    264 The INIBIN format conceptually collates all of the primitive and compound
    265 types declared above into a single, tagged union (`riot_bin_node`). The prop
    266 entries, patch entries, and (pseudo-)collections then store this union type.