50 replies to this topic

[Argent77]

File Name: TileConv - A MOS/TIS Compressor
File Submitter: Argent77
File Submitted: 30 Aug 2014
File Category: IE Modding Tools

TileConv is an advanced tiled image compressor and decompressor for MOS and TIS files used in Infinity Engine games (such as Baldur's Gate, Icewind Dale or Planescape Torment). It uses modern compression algorithms to achieve great compression ratios without noticeable degradation in quality. The average compression size lies around 20-30% of the original size and will never exceed 40% when using default compression options.

The tool can convert between the following formats:

• TIS files will be automatically compressed into the TBC format.
  
• MOS files will be automatically compressed into the MBC format.
  
• TBC files will be automatically decompressed into the TIS format.
  
• MBC files will be automatically decompressed into the (compressed or uncompressed) MOS format.
  
• TIZ files will be automatically decompressed into the TIS format.
  
• MOZ files will be automatically decompressed into the (compressed or uncompressed) MOS format.

TileConv is aimed at modders who wish to provide their own TIS or MOS files without having large downloads. The package contains the source code as well as binaries for Windows, Mac OS X and Linux. The latest source code is also available on GitHub. A detailed description of the TBC and MBC formats can be found here.
 
This package also contains a collection of WeiDU functions to deal with tileconv executables for multiple platforms and architectures, including a detailed description and example code.
 
Usage

Spoiler

Usage: tileconv [options] infile [infile2 [...]]

Options:
  -e          Do not halt on errors.
  -s          Be silent.
  -v          Be verbose.
  -t type     Select pixel encoding type.
              Supported types:
                0: No pixel encoding
                1: BC1/DXT1 (Default)
                2: BC2/DXT3
                3: BC3/DXT5
  -u          Do not apply tile compression.
  -o output   Select output file or folder.
              (Note: Output file works only with single input file!)
  -z          MOS only: Decompress MBC into compressed MOS (MOSC).
  -q Dec[Enc] Set quality levels for decoding and, optionally, encoding.
              Supported levels: 0..9 (Defaults: 4 for decoding, 9 for encoding)
              (0=fast and lower quality, 9=slow and higher quality)
              Specify both levels as a single argument. First digit indicates
              decoding quality and second digit indicates encoding quality.
              Specify '-' as placeholder for default levels.
              Example 1: -q 27 (decoding level: 2, encoding level: 7)
              Example 2: -q -7 (default decoding level, encoding level: 7)
              Example 3: -q 2  (decoding level: 2, default encoding level)
              Applied level-dependent features for encoding (DXTn only):
                  Iterative cluster fit:   levels 7 to 9
                  Single cluster fit:      levels 3 to 6
                  Range fit:               levels 0 to 2
                  Weight color by alpha:   levels 5 to 9
              Applied level-dependent features for decoding:
                  Dithering:               levels 5 to 9
                  Posterization:           levels 0 to 2
                  Additional techniques:   levels 4 to 9
  -j num      Number of parallel jobs to speed up the conversion process.
              Valid numbers: 0 (autodetect), 1..256 (Default: 0)
  -T          Treat unrecognized input files as headerless TIS.
  -I          Show file information and exit.
  -V          Print version number and exit.

Supported input file types: TIS, MOS, TBC, MBC, TIZ, MOZ
Note: You can mix and match input files of each supported type.

Note: The tileconv executables have been compressed to reduce file size. Some overzealous antivirus tools may trigger false alarms because of this.

Click here to download this file

[Argent77]

In contrast to the old and known tispack which uses jpeg compression to reduce size, tileconv applies methods originally developed to compress textures.

The main advantage of tileconv over tispack is the ability to preserve complex details much more faithfully. You can compare the respective quality in the following images, taken from various TIS files. For simplicity I have labeled the image with numbers (1: original image from the game, 2: compressed with tileconv, 3: compressed with tispack using default quality, 4: compressed with tispack using a quality that produces files of comparable size).

This is a section of the BG2 Harper building:

Spoiler

The entrance to the Umar Inn:

Spoiler

More examples in the next post.

Edited by Argent77, 30 August 2014 - 08:30 AM.

[Argent77]

The Oasis on the way to Amkethran:

Spoiler

 

 

 

Underdark area in Icewind Dale 2:

Spoiler

 

 

 

More examples in the next post.

[Argent77]

Underground cave in Icewind Dale:

Spoiler

 

 

 

 

Another section from the same underground cave in Icewind Dale:

Spoiler

 

 

 

More examples in the next post.

[Argent77]

And last but not least, inside of the Great Foundry in Planescape Torment:

Spoiler

 

 

 

Edited by Argent77, 30 August 2014 - 08:34 AM.

[Yovaneth]

Just as I was starting to get annoyed with tispack's poor performance with details...

[The Imp]

How much is this really going to effect the size of the archive, realistically ? I would want an example if you got one.
This especially if we cut and paste the .tis files directly from the mod folder to the game folders filing system... so no copy pasting.

Edited by The Imp, 31 August 2014 - 03:05 AM.

[Argent77]

Compression size varies a lot depending on the sources. During my tests I have compressed all MOS and TIS files from BG, BG2, IWD, IWD2 and PST. The Compression size was as low as 3% and as high as 39%. Because of the algorithm used to compress files, it will never exceed 40%. The only downside at the moment is the slow encoding and decoding speed, but I'm already trying to solve this issue. (Edit: Already solved.)

@Jarno You want examples? Here they are:

Map name                              TIS (in bytes)  TBC (bytes / % of TIS)  Zipped TIS (bytes / % of TIS)  Zipped TBC (bytes / % of TIS)
Athkatla Docks (AR0300)                     26982424         8917965 / 33.0%               18711363 / 69.3%                8916762 / 33.0%
Council of Six Interior (AR1002)             2688024          420658 / 15.6%                 795337 / 29.5%                 411593 / 15.6%
Druid Grove (AR1900)                        23490584         8635792 / 36.7%               19773250 / 84.1%                8638342 / 36.7%
Trademeet (AR2000)                          18595864         6904333 / 37.1%               16254220 / 87.4%                6906219 / 37.1%
Watcher's Keep (AR3000)                     10388504         3517325 / 33.8%                7847019 / 75.5%                3517915 / 33.8%
Throne of Bhaal (AR6200)                     6799384         1885648 / 27.7%                3101246 / 45.6%                1885415 / 27.7%
Wilderness outside Saradush (AR6400)         5990424         1438101 / 24.0%                5202229 / 86.8%                1438537 / 24.0%

As you can see, TBC files can't be compressed much more because they're using a similar compression technique internally. But there are tricks to increase the compression ratio even further.

I've also tested it on my own mod and the overall file size of the 7zip archive dropped from 27 MB to 19 MB. The mod contains 14 MOS files and 5 TIS files with a total size of 34 MB.

Edited by Argent77, 01 September 2014 - 05:01 AM.

[The Imp]

Compression size varies a lot depending on the sources. D....

Obviously. Also if we use different archive tool, we might get different results as well. But I don't know if the non -zip archive tools(.rar, .7z, ) lessen the quality of the picture, but that seems doubtful.
Thanks for the example. Now, then... ... see the .tbc looses it's efficiency partially little to the ideal .tis packaging where the mod folder is emptied of the excess files, aka the .tis files are moved to the game folder instead of copying them over. So the end result taken from the last line(ar6400):
Download space in .zip archive file size 5990424:  5202229 vs. 1438537 (bytes, duh), difference per original file: ~62.8% . Or in other words the .tbc is about 3 times faster to download.
Difference on the final install folder with the archive: (5990424+5202229) vs. (5990424+1438101+1438537) => 11192653 vs. 8867062 => 186.8% vs 148.0 % =>
So the complete difference is: 38.8% for the space usage of the .tbc files.
The fact that the archive can be then deleted from the computer, brings a grim note here. 0 vs. 1438101 => -24.0%

All this says that it might be better off to, for me in my opinion using the original .tis files, but that's cause I don't mind the extra download size... but whatever...

And now, I thank you for the entertainment.

Edited by The Imp, 31 August 2014 - 08:01 AM.

[Daulmakan]

Seems great, Argent. Kudos.

[Argent77]

Version 0.2 has been released!

 

It provides a number of new tweaking options, and conversion speed has been greatly increased, depending on the number of available cores of your CPU. Additionally I have tweaked several quality features to increase speed even further. On my system (Intel Core i7, Quad-Core) conversion times have been decreased to 20%.

[Argent77]

webp looks definitely interesting, provided the lossy version meets certain quality standards. But it will probably lose some of its efficiency when applied to tile based graphics formats.
 

[Sam.]

1: BC1/DXT1 (Default)
2: BC2/DXT3
3: BC3/DXT5

 

Are these the underlying compression algorithms used in the .PVR format in the Enhanced Editions?

[Argent77]

Yes, they are. At least in the Windows and Mac versions. The PVR files in mobile versions are using some proprietary texture compression format.
 

[The Imp]

Yes, they are. At least in the Windows and Mac versions. The PVR files in mobile versions are using some proprietary texture compression format.

*Throws the Sam.'s wrench to Argen77* -Are you sure those are not copyrighted ? & They are ok with it ? Just asking cause I would hate to see you & the community to loose ...

[Argent77]

I'm NOT touching the (proprietary) PVR container format in any way. The library to encode and decode BCn/DXTn compressed pixels is open source.

[The Imp]

I'm NOT touching the (proprietary) PVR container format in any way. The library to encode and decode BCn/DXTn compressed pixels is open source.

*and the boomerang formed wrench goes around Argent77's head and circulates back to the wonderfully surprised Imp's face, that's wickedly smiling to find it's friend coming at it*

Auch, erhm... THANKS.

Edited by The Imp, 01 September 2014 - 02:17 PM.

[Solaufein]

That is lovely. I have an 800MB mod and most of those megabytes is TIS files. So this would be helpful. Where is the instructions on how to use it?

[Argent77]

You can find the usage described in the included README and also in the first post of this thread. It should be pretty self-explanatory. The tool detects the right format automatically and converts it into its counterpart (i.e. TIS↔TBC and MOS↔MBC). For example, calling "tileconv test1.tis test2.mos test3.tbc test4.mbc" will result in the files test1.tbc, test2.mbc, test3.tis and test4.mos.

TileConv's defaults should be sufficient for the majority of cases, so you'll rarely need to specify any of the available options (except maybe option -z to produce compressed MOS files). Do you have any specific questions?