Jump to content


Photo

Tile2EE - A MOS and TIS converter

TIS MOS converter

  • Please log in to reply
30 replies to this topic

#1 Argent77

Argent77
  • Administrator
  • 1419 posts

Posted 03 July 2015 - 11:08 AM

File Name: Tile2EE - A MOS and TIS converter
File Submitter: Argent77
File Submitted: 03 Jul 2015
File Category: IE Modding Tools

Tile2EE is a conversion tool for MOS and TIS files. It allows you to freely convert files between the palette-based format (MOS/TIS V1, as used by the original Infinity Engine games) and the new PVRZ-based format (MOS/TIS V2, as used by the Enhanced Edition games).

Supported conversions:
  • MOS V1 files (palette-based) -> MOS V2 files (pvrz-based)
  • MOS V2 files (pvrz-based) -> MOS V1 files (palette-based)
  • TIS V1 files (palette-based) -> TIS V2 files (pvrz-based)
  • TIS V2 files (pvrz-based) -> TIS V1 files (palette-based)
Tile2EE is aimed at modders who wish to provide their own TIS or MOS files in the format most suitable for the target game installation. The package contains the source code as well as binaries for Windows, Mac OS X and Linux. The latest source code is also available on GitHub.
 
This package also contains a collection of WeiDU functions to deal with tile2ee executables for multiple platforms and architectures, including a detailed description and example code.

Usage:
Spoiler

Note: The tile2ee executables have been compressed to reduce file size. Some overzealous antivirus tools may trigger false alarms because of this.

Click here to download this file

#2 K4thos

K4thos
  • Modder
  • 315 posts

Posted 03 July 2015 - 12:36 PM

Due to performance differences between 32-bit and 64-bit versions of the app for windows I suggest modders to implement tp2 detection for it and assigning appropriate version of the tool. Example code.

And yes. this tool is amazing. Great work, Argent.


Edited by K4thos, 05 July 2015 - 05:40 AM.


#3 ScuD

ScuD
  • Member
  • 492 posts

Posted 07 July 2015 - 07:19 PM

K4thos, does that mean you can release the EET now? :)



#4 Argent77

Argent77
  • Administrator
  • 1419 posts

Posted 08 July 2015 - 06:37 AM

K4thos, does that mean you can release the EET now? :)

Looks like there are a few more issues that have to be solved before an official release.



#5 Argent77

Argent77
  • Administrator
  • 1419 posts

Posted 08 July 2015 - 07:24 AM

New update: tile2ee-0.2

Changes:

  • Prevent overwriting input files by output files.
  • Existing output filenames will now be overwritten. (Replaces old behavior from tile2ee v0.1 where output filenames were renamed to prevent overwriting input files.)
  • Added a collection of WeiDU functions to help modders using this tool in their TP2 code along with example code.


#6 K4thos

K4thos
  • Modder
  • 315 posts

Posted 26 January 2016 - 10:20 AM

-i index (MOS only) Start index for PVRZ files. (Default: 1000) (Note: Existing PVRZ files will be detected and skipped unless option -w has been specified.)

is it possible to assign custom indexes for TIS files? I think I done goofed when it comes to prefixes in IWD-in-EET mod:
- IWD:EE - ID
- IWD2 - IW
area numbers are unchanged compared to original games, so now when I try to convert IWD2 tis files some of them overwrite IWD:EE PVRZ files :( From what I see maximum amount of PVRZ files used per area is about twenty few (even less in IWD games), so if it would be possible to start indexes for IWD2 areas from 50+ than it would solve the problem. The other benefit would be only "I" letter reserved for this mod, so better compatibility with mods that uses TISv2 compression.


Edited by K4thos, 26 January 2016 - 10:25 AM.


#7 Argent77

Argent77
  • Administrator
  • 1419 posts

Posted 26 January 2016 - 01:33 PM

Yeah, the naming convention for PVRZ files is pretty annoying. Too bad that filenames are still limited to 8 characters.

I could probably add a new option. Maybe something like this:

Options:
  -p page       (TIS only) Starting page number for PVRZ files. (Default: 0)
                (Caution: Total range of this value is limited to 0..99.)


Btw, PVRZ indices can go up to 999 as long as a map doesn't use separate night maps. The only restriction appears to be the filename length.

 



#8 K4thos

K4thos
  • Modder
  • 315 posts

Posted 26 January 2016 - 02:58 PM

I could probably add a new option. Maybe something like this:

that would be very useful, thanks!



#9 Argent77

Argent77
  • Administrator
  • 1419 posts

Posted 15 February 2016 - 06:25 AM

New update: tile2ee-0.3
 
Note: The included collection of WeiDU function makes use of new WeiDU v239 features.

Changes:

  • Added option -p: Starting page number for TISv2-related PVRZ files.
  • Updated collection of WeiDU functions to v1.1. A detailed changelog can be found in the file a7_tools.tpa itself.


#10 K4thos

K4thos
  • Modder
  • 315 posts

Posted 05 March 2016 - 01:02 AM

works great, thanks.

 

Added option -p: Starting page number for TISv2-related PVRZ files.

would it be possible to use this option without repacking whole TIS? You know, simply changing index numbers assigned inside TISv2 file and do nothing else. If this is not possible could you please give me a hint which offset should be modified and how many bits in order to do it manually with weidu? Thanks in advance.


Edited by K4thos, 05 March 2016 - 01:07 AM.


#11 Argent77

Argent77
  • Administrator
  • 1419 posts

Posted 05 March 2016 - 02:45 AM

PVRZ references in TIS V2 files can be easily modified with WeiDU. The header structures of TIS V1 and V2 are identical, except that V2 uses a different value for the size field at offset 0x0C. The TIS V2 format isn't yet available in the IESDP, but it's very straightforward. In your case you simply need to parse through all tile definitions and update non-negative pvrz references to their new values.



#12 K4thos

K4thos
  • Modder
  • 315 posts

Posted 05 March 2016 - 04:04 AM

thanks! Here is a sample code in case someone will need to do something similar and finds this topic:

COPY_EXISTING ~AR0011.tis~ ~override~
    READ_LONG 0x8 "tiles_cnt"
    READ_LONG 0x10 "tiles_off"
    READ_LONG 0xc "tiles_length"
    FOR (cnt=0; cnt<"%tiles_cnt%"; cnt=cnt+1) BEGIN
        READ_LONG ("%tiles_off%"+"%tiles_length%"*cnt+0x0) "pvrz_page"
        PATCH_IF (pvrz_page >= 0) BEGIN
            PATCH_PRINT ~cnt = %cnt%, pvrz_page = %pvrz_page%~
            //do your patching here
        END
    END

Edited by K4thos, 05 March 2016 - 04:05 AM.


#13 -White Agnus-

-White Agnus-
  • Guest

Posted 05 March 2016 - 07:57 AM

Damn, did the same coding today... :D
 
Here is my result (with GET_OFFSET_ARRAY):
 
GET_OFFSET_ARRAY tis_v2_array 0x10 4 0x8 4 0 0 0xc
PATCH_PHP_EACH tis_v2_array AS tiles_cnt => tiles_off BEGIN
READ_LONG tiles_off pvrz_page
PATCH_IF (pvrz_page >= 0) BEGIN
PATCH_PRINT ~pvrz_page = %pvrz_page%~
//do your patching here
END
END

 

 


#14 Roxanne

Roxanne

    Modder

  • Member
  • 3564 posts

Posted 20 September 2016 - 09:55 AM

This continues a thread from G3

 

The tool tile2ee can be used to convert TIS and MOS files between the old and new format. I would recommend it to every modder who installs new tilesets in EE games. The package also contains a WeiDU library which takes care of the details in a platform-independent way.

Here are my questions:

1. I have a mod which has nearly 200 new areas of all sizes, including large outdoor areas in TIZ format and some that currently require COPY_LARGE in the tp2

2. All area related files (MOS, TIS, ARE etx) are in a subfolder /areas

3. The mod is only useable in EET, i.e. I simply would replace the current method (copying over, extracting TIZ files) by the code provided in your guidance.

4. Do I need to separate the MOS and TIS files from the rest of the area stuff (e.g. your /graphics)?

5. Do I need to provide the current TIZ files unpacked as TIS?

6. Is the  num_to_reserve = xxx essential - it would be pretty high in my case (like 400) ?

7. Is the check for num_converted and fail if wrong essential or just a verification of successful operation?

8. Do you have some idea of how long such an operation would take in the mod install?

 

Alternative - is there a way to just convert all files one time (by me) and then provide the ready files for installation? How to establish a relation between TIS and corresponding PVRZs?

 

Any other advice for such a large scale operation would be appreciated.


Edited by Roxanne, 20 September 2016 - 09:56 AM.

The Sandrah Saga

another piece of *buggy, cheesy, unbalanced junk*

 


#15 Argent77

Argent77
  • Administrator
  • 1419 posts

Posted 20 September 2016 - 11:13 AM

tile2ee is mainly intended for on-the-fly conversions of tilesets, e.g. which may be required if your mod supports both original games and Enhanced Editions. It's main function is to reduce file size of tilesets. For example, the original Candlekeep tileset (AR2600) is about 26 MB big. The PVRZ-based version is only 8 MB big while still preserving the visual quality.

 

4. Do I need to separate the MOS and TIS files from the rest of the area stuff (e.g. your /graphics)?

tile2ee is very flexible when it comes to file conversion. You can specify one or more MOS or TIS files, or folders. Folders will be searched for TIS and MOS files which will be automatically converted from one format into another, unless specified otherwise.
If you want to convert everything with a single call you can place all your TIS and MOS files into a folder and create an empty target folder.
A call like "tile2ee.exe -o my\target\folder my\source\folder" will convert all TIS and MOS files from my\source\folder and place the converted files into my\target\folder in one go. You can also specify multiple source folders if you don't want to place everything into a single one. However, you can only specify a single target folder.
 

5. Do I need to provide the current TIZ files unpacked as TIS?

TIZ and MOZ are not supported by tile2ee. You have to unpack it into regular TIS or MOS files beforehand. I would recommend to use the original tilesets if available. The TIZ/MOZ format uses jpeg compression internally which results in a noticeable loss of quality.

 

6. Is the  num_to_reserve = xxx essential - it would be pretty high in my case (like 400) ?

num_to_reserve is a parameter of the WeiDU function FIND_FREE_PVRZ_INDEX. This function is only required for MOS V2 or BAM V2 files, since both formats are referring to PVRZ files with the name MOSxxxx.PVRZ. Setting num_to_reserve to a higher value helps to find a free contiguous block of filenames big enough to fit all your files into. Most game installations should not contain any filename fragmentations, so it's fairly safe to keep the default value 1.

 

7. Is the check for num_converted and fail if wrong essential or just a verification of successful operation?

num_converted is the only way to detect whether the call of tile2ee has been successful (unless you want to check the result manually). The return value of tile2ee itself is platform-specific and can't be evaluated by WeiDU automatically.

 

8. Do you have some idea of how long such an operation would take in the mod install?

That very much depends on the target system. tile2ee makes heavy use of multithreading. Scaling is almost linear, so using tile2ee on a CPU with 8 cores will actually be about 8 times as fast as on a CPU with a single core. A TIS V1 -> TIS V2 conversion is very fast. The Candlekeep map (AR2600) took less than a second on my system. Conversion TIS V2 -> TIS V1 is much slower however. The same map took about 8 seconds to convert on my system (using 8 cores).

 

You can also use Near Infinity to convert all your maps. The main difference between both tools is how tiles are laid out on the PVRZ textures. Near Infinity will make use of the tileset dimension specified in the associated WED file (if available), so tiles are laid out pretty much like they are displayed in the game. tile2ee does not have the information from the WED, that's why tiles are simply laid out sequentially (from left to right, top to bottom). tile2ee may produce fewer PVRZ files because tiles are laid out more efficiently.

Btw, I have just noticed that the horizontal lines seen in games patched to v2.0 are not necessarily caused by the tileset format itself. It's more the way how tiles are laid out on PVRZ textures. I could reproduce the same visual glitches with PVRZ-based tilesets. I really hope Beamdog provides a fix for this issue soon.
 


Edited by Argent77, 20 September 2016 - 11:14 AM.


#16 Roxanne

Roxanne

    Modder

  • Member
  • 3564 posts

Posted 20 September 2016 - 11:56 AM

Thank you for the detailed answer.

 

Btw, I have just noticed that the horizontal lines seen in games patched to v2.0 are not necessarily caused by the tileset format itself. It's more the way how tiles are laid out on PVRZ textures. I could reproduce the same visual glitches with PVRZ-based tilesets. I really hope Beamdog provides a fix for this issue soon.

This issue was my sole motivation to consider such a major rework of my mod, it was really the only issue I found with my new areas in EET. If you say that the problem may not even be solved...I guess I will leave the things as they are (which is proven and tested) rather than to risk some new issues by applying a conversion of such magnitude.

Unless there is something I overlooked.


The Sandrah Saga

another piece of *buggy, cheesy, unbalanced junk*

 


#17 -Yovaneth as Guest-

-Yovaneth as Guest-
  • Guest

Posted 21 September 2016 - 02:26 AM

Thank you for the detailed answer.

 

Btw, I have just noticed that the horizontal lines seen in games patched to v2.0 are not necessarily caused by the tileset format itself. It's more the way how tiles are laid out on PVRZ textures. I could reproduce the same visual glitches with PVRZ-based tilesets. I really hope Beamdog provides a fix for this issue soon.

This issue was my sole motivation to consider such a major rework of my mod, it was really the only issue I found with my new areas in EET. If you say that the problem may not even be solved...I guess I will leave the things as they are (which is proven and tested) rather than to risk some new issues by applying a conversion of such magnitude.

Unless there is something I overlooked.

I do look in here on occasions, so... The pvrz tile problem has been there since the release of the EE editions. Worse, sometimes the EE editions also reject tilesets that have been decompressed from tiz to tis. The only safe solution is to provide uncompressed tis files. Not nice.

 

-Y-



#18 Roxanne

Roxanne

    Modder

  • Member
  • 3564 posts

Posted 21 September 2016 - 02:26 AM

A final question

Is there a size limit for mos or tis files installed by this method?


The Sandrah Saga

another piece of *buggy, cheesy, unbalanced junk*

 


#19 Argent77

Argent77
  • Administrator
  • 1419 posts

Posted 21 September 2016 - 03:48 AM

A final question

Is there a size limit for mos or tis files installed by this method?

Only theoretically. The number of PVRZ textures is limited to 100 (00..99) for TIS because of filename length limited to 8 characters. Since each texture can be at most 1024x1024 pixels you can store up to 25600 tiles which means a map of 160x160 tiles is probably the maximum possible dimension.

MOS is even more generous. The graphics dimensions are stored in DWORD fields (4 bytes) and the number of associated PVRZ files is limited to 100000 (MOS0000.PVRZ up to MOS99999.PVRZ).

Even if you don't want to install TIS in the new format you can still use it or the one supported by tileconv for size reasons. The compression ratio is almost as good as with TIZ/MOZ, but preserves details much more faithfully (usually without a noticeable loss of quality compared to the original). You can see example screenshots in my tileconv thread.
 


Edited by Argent77, 21 September 2016 - 03:52 AM.


#20 Roxanne

Roxanne

    Modder

  • Member
  • 3564 posts

Posted 21 September 2016 - 06:34 AM

Is there maybe a much simpler way to do what I aim at?

 

I have converted an CVSH12.tis file in NI to PRVZ format.

I have now a new Tis file (32kb instead of 13mb of original) + 12 PRVZ files named CSH1200 to CSH1211.

Comparing this with other files, it looks like this is the naming convention to associate the PRVZs to TIS CVSH12(00...11).

Looks to me like I could convert large area graphics just offline and provide the fnished TIS + PRVZ files within my mod (given that my area names do not collide with any existing ones.) by simply copying the whole stuff to override?

 

With the new tis + prvz it seems that I no longer have the graphical issue that started the whole discussion.

 

EDIT

Seems I traded one problem for the next - areas with water do not work this way.....


Edited by Roxanne, 21 September 2016 - 06:52 AM.

The Sandrah Saga

another piece of *buggy, cheesy, unbalanced junk*

 






Also tagged with one or more of these keywords: TIS, MOS, converter