Jump to content


Photo

Lionheart-Extracting the Maps


  • Please log in to reply
118 replies to this topic

#81 Balder

Balder
  • Member
  • 16 posts

Posted 24 August 2009 - 01:47 AM

That's truth Sam. And what he means is that this is going to take some time, I have to figure out a bunch of formats used by the game for the maps (mentioned above) and put them together based on the info in the .zax.

Thanks for the colaboration Sam I'm sure people will find your recopilation very useful. (Specially when using the tool is a bit awkward cause I didn't have time to code a GUI with buttons to open files, batch convert and such things)

As I previously said frm type 68 is now known too, and before I update the tool (i'm lacking time right now) I'll explain a bit:

Header: The same as the one in frm16 type 64 with a little diferrence:
-Instead of ending at offset 36, frm16 type 68 has 4 extrabytes at the end, which represent pixel data size (repeated, as frm16 type 64 already has that entry in it's header).

The first thing we notice is the mentioned entry (pixel data size) indicating a size of 37573. If you multiply width*height you'll get 779*339= 257070 which is much more bytes than the specified by pixel data, even if pixel data is a byte per pixel it's still much less than the actual pixels of the image. So we guess this is compressed somewhat.

Then we have actual pixel data, starting at offset 40, something like this in hex:
3F0000 3F0000 3F0000 3F0000 3F0000 3F0000 3F0000 3F0000 3F0000 3F0000 3F0000 3F0000 170000

As you can see there's a pattern there, and knowing that the first pixels of the image will be in black (we guess again) and that frm16 type 64 uses 16-bit colour representation (2bytes) you see that the upper hex numbers are just 3 bytes, the first one a instruction the two second the 16-bit colour. So, what they tell is 3F = 63 decimal -> Write 63 times colour 00 00 (RGB 00 00 00 -> black)

We find the same groups of three bytes 12 times. So what we actually are doing is writing colour black 12*63 times = 756 times. Mmm. isn't that very close to the widht entry value, which is 779?

Let's see the last instruction 17 00 00: It's clear that it instructs us to write black 17hex=23dec times. Voilá! 756+23= 779. That's the image width and so we have the first line of the image.

What we have here is a manner of Run Lengh Encoding (RLE) and that's why the .txt associated to this images have RLE = 1 (on)

Of course I didn't read the full image which is very big, and the total pixel size is not divisible by 3 so sure we'll find some more instruction code, surely one which says "this part doesn't has RLE" as RLE losses compression if it's used on different pixels (there must be more than 3 equal pixels to have compression benefits from RLE) Anyway this don't worry me as now we've figured the format we can look at it knowing what he wants to do and how do you expect it to do.

I'm updating the pixel writing algorithm for this type of frms:
if( frmType == 64 )    {        fseek( fichero, 36, SEEK_SET );        for( int j = 0; j < height; j++ )        {             for( int i = 0; i < width; i++ )             {                  fread( &pixel, sizeof(pixel), 1, fichero );                  fillRect.x = i;                  SDL_FillRect( spfcePrincipal, &fillRect, pixel );             }             fillRect.y = j;        }     }    else if( frmType == 68 )    {         fseek( fichero, 40, SEEK_SET );         for( int j = 0; j < height; j++ )         {              for( int i = 0; i < width; i++ )              {                   fread( &byte, sizeof(byte), 1, fichero );                   fread( &pixel, sizeof(pixel), 1, fichero );                   for( int count = i; count < i+byte; count++ )                   {                        fillRect.x = count;                        SDL_FillRect( spfcePrincipal, &fillRect, pixel );                             }                   i += byte;              }              fillRect.y = j;         }     }

We'll see when I get this to work (or when I have time to actually do it, now I have to study hehe)

Edited by Balder, 24 August 2009 - 01:49 AM.


#82 Sam.

Sam.
  • Administrator
  • 1339 posts

Posted 10 October 2009 - 09:09 AM

We'll see when I get this to work (or when I have time to actually do it, now I have to study hehe)

Any luck? I'm very anxious to see this endeavor progress.

"Ok, I've just about had my FILL of riddle asking, quest assigning, insult throwing, pun hurling, hostage taking, iron mongering, smart-arsed fools, freaks, and felons that continually test my will, mettle, strength, intelligence, and most of all, patience! If you've got a straight answer ANYWHERE in that bent little head of yours, I want to hear it pretty damn quick or I'm going to take a large blunt object roughly the size of Elminster AND his hat, and stuff it lengthwise into a crevice of your being so seldom seen that even the denizens of the nine hells themselves wouldn't touch it with a twenty-foot rusty halberd! Have I MADE myself perfectly CLEAR?!"

--<CHARNAME> to Portalbendarwinden

--------------------

post-10485-0-15080600-1348188745.jpg
___________Old pen and paper modules of the 70s and 80s.___________

CA Forums CA Homepage


#83 Edheldil

Edheldil
  • Member
  • 8 posts

Posted 15 October 2009 - 11:43 PM

That's truth Sam. And what he means is that this is going to take some time, I have to figure out a bunch of formats used by the game for the maps (mentioned above) and put them together based on the info in the .zax.


I have decided to help a bit, so I started to look at the other file formats. Most of them are either textual object dumps (*.zax, *.txt, ...) or what seems to be binary object dumps (*.mdl16, *.way). I have created some parsers for the former, which is important to be able to understand the latter. I will post it on my wiki when I get them to a presentable state.

Speaking of that, I created a wiki and git repository dedicated to Lionheart data documentation project. There's not much info at the moment, as I still try to refine the tools, but I hope to show something useful soon.

Edheldil

#84 Balder

Balder
  • Member
  • 16 posts

Posted 20 October 2009 - 12:05 PM

Great! If you start figuring out .zax and .txt files and documenting what we already have we'll be much more productive!

To Sam:
I had not time to continue the project because of university, but now (today) I decided to resume this in my little free time. At the cost of some social life hahah

So, if we have a little luck, I'll post the final tool one of these days. After that, I'll move on to another format, .mdl16, .seq16, .way, starting with the one I find easier. Anyway I will keep the topic updated so people don't work on the same things and the work keeps well-distributed.

See you soon ^_^

#85 Edheldil

Edheldil
  • Member
  • 8 posts

Posted 21 October 2009 - 11:53 PM

Great! If you start figuring out .zax and .txt files and documenting what we already have we'll be much more productive!


Actually, I am not looking into the meaning of the fields yet, just the files' syntax. I have semi-working parsers for the most of text files of all kinds (they are just textual class dumps), for *.WAY files (they are binary class dumps) and for headers of *.MDL16 files (binary class dumps too, up to +- the POLY2D string). If you want access to my wiki, write me to Edheldil at eowyn dot cz.

E.g. this is how Rock1.mdl16 "header" looks: (plus Dynamic Properties, not shown here)
CMDL16_bin
{
    Legacy Name=Rock 1
    unk1=0
    unk2=0
    unk3=CEntityBase
    {
        Name=
        Child List=Array
        {
            Item Count=0
        }
        Visible=1
        Collideable=1
        Half Height=1
        Full Height=0
        Tries To Collide=0
        Has Hit Points=0
        Stationary=1
        Active=1
        Is Temporarily Excluded=0
        Is Marked For Deletion=0
        Activity=Array
        {
            Item Count=0
        }
        Category=
        Team Number=4294967295
        Used In=QuestMode
        Current Target=
        Publisher=
        Model=!Unknown Model
        Position X=0.0
        Position Y=0.0
        Rendering Height=0
        Rendering Height Float=0
        Cur Sequence=Idle
    }
    unk4=!Unknown Model
    unk5=!Unknown Model
    unk6=0    unk7=0    unk8=0    unk9=0    unk10=!None    unk11=0    unk12=0    unk13 Action=CPlaySoundAction    {        Sound=Error.wav        Position=$Trigger        Multiplayer=Heard by all players        unk=4294967238        unk Sound1=WEAPONS/Laser Bounce.ogg        unk Sound2=WEAPONS/Laser Bounce.ogg        unk Sound3=!None.wav    }    unk14=!None    unk15=1140457472    unk16=1127481344    unk17=1065353216    unk18=1036831949    unk19=0    unk20=1036831949    unk21=Idle    hdr=POLY2D}

Regards,
Edheldil

#86 Miloch

Miloch

    Barbarian

  • Modder
  • 6579 posts

Posted 22 October 2009 - 09:30 AM

Good to see folks are making progress with this. If you can describe the actual structure of the resources, particularly the mdl16 files, I might be able to convert them to other formats like I did with the BAM Batcher tool. Though that was really difficult and I was working with known formats, so I can't promise anything.

Infinity Engine Contributions
Aurora * BG1 NPC * BG1 Fixpack * Haiass * Infinity Animations * Level 1 NPCs * P5Tweaks
PnP Free Action * Thrown Hammers * Unique Containers * BG:EE * BGII:EE * IWD:EE
================================================================
Player & Modder Resources
BAM Batcher * Creature Lister * Creature Checker * Creature Fixer * Tutu/BGT Area Map & List * Tutu Mod List
================================================================
"Infinity turns out to be the opposite of what people say it is. It is not 'that which has nothing beyond itself' that is infinite, but 'that which always has something beyond itself'." -Aristotle


#87 Balder

Balder
  • Member
  • 16 posts

Posted 22 October 2009 - 01:01 PM

Yeah, that's the bigger problem. But at least there's people interested so luckly the project will keep advancing.

Now the bad news, as I've "finnished" the updated tool now with the testing it was clear than my suppositions were wrong. Just the example I was working too matched the scheme I thought about RLE compression, but at the end the image produced was totally garbled.

Analyzing further examples of frm16 type 68 I believe that the compression used is jpeg, so we'll deal with chrominance and all that stuff. Additionally it has an alpha channel probably the one which is on the end-of-file-appended 4-bit palette.

Never worked with this kind of complex compression so like Miloch I won't promise anything. That's completely a computer science topic.

I'm a bit anxious now to start coding the game engine myself, using those resources. But extracting has proved to be more difficult than expected.

I'll come over here if I make any progress.

#88 Edheldil

Edheldil
  • Member
  • 8 posts

Posted 23 October 2009 - 12:23 AM

Now the bad news, as I've "finnished" the updated tool now with the testing it was clear than my suppositions were wrong. Just the example I was working too matched the scheme I thought about RLE compression, but at the end the image produced was totally garbled.

Analyzing further examples of frm16 type 68 I believe that the compression used is jpeg, so we'll deal with chrominance and all that stuff. Additionally it has an alpha channel probably the one which is on the end-of-file-appended 4-bit palette.


Would you care to put the current tool sources on the web somewhere?

I was expecting this snag, since I have read some interview with an LH developer where he was touting their custom jpeg compression. At the end of mdl16 files is something which at the first sight looks like 4byte palette, but I haven't looked into it further yet. I expect it to be the case, though.

Edheldil

#89 Balder

Balder
  • Member
  • 16 posts

Posted 25 October 2009 - 01:02 PM

The tool source. Pretty straight forward, it just does what it's supposed to do without much refinment.

In this case I used SDL as graphic layer. Nowadays I prefer SFML over SDL (it's object oriented). Just type SDL in google to download the development library. Remember to link against it when compiling and you're done.

#include "SDL.h"
#include <cstdio>

SDL_Surface* spfcePrincipal;
SDL_Event evento;
SDL_Rect fillRect;
SDL_Color colores[99999];
FILE* fichero;

int main( int argc, char* argv[] )
{
	if( SDL_Init( SDL_INIT_VIDEO ) != 0 )
	{
		fprintf( stderr, "Error inicializando el subsistema de vídeo: %s\n", SDL_GetError() );
		exit( 1 );
	}
	else
	{
		fprintf( stdout, "ˇSubsistema de vídeo inicializado correctamente!\n" );
		atexit( SDL_Quit );	
	}
	
	// Abrir archivo, read-binary
	fichero = fopen( "sample.frm16", "rb" );
	if( fichero == NULL )
	{
		fprintf( stderr, "ˇError abriendo el fichero de lectura!\n" );
		exit( 1 );
	}
	
	// Inicializar algunas variables para el archivo origen
	unsigned short width, height, pixel, paletteSize;
	unsigned int frmType, pixelDataSize;
	char byte;
	fillRect.w = 1;
	fillRect.h = 1;
	fseek( fichero, 6, SEEK_SET );
	fread( &width, sizeof(width), 1, fichero );
	fread( &height, sizeof(height), 1, fichero );
	fseek( fichero, 12, SEEK_SET );
	fread( &frmType, sizeof(frmType), 1, fichero );
	if( (frmType != 64) && (frmType != 68) ) exit( 1 ); // Salir si es un frm16 no soportado
	fread( &pixelDataSize, sizeof(pixelDataSize), 1, fichero );
	fseek( fichero, pixelDataSize+36, SEEK_SET );
	paletteSize = 0;
	while( !feof( fichero ) )
	{
		   getc( fichero );
		   paletteSize++;
	}
	
	// Inicializar colores
	fseek( fichero, pixelDataSize+36, SEEK_SET );
	for( int i = 0; i < paletteSize; i++ )
	{
		 colores[i].r = getc( fichero );
		 colores[i].b = getc( fichero );
		 colores[i].g = getc( fichero );
		 colores[i].unused = getc( fichero );
	}
	
	spfcePrincipal = SDL_SetVideoMode( width, height, 16, SDL_ANYFORMAT );
	if( spfcePrincipal == NULL )
	{
		fprintf( stderr, "Error estableciendo el modo de vídeo: %s\n", SDL_GetError() );
		exit( 1 );
	}
	
	SDL_WM_SetCaption( "FRM16 Viewer", "" );
	
	if( frmType == 64 )
	{
		fseek( fichero, 36, SEEK_SET );
		for( int j = 0; j < height; j++ )
		{
			 for( int i = 0; i < width; i++ )
			 {
				  fread( &pixel, sizeof(pixel), 1, fichero );
				  fillRect.x = i;
				  SDL_FillRect( spfcePrincipal, &fillRect, pixel );
			 }
			 fillRect.y = j;
		} 
	}
	else if( frmType == 68 )
	{
		 fseek( fichero, 40, SEEK_SET );
		 for( int j = 0; j < height; j++ )
		 {
			  for( int i = 0; i < width; i++ )
			  {
				   fread( &byte, sizeof(byte), 1, fichero );
				   fread( &pixel, sizeof(pixel), 1, fichero );
				   for( int count = i; count < i+byte; count++ )
				   {
						fillRect.x = count;
						SDL_FillRect( spfcePrincipal, &fillRect, pixel );		  
				   }
				   i += (byte-1);
			  }
			  fillRect.y = j;
		 } 
	} 
	
	SDL_UpdateRect( spfcePrincipal, 0, 0, 0, 0 ); // Actualizar pantalla
	
	if( SDL_SaveBMP( spfcePrincipal, "convertida.bmp" ) != 0 )
	{
		fprintf( stderr, "Error guardando la imagen convertida a BMP: %s\n", SDL_GetError() );
		exit( 1 );
	}	
	
	// Cerrar el archivo
	fclose( fichero );
		
	for(;; )
	{
		if( SDL_PollEvent( &evento ) == 0 )
		{
			
		}
		else
		{
			switch( evento.type )
			{
					case SDL_QUIT: 
						 exit( 0 );
						 break;
			}
		}
	}
	
	return 0;
}


#90 Balder

Balder
  • Member
  • 16 posts

Posted 19 November 2009 - 09:08 PM

Don't know if anyone has made any progress on this topic. I'll give a try to those perky frm16 of type 68 using the jpeg hypothesis very soon.

Will post again then (the project is not discontinued)

#91 leahnkain

leahnkain
  • Modder
  • 8485 posts

Donator

Posted 20 November 2009 - 08:00 PM

I too have been waiting to see if any progress has been made.

Longing for the old pen and paper modules of the 70's and 80's. Experience AD&D's greatest adventures using the infinity engine: Visit our homepage at http://classicadventuresmod.com/


#92 leahnkain

leahnkain
  • Modder
  • 8485 posts

Donator

Posted 25 November 2009 - 07:45 AM

Eagerly awaiting to hear if any progress has been made.

Longing for the old pen and paper modules of the 70's and 80's. Experience AD&D's greatest adventures using the infinity engine: Visit our homepage at http://classicadventuresmod.com/


#93 Sam.

Sam.
  • Administrator
  • 1339 posts

Posted 24 January 2010 - 01:44 PM

I just stumbled upon an internet archiving service and it occurred to me to look for the converter mentioned here:

Way back in August 2003 a guy called Galian on the Interplay Forums released a tool to convert the game files with extension FRM16 to BMP format.

The only link I can find to the program is on the website http://lionheart.cdp...ult.asp?id=news which now only exists in googlecache. (search for frm16 converter in google).

The last release mentioned there was frm2bmp0.0.3.zip

Would anyone by any chance have a copy of this floating around?

Sure enough, I found it here and here. Problem is, I can't get it to work. I tested it for viruses, spyware, and malware, and didn't find anything. Here is a version repackaged into RAR:
Attached File  frm2bmp0.0.3.rar   26.31K   365 downloads
It can supposedly convert .seq16 files to .bmp as well. Can someone else try it to see if it works for them and if it works with type 68 .frm16 files and .seq16 files.

"Ok, I've just about had my FILL of riddle asking, quest assigning, insult throwing, pun hurling, hostage taking, iron mongering, smart-arsed fools, freaks, and felons that continually test my will, mettle, strength, intelligence, and most of all, patience! If you've got a straight answer ANYWHERE in that bent little head of yours, I want to hear it pretty damn quick or I'm going to take a large blunt object roughly the size of Elminster AND his hat, and stuff it lengthwise into a crevice of your being so seldom seen that even the denizens of the nine hells themselves wouldn't touch it with a twenty-foot rusty halberd! Have I MADE myself perfectly CLEAR?!"

--<CHARNAME> to Portalbendarwinden

--------------------

post-10485-0-15080600-1348188745.jpg
___________Old pen and paper modules of the 70s and 80s.___________

CA Forums CA Homepage


#94 leahnkain

leahnkain
  • Modder
  • 8485 posts

Donator

Posted 27 January 2010 - 06:24 AM

I tried and I can't get it to work, maybe someone else could give it a shot and post their results.

Longing for the old pen and paper modules of the 70's and 80's. Experience AD&D's greatest adventures using the infinity engine: Visit our homepage at http://classicadventuresmod.com/


#95 Sam.

Sam.
  • Administrator
  • 1339 posts

Posted 04 February 2010 - 01:47 PM

I tried and I can't get it to work, maybe someone else could give it a shot and post their results.

Well, the utility came from a Polish site, so maybe one of our Polish friends can give it a try? I can upload a few files to test it on if that would help...

"Ok, I've just about had my FILL of riddle asking, quest assigning, insult throwing, pun hurling, hostage taking, iron mongering, smart-arsed fools, freaks, and felons that continually test my will, mettle, strength, intelligence, and most of all, patience! If you've got a straight answer ANYWHERE in that bent little head of yours, I want to hear it pretty damn quick or I'm going to take a large blunt object roughly the size of Elminster AND his hat, and stuff it lengthwise into a crevice of your being so seldom seen that even the denizens of the nine hells themselves wouldn't touch it with a twenty-foot rusty halberd! Have I MADE myself perfectly CLEAR?!"

--<CHARNAME> to Portalbendarwinden

--------------------

post-10485-0-15080600-1348188745.jpg
___________Old pen and paper modules of the 70s and 80s.___________

CA Forums CA Homepage


#96 McG

McG
  • Member
  • 1 posts

Posted 30 March 2010 - 04:36 PM

Hi, i have just started playing the game last weekend and was facinated by the special features it had( i liked the idea that battles were more fast fun and less of a chess game). But the role playing features were too unimportant in the combat system - so the game became boring - and old reviews left little to hope for in the story department. So i thought about trying to mode the game, surfed the web, read too much, played with the Assembly of the exe to no avail: But i got some good links and some afterthoughts.
So here are the links i have from the Lead Programmer of the game (James [C] Smith) posts:
On graphics:
http://forums.indieg...hread.php?t=668
http://forums.indieg...read.php?t=1926
http://forums.indieg...read.php?t=1371

You will probably find more by googling his name on the site(I only searched his name+lionheart).

Also in these posts:
http://forums.indieg...read.php?t=2099
http://forums.indieg...read.php?t=2326
http://forums.indieg...ead.php?p=97253
[i realy like his good will in spending his knowledge]

He says that the game had a built in Editor that was dissabled on cummercial games:

the editor was disabled with a #if at compile time setting so the game that shipped to the customers didn't have an editor in it.

So thought i might find a way of enabling it, if the code was still there - hence the Assembly fun(There are 'editor' 'Gold editor' 'editor only' and 'editor &f6&' bit strings in the Assembly code). Seeing the diffrence between 'Ricochet Xtreme'(Demo - which comes with the game and has no enabled editor) and 'Ricochet Lost Worlds' (Demo - that can be downloaded from the company site, whice has an editor: excessed from main menu or from in game by pressing f6) - I thought maybe, maybe... But i guess i kind of fought with windmills(if only because my usage on Assembly codes was good but too infrequent).
So I hope I found something useful to you guys. I realy appreaciate your devotion to code understanding - And hope the best for the effort :)

p.s.
This game zax by the same developers might also have maps you can use. It quite likely uses the same picture compression method.

Edited by McG, 30 March 2010 - 04:44 PM.


#97 Miloch

Miloch

    Barbarian

  • Modder
  • 6579 posts

Posted 02 April 2010 - 12:54 PM

Interestin'. But I think even if there is some sort of editor/converter, one would still have to place all the buildings and other objects on the background maps. They're not full-fledged static maps like in BG2 etc.

Infinity Engine Contributions
Aurora * BG1 NPC * BG1 Fixpack * Haiass * Infinity Animations * Level 1 NPCs * P5Tweaks
PnP Free Action * Thrown Hammers * Unique Containers * BG:EE * BGII:EE * IWD:EE
================================================================
Player & Modder Resources
BAM Batcher * Creature Lister * Creature Checker * Creature Fixer * Tutu/BGT Area Map & List * Tutu Mod List
================================================================
"Infinity turns out to be the opposite of what people say it is. It is not 'that which has nothing beyond itself' that is infinite, but 'that which always has something beyond itself'." -Aristotle


#98 -Jarl-

-Jarl-
  • Guest

Posted 03 May 2010 - 07:30 AM

I saw a screenshot of the "Classic Adventures" - Mod, which takes place in Barcelona Port District of Lionheart Game. Does anybody know how they extracted the area-graphics? Or is it a collage of ingame-screenshots?
Barcelona-Areas would be ideal to implement Deepwater to BG :)

Jarl

#99 Creepin

Creepin
  • Administrator
  • 1676 posts

Posted 03 May 2010 - 09:01 AM

Hi Jarl!

I'm pretty sure that asking here or PM'ing directly to Sir BillyBob will yield you better and faster answers ;)

The Old Gold - v0.2 WIP (mod for BGT/BWP/BWS)


#100 davemckay

davemckay
  • Member
  • 12 posts

Posted 07 July 2011 - 06:03 AM

.


Edited by davemckay, 13 June 2015 - 12:06 PM.