1168 replies to this topic

[Argent77]

Update: NearInfinity v1.36-20160112

Note: Because of a bug in Near Infinity this release won't be detected by NI's own manual or automatic update check. All releases prior to this version (v1.36-20160112) are affected.

Changes:

• Added spawned actors by INI file to the Area Viewer (currently used by IWD(EE), IWD2 and PST), including separate icons to distinguish them from ARE-based actors.
• SAV resources: Added option to add new internal or external files or replace file entries. Added confirmation when deleting files.
• Added "Edit as text" option and hex edit tab to "unknown file" viewer.
• Improved consistency check for structured resources to reduce the number of incorrectly detected unused data or holes. This improvement should also reduce the number of hits in "Tools->Check->For corrupted files...".
• Fixed an issue with language selections in EET games.
• Various text search functions: Fixed an issue with the wrong line auto-highlighted in text files if the first line in the text file was empty.
• Lots of internal optimizations in structured resources.
• Improved component structure names in VEF resources.
• Fixed a bug in NI's update check routine.
• Several minor fixes.

[Fiann of the Silver Hand]

Sweet.  Thanks.

[Sam.]

• Improved consistency check for structured resources to reduce the number of incorrectly detected unused data or holes. This improvement should also reduce the number of hits in "Tools->Check->For corrupted files...".

What all does this entail?  In particular, a run of "Tools->Check->For corrupted files..." in my BGEE installation returns 22 fewer hits, all of which would have presumably been text overlaps and unused bytes in DLG files.  I took a look at IGNATI.DLG in the previous and current version of NI.  I am not familiar with the particulars of the DLG file format, but there appears to be quite a few bytes at the end of the file that the latest version of NI can no longer access.  Is this intentional behavior?  Loading and re-saving the file using DLTCEP seems to completely restructure it, so I am unable to use that to tell if the file has legitimate issues or not.
 

Edited by Sam., 16 January 2016 - 04:47 PM.

[Argent77]

Your observations are correct. The improved consistency check mainly affects DLG resources, that's why you see much less issues in these files.

A more detailed answer:
DLG files showing up unused byte entries in past releases were in most cases the result of a bug in the consistency check algorithm for (all kinds of) structured resources. It was difficult to pinpoint the actual source of the bug or I would have fixed it much sooner.

The old algorithm to create table entries was unable to correctly handle a particular (but legal) trait of DLG resources which allows to reference the same offset to trigger or action blocks in multiple states or responses. It was also legal to use the same offset and define a length of 0 to indicate that it doesn't actually reference any script data. Since all these states or triggers were pointing to the same script block location the algorithm would take the last offset entry in a (sorted) list and use it to detect holes, overlapping or unused data remaining in the file. If the size field happened to be 0 the algorithm incorrectly assumed that the script block at that location were unused and marked it as unused bytes. The more cases like this happened, the more "Unused bytes?" fields were added by the editor. A "Save" operation could have corrupted the file in this case. Afaik, this bug was present since the beginning (of recorded history) of Near Infinity.

The improved algorithm has fixed this issue. It prevents showing "imaginary" bogus data and doesn't produce corrupt files on save anymore. As a side effect the offset table of the trigger or action block has to be recreated and will therefore deviate a bit from NearInfinity's design philosophy to retain the original structure on save if nothing has been modified.
 

[Wisp]

New Resource->CRE creates a CRE with an illegal value (1000) in the selected-weapon slot. I don't know about everyone else, but I'd prefer a legal value (like 0) instead.

[Argent77]

I can do that. However, IESDP states that a value of 1000 means "fist" is selected. I think the current default value for the "Selected weapon" field is more appropriate for a blank CRE resource.

[Wisp]

Never mind, then. I didn't know 1000 was a perfectly cromulent value.

[Sam.]

The old algorithm to create table entries was unable to correctly handle a particular (but legal) trait of DLG resources which allows to reference the same offset to trigger or action blocks in multiple states or responses. It was also legal to use the same offset and define a length of 0 to indicate that it doesn't actually reference any script data.

Considering how infrequently DLGs seem to make use of this particular "trait" (just 16 out of more than 10,000 in my IE games), I am inclined to believe that this behavior, while technically legal, is unorthodox and perhaps not necessarily a recommended technique.  This appears to be backed up by the fact that decompiling and recompiling the DLG with WeiDU AND loading it and saving it with DLTCEP AND now loading it and saving it with NI all restructure it in such a way that the old NI no longer finds holes, overlaps, and unused bytes in it.  I find it interesting that while all three of the above methods remove the unorthodox behavior that NI originally complained about, they all restructured the DLG somewhat differently.  Presumably the results should be functionally identical to the original, tho.  All of this is to say, I am wondering if it might be beneficial for NI to still have a way to flag these DLG files for consideration to be restructured to not utilize methods that obviously go against the industry-standard and universally accepted construct.  Thoughts?

[Argent77]

Imho, since the (somewhat sloppy) DLG format specification allows those unorthodox structures, NI is responsible to parse them correctly. I have only noticed this particular structure in some of the Enhanced Edition DLG files, either because WeiDU used to generate DLG files of this kind at that time or modded DLG files were based on these special DLG files from the EE games. This issue has been solved in WeiDU v238 anyway (see v238 changelog), so it shouldn't be a real issue anymore.

[The Imp]

So to re-iterate, that's an actual fix ?

[Argent77]

That's correct. If Beamdog (or whoever) didn't come up with these particular DLG structures then this bug would have passed unnoticed forever.

[Sam.]

That's correct. If Beamdog (or whoever) didn't come up with these particular DLG structures then this bug would have passed unnoticed forever.

For the record, I found 2 in PST from GOG, 9 in BG2EE and 5 in BGEE.  None from Classic Adventures (built on ToB), IWDEE, ToSC, or IWD.

[Argent77]

I have found two DLG files in PST (original version) as well, but they are not related to this bug. I can (partially) fix the issues for one of the files (DCRSTTHG.DLG). The other file (DILQUIX.DLG) contains references to non-existing code blocks which can't be fixed in NI without adding ugly hacks.

The original BG1, BG2 and IWD2 don't appear to have any problematic DLG files.
 

Edited by Argent77, 18 January 2016 - 04:18 AM.

[The Imp]

I think this has been asked before, but I'll ask again: The Near Infinity's tileset view is nice, but is there a possibility that you could add a zoom function to it, especially one that would allow to see the maps in smaller sizes/aka unzoom them ? Eventually.

[Argent77]

I don't know how useful a zoom option would be for the raw TIS viewer. A TIS file is merely a sequence of 64x64 tiles. You can already set a zoom factor for maps in the Area Viewer. If you really need to zoom the TIS file you can export it as PNG and use a graphics viewer instead.
 

[K4thos]

For some reason saving EET Worldmap or BP-BGT Worldmap in Near Infinity breaks them (read out of bounds). Until it's saved the map is working in game and it can be viewed in NI. Both have been generated with BP-BGT Worldmap macros. Vanilla maps can be edited without problems. I'm wondering if it's due to flawed code used to generate WMP file from scratch or maybe there is some issue with NI. Here is a problematic WMP file in case you would find some time to check it:

https://www.sendspace.com/file/m3vqfg

 

Thanks in advance.

 

edit: I've tested it a little more and found out that the BP-BGT Worldmap becomes corrupted after saving in NI only if it is generated on top of EET Worldmap (it uses WMP file already existing in game). Editing it with NI when generated on vanilla BG2:EE works fine. Which means the problem must be with the EET Worldmap WMP file alone. Not sure what may cause it, but will try to find out.

Edited by K4thos, 25 January 2016 - 03:38 PM.

[Argent77]

I'll look into it, but from a first glance this file appears to contain invalid data. Even DLTCEP complains about it and reduces size by almost 50% on save.