64 replies to this topic

[Miloch]

Erephine, have you seen this issue? It seems some of the special characters used in animation prefixes may not convert properly on non-Western (particularly Asian) platforms. This could be a real compatibility showstopper if it is the case, since it may require a whole lot of renaming and reengineering to work around, if that's even possible. Is there a reason for the non-standard characters?

Incidentally, were you going to upload the update? It would be good to have some testing on it so we can iron out issues like these.

[Jarno Mikkola]

This would suggest that the solution is this:

Control Panel
Regional and Language Settings in Classic View (or Clock, Language and Region then Regional and Language Options in fancy view)
Click the Administrative tab
Then where it talks about non-Unicode programs, "Change System Locale" to English

This needs to be done before the animations are installed... and after the install, it can be changed back to 'normal'.

[Erephine]

If the solution Jarno mentioned works, that would probably be the easiest way to go about it for now.

They are mainly there to make absolutely sure we don't run into conflicts with anything else, given the approach to slot identifiers.

Edited by Erephine, 17 January 2010 - 04:20 PM.

[Erephine]

Updated the download.

See the included non-ANSI notes. Obviously I haven't tested it, but compatibility should be fairly easy to patch in from the WeiDU side as well (I think, anyway) by batch renaming the content before it's copied.

[Miloch]

Since it's kind of hard to find easily, the core IA download is here. It includes everything discussed here, most importantly, WeiDU and megamod compatibility.

I had some "fun" with batch files and WeiDU to check the code page. Added the following code to t-ansi.bat, which checks your code page version and writes it to a temp file (the only way I could figure to get the OS to return a registry value that WeiDU can check).

@ECHO OFF
IF EXISTS infinityanimations\batch\t-ansi.txt DEL infinityanimations\batch\t-ansi.txt
START /W REGEDIT /E infinityanimations\batch\t-ansi.reg "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\CodePage"
FOR /F "tokens=1* delims==" %%A IN ('TYPE infinityanimations\batch\t-ansi.reg ^| FIND "ACP"') DO SET AnsiCodePage==%%B
SET AnsiCodePage=%AnsiCodePage:"=%
SET AnsiCodePage >> infinityanimations\batch\t-ansi.txt 2>&1
DEL infinityanimations\batch\t-ansi.reg

Added the following to the beginning of the WeiDU tp2 installer:

OUTER_SET t-acp = 0
ACTION_IF (~%WEIDU_OS%~ STRING_EQUAL_CASE ~win32~ = 1) BEGIN //if Windows
  AT_NOW ~infinityanimations/batch/t-ansi.bat~
  COPY ~infinityanimations/batch/t-ansi.txt~ ~infinityanimations/batch/t-ansi.txt~
    REPLACE_EVALUATE ~AnsiCodePage==\([0-9]+\)~ BEGIN
      SET t-acp = %MATCH1%
    END ~AnsiCodePage==%t-acp%~
  BUT_ONLY

  ACTION_IF t-acp != 1252 BEGIN
    PRINT ~Your ANSI code page is set to a value of %t-acp%. Infinity Animations needs it
set to 1252 during the install to enable Latin characters. Go to Start >
Settings > Control Panel and click on Regional and Language Settings (or
Clock, Language and Region > Regional and Language Options). Click on the
Advanced or Administrative tab then where it talks about non-Unicode programs,
change the language to English and click OK. After Infinity Animations is
installed, you can change it back to your original language.~
    FAIL ~Code page is set to %t-acp% instead of 1252.~
  END
END

Basically, if the code page isn't set correctly, the mod doesn't install. This is a "soft" option that tells the user to change their code page manually as mentioned in the readme if it isn't set correctly. A more aggressive option would be to set the codepage to 1252 if it isn't, then do the install, then set it back to what it was. This is quite possible, but I went with the "soft" option for starters. Would people prefer the more aggressive option so they don't have to mess with the settings manually?

An open question is whether you can play the game and see the new animations even with a non-Latin codepage (edit: even if the install is done with the codepage set to 1252). If not, it would still be an option to provide a batch file to set the code page to 1252 while playing the game, and perhaps another one to set it back to what it was.

I'm a bit leery of renaming all the animations if a different codepage is detected. It just sounds a bit messy. Wouldn't we also have to rename all the IDS entries? Apart from that, the installer uses specific BAM names for component and mod-related file checks. That would indeed be messy to recode, and I don't think it's necessary if the above options work, which seem a bit cleaner to me.

Nice and "prettified" context highlighting there, eh? I've got purple and blue text all over the place... doesn't really make it a whole lot more readable, does it?

Edited by Miloch, 18 January 2010 - 02:02 AM.

[Erephine]

Basically, if the code page isn't set correctly, the mod doesn't install. This is a "soft" option that tells the user to change their code page manually as mentioned in the readme if it isn't set correctly. A more aggressive option would be to set the codepage to 1252 if it isn't, then do the install, then set it back to what it was. This is quite possible, but I went with the "soft" option for starters. Would people prefer the more aggressive option so they don't have to mess with the settings manually?

An open question is whether you can play the game and see the new animations even with a non-Latin codepage (edit: even if the install is done with the codepage set to 1252). If not, it would still be an option to provide a batch file to set the code page to 1252 while playing the game, and perhaps another one to set it back to what it was.

No, that shouldn't work.

If the solution posted above works (which it seems to from the screenshots in the other thread), the codepage has to be set while the executable is run.

I'm a bit leery of renaming all the animations if a different codepage is detected. It just sounds a bit messy. Wouldn't we also have to rename all the IDS entries? Apart from that, the installer uses specific BAM names for component and mod-related file checks. That would indeed be messy to recode, and I don't think it's necessary if the above options work, which seem a bit cleaner to me.

Nice and "prettified" context highlighting there, eh? I've got purple and blue text all over the place... doesn't really make it a whole lot more readable, does it?

Why would it sound messy?

We don't really have to rename the IDS entries (though we could?). You'd just have to do a batch replace along the lines 'every time you find this symbol anywhere you replace it with that one'. I could write a batch script for people to run at any point after installing as well, if you prefer, but since you like keeping everything in WeiDU...

The component checks by filename would only have to be extended to cover the different prefixes (if exist A or if exist B or if exist C, etc).

Edited by Erephine, 22 January 2010 - 07:56 PM.

[Erephine]

I can't seem to attach anything (and the site seems odd today) so here is an offsite link:

-edit: actually it seems I can, even if it gives me an error

Run the file and navigate to your override folder. This should rename all IA prefixed animations in question to work with your language settings (regardless of what they're actually set to). Could anyone on a non 1252-compatible code page please test this and let us know if it resolves the problems? Note that this only works if the files have not been compressed into .bif files obviously.

The patcher accepts the override path as a command line argument as well if you prefer.

Attached Files

•  filename_nonANSI.rar   293.88K   544 downloads

Edited by Erephine, 18 January 2010 - 10:06 AM.

[Miloch]

Basically, if the code page isn't set correctly, the mod doesn't install. This is a "soft" option that tells the user to change their code page manually as mentioned in the readme if it isn't set correctly. A more aggressive option would be to set the codepage to 1252 if it isn't, then do the install, then set it back to what it was. This is quite possible, but I went with the "soft" option for starters. Would people prefer the more aggressive option so they don't have to mess with the settings manually?

An open question is whether you can play the game and see the new animations even with a non-Latin codepage (edit: even if the install is done with the codepage set to 1252). If not, it would still be an option to provide a batch file to set the code page to 1252 while playing the game, and perhaps another one to set it back to what it was.

No, that shouldn't work.

If the solution posted above works (which it seems so from the screenshots in the other thread), the codepage has to be set while the executable is run.

Eh... we're talking about the same thing (see bolded text). At any rate, either of the italicised options should work, no?

I'm a bit leery of renaming all the animations if a different codepage is detected. It just sounds a bit messy. Wouldn't we also have to rename all the IDS entries? Apart from that, the installer uses specific BAM names for component and mod-related file checks. That would indeed be messy to recode, and I don't think it's necessary if the above options work, which seem a bit cleaner to me.

Why would it sound messy?

The component checks by filename would only have to be extended to cover the different prefixes (if exist A or if exist B or if exist C, etc).

You just explained part of it. And it's not just IA that'd have to do this, but any other mods that use the new animations. Why is it necessary if the above solution works?

We don't really have to rename the IDS entries (though we could?). You'd just have to do a batch replace along the lines 'every time you find this symbol anywhere you replace it with that one'. I could write a batch script for people to run at any point after installing as well, if you prefer, but since you like keeping everything in WeiDU...

It's not just me, but as I've said, it's the mod standard whether we like it or not. But why have two installers when we can have one? I'll include it if it's necessary but I'm still not sure that it is.

[Erephine]

Eh... we're talking about the same thing (see bolded text). At any rate, either of the italicised options should work, no?

I should have quoted more precisely:

then do the install, then set it back to what it was

...will not work.

You just explained part of it. And it's not just IA that'd have to do this, but any other mods that use the new animations. Why is it necessary if the above solution works?

Because not everyone would like to change their system settings (or even know how to) each time they'd like to play BGII?

I've actually thought about this some more, and if this works, depending on how WeiDU handles filenames (is it a Unicode application?) we might not even have to change the checks. If the installer behaves the same way as the executable, the patched filenames will match after all, and if not, we could just check if they're in content/ before installing and renaming (that's all the checks are there for currently, if I'm not mistaken?).


-edit to add: Besides, if other mods went about it the smart way, they could also check for the IA component rather than an actual .BAM file.

Edited by Erephine, 18 January 2010 - 04:20 PM.

[Miloch]

Because not everyone would like to change their system settings (or even know how to) each time they'd like to play BGII?

Well, then it's possible to change it with the installer and provide a batch file to toggle it before and after gameplay. WeiDU can even generate one that's specific to the user's system (changes it back to the original language read before the install).

I've actually thought about this some more, and if this works, depending on how WeiDU handles filenames (is it a Unicode application?) we might not even have to change the checks. If the installer behaves the same way as the executable, the patched filenames will match after all.

As for that, we probably won't know until we get someone who can do some targetted testing (and we probably don't want to recode a bunch of stuff until that happens).

-edit to add: Besides, if other mods went about it the smart way, they could also check for the IA component rather than an actual .BAM file.

What I mean is that other mods which use new animations would have to handle that renaming too. Aurora, for example, is install-order-independent, which makes sense, because IA should be one of the last mods installed (and Aurora also supports SoA). So it includes just the animations and code it needs rather than requiring IA, though if IA is installed that's fine too.

[Erephine]

What I mean is that other mods which use new animations would have to handle that renaming too. Aurora, for example, is install-order-independent, which makes sense, because IA should be one of the last mods installed (and Aurora also supports SoA). So it includes just the animations and code it needs rather than requiring IA, though if IA is installed that's fine too.

That's true. You could just run the batch after installing Aurora though (it doesn't change anything on 1252 code page systems) or have a note in your compatibility section for people with non-Western languages to do so. You'd need that note for switching the user locale anyway.

It doesn't matter when it's run either, as long as the files aren't biffed (a WeiDU version could even support that).

[Miloch]

It doesn't matter when it's run either, as long as the files aren't biffed (a WeiDU version could even support that).

Unfortunately not, since WeiDU can only MAKE_BIFFs, it can't unmake or remake them. You would have to uninstall biffing first (if you have an uninstallable version like I think Generalised Biffing is) or better yet, follow the cardinal rule of not biffing your game until you're absolutely comfortable with it.

[leahnkain]

It isn't working on a Japanese computer. I am using Classic Adventures. The game still crashes when I come to a map which has a stirges animation I followed your instructions and tried it on two computers. (Please do not expect a quick reply from me, I am very busy this week)

[Erephine]

It isn't working on a Japanese computer. I am using Classic Adventures. The game still crashes when I come to a map which has a stirges animation I followed your instructions and tried it on two computers. (Please do not expect a quick reply from me, I am very busy this week)

That is very strange. Are you sure your files aren't biffed or otherwise inaccessible?

I'm asking because I just tested it with a system set to Japanese in the non-Unicode tab and can confirm it's working for me. Crashes without running the batch rename, loads fine with it.

The animation I tested was the green abishai, if it helps.

[Sir BillyBob]

green abishai

Unfortunately, we haven't used that animation yet. So there aren't any CRE files to spawn in to test.

I was hoping to have more players test this bug before telling you about it. However, it seems that we only have 3 players who have experienced the problem. The languages in question are Russian, Chinese, and Japanese. Jarno's fix listed above is the first one I have seen that comes close to solving the problem.

The CA files are biffed. Leahnkain uses Vista and he plus the other 2 players have mentioned that CA errors during part of the copying process. I suspect it is when Weidu copies the BAM files into the override folder and then into the temp folder for biffing.

Could it be that a player would need to convert their locale settings before unRARing the files? Since the BAM files would already have a character the computer doesn't like, would setting the locale first help any?

[Erephine]

I don't think so.

The problem is, the Baldur's Gate executable uses the non-Unicode system code page to locate game resources.

Think of it this way: The executable contains a set of numbers, which are then looked up in your set code page to translate them to the characters of the filename. Different code pages do not resolve these numbers the same way, so the exact same exe reference links to different filenames depending on how you set your system locale.

Windows (XP+ anyway) does support Unicode-like filenames, so the original IA animation .BAM names should definitely be preserved upon copying or unRAR-ing. The game just won't find them because it goes by the older non-Unicode standard. Ironically enough, if the operating system did not store the filenames in Unicode, this problem would not occur.

What the patcher I posted above does is, it goes through all BAM/PLT/2DA files it finds (i.e. not biffed ones) and renames them to match the .exe references for your currently set locale.

For instance the IA identifier with the value B5 translates to 'µ' on 1252 code pages, so the files will be named like that (the game finds and loads the file µXX.BAM). If you set the non-Unicode locale to Japanese, however, the same number resolves to the symbol 'ｵ' instead. So the game doesn't even check for µXX.BAM, but tries to find ｵXX.BAM which doesn't exist (and crashes).

The patcher locates µXX.BAM and renames it to ｵXX.BAM. No crash.

It might be that WeiDU has the same problem as the .exe. In that case, you'd have to run the patcher on the IA content before trying to install. It should work for any locale, and any animation as far as I can tell.

I hope that made sense.

Edited by Erephine, 19 January 2010 - 02:53 PM.

[Miloch]

To repeat what you've already said though, none of that will work if the files are biffed.

The CA files are biffed.

To test this properly, someone would have to comment out the biffing from CA's install then run your tool, or somehow include it in CA's install before the biffing. It could probably be included using a variant of the code I posted above (would only run if the code page isn't set to 1252).

[Sir BillyBob]

At the end of the Weidu installer, I call a CA_install.bat file to unpack the worldmap, copy the Setup.exe to Weidu.exe and run all the biffing. At the top of that file, I also run the IA installer.

So... in theory, I could put your check prior to the IA part. That would fail out the whole batch file so the player could update their settings. They could then just double-click on the BAT file to rerun the process and install IA and biff everything. The player would still need to remember to set the locale back. Otherwise, I think that may be the best solution without making setting changes in the background.

[Erephine]

So... in theory, I could put your check prior to the IA part. That would fail out the whole batch file so the player could update their settings. They could then just double-click on the BAT file to rerun the process and install IA and biff everything. The player would still need to remember to set the locale back. Otherwise, I think that may be the best solution without making setting changes in the background.

Except it wouldn't work, as the locale would still need to be set when the game is run.

Changing the non-Unicode code page requires a reboot by the way.

Edited by Erephine, 20 January 2010 - 07:51 PM.

[Erephine]

It might be that WeiDU has the same problem as the .exe.

^ I've just confirmed that, which means we don't actually have to edit file name checks. If you batch rename the content before installing, all should be fine.

--

Updated the core download with non-ANSI support patched in.

IA should now install properly with any system locale (as long as content is handled via the installer).

Edited by Erephine, 22 January 2010 - 05:28 PM.