Javascript Disabled Detected

You currently have javascript disabled. Several functions may not work. Please re-enable javascript to access full functionality.

[SOLVED]Time to deal with the stuttering

Started by Suslik, Apr 11 2012 06:08 PM

Next
»

Please log in to reply

135 replies to this topic

#1 Suslik

Investigator

Member
500 posts

Posted 11 April 2012 - 06:08 PM

Mystery is solved and solution is included into TobEx 0.24! If you are not willing to use TobEx, I have provided a script-side workaround.

Description of the problem

Ok, serious mode on. I have encountered this issue since BWP 9.0 and every time *that* starts I am forced to restart the game/do or do shaman-like things like loading a few other savegames. I am not posting WeiDU log, because, the problem was both in BWP 9.0, 9.1, 9.4, 9,6 standard and now in 10.3 tactical. All mods are from the preset. Actually it can even appear in a lightly-modded or vanilla game, but in later stages. It affects BG1, SoA, ToB, BGT in the same way.

We all know that there are different kinds of stuttering: containers sometimes cause the game to stutter, bugged scripts, a lot of stuff in "override", that weird kind of stuttering which vanishes after loading a few games you played before and so on. Now I am going to discuss a particular kind of stuttering related to a strange state, when regular scripts start executing slower than they normally do.

When you play and *this* kind of stuttering starts, symptoms are following:
- It seems to start after about 10-25 hours of gameplay in SoA. The more mods you have installed, the less it takes to start stuttering.
- The game goes for about 1 second, then freezes for about 0.5 seconds. Goes for 1 more second, freezes for 0.5 more seconds.
- It gradually becomes worse and worse. If I continue playing, it will freeze for 1 second and probably more.
- Modded game especially suffers from such lags, but they can appear in later stages of vanilla too.
- Only happens when the game is unpaused. Everything goes smooth when paused.
- Debugging the scripts shows nothing. It just seems like some scripts/blocks start executing about 100 times slower(actually they can be executed up to 1024 or 2048 times slower).
- The game will run faster if you load earlier savegame.

Reason of such stuttering
GLOBAL variables which are used in various scripts are stored inside a hashmap which is resized only when it's full. When there are many variables and hash table is nearly full, its performance degrades to O(n) instead of average case O(1). As a result under certain point of gameplay, when number of variables is close to 1024/2048 or even 4096, some GetGlobal's will be executed in linear time instead of O(1). And some script variables will be accessed by far longer than the others.

As a workaround you can try this:

Spoiler

Solution
I have implemented my own hashmap and injected it into BGMain via Ascension64's TobEx. I have removed the link since Ascension64 has implemented pretty the same thing in his TobEx 0.24.

Edited by Suslik, 23 April 2012 - 05:08 PM.

Howto: mage in Baldur's Gate. My lame guide.
List of issues I have encountered in BWP 10.3 tactical
Stutter-fixer for Infinity Engine
While you were reading this signature, I have probably edited the post above. Twice.

Back to top

#2 -max-

Guest

Posted 11 April 2012 - 09:08 PM

Hi Suslik, I too have encountered these issues so I wish you luck with this - it sounds like you know something about scripts and things. Jaheira stuttering would happen after she left the party the first time iirc. and might clear after you clean out the Harper base. Aerie's stutter I couldn't figure out either (and also tried the things you mentioned), and ended up cntrl-j all over the place. In the end I didn't keep her long enough to romance, or got it over with as soon as possible, then dump her.

Never tried Imoen mod, but tried Nalia once (way too much talking with her, sorry). Also I did not have multi romances installed, so that's not the issue.

Back to top

#3 Suslik

Investigator

Member
500 posts

Posted 12 April 2012 - 08:17 AM

Finally, some good news. After a few hours of investigation I have figured out following:

1) NPC stuttering code is fully in baldur.bcs. Erasing all code from it makes the stuttering vanish.
2) It seems to be impossible to localize the exact location of problematic code, since it seems to be spread throughout the whole Baldur.bcs. I have started dividing the whole code from Baldur.bcs by halves and sometimes when I divided large chunks of code(somewhat about 5000 lines) stuttering started lowering, but refused to vanish completely.

Even more. It seems that NPC stuttering is not bound to a single NPC and has nothing to do with romances. But it does have something to do with PartyHasItem checks!

Take a look. I have erased all code from baldur.bcs and replaced it with afollowing block:

IF
	True()
	PartyHasItem("KovaArm5") // Haeloven +2 //some random item. does not matter actually
	PartyHasItem("KovaArm5") // Haeloven +2
	//about 100 of the same lines
THEN
	RESPONSE #100
		SetGlobal("TestValue", "GLOBAL", 1)
END

It started to lag exactly the same way! And I have figured out even more:
My current party is following:
<charname>, Nalia, Tashia, Kiara, Sheena

If I kill Tashia and/or Kiara, nothing happens.
If I kill Nalia OR Sheena, stuttering vanishes! But if I remove them(dead or alive) from the party, stuttering starts anew.

Apparently it seems that some NPC's under unknown circumstances start executing PartyHasItem extremely slowly. But how in the nine hells is it possible to kill only one out of two such NPC's to make the stuttering disappear? Why does the second one not stutter the game?</charname>

Edited by Suslik, 12 April 2012 - 08:28 AM.

Howto: mage in Baldur's Gate. My lame guide.
List of issues I have encountered in BWP 10.3 tactical
Stutter-fixer for Infinity Engine
While you were reading this signature, I have probably edited the post above. Twice.

Back to top

#4 William Imm

Obsessive Penguin Lover

Member
486 posts

Posted 12 April 2012 - 08:27 AM

I think TobEx has a component that optimizes the PartyHasItem() checks. By any chance is that enabled?

At this point, I'm not really doing much Baldur's Gate related. More focused on Skyrim modding and the Born of Legend tabletop roleplaying game. Don't expect much activity here.

Back to top

#5 Suslik

Investigator

Member
500 posts

Posted 12 April 2012 - 08:37 AM

I think TobEx has a component that optimizes the PartyHasItem() checks. By any chance is that enabled?

Um.. I have never heard of it. I have version 23 of TobEx and cannot see anything regarding to such fix.

If you mean "Optimise Bag Search Code" then it does not help, since problematic NPC's do not even have any bags =(

Edited by Suslik, 12 April 2012 - 08:53 AM.

Howto: mage in Baldur's Gate. My lame guide.
List of issues I have encountered in BWP 10.3 tactical
Stutter-fixer for Infinity Engine
While you were reading this signature, I have probably edited the post above. Twice.

Back to top

#6 Galactygon

Modding since 2002

Member
938 posts

Posted 12 April 2012 - 09:17 AM

If this is true then the BGEE team should be notified. This (as well as the higher resolution, faster game speed) alone will be worth the buy.

-Galactygon

Edited by Galactygon, 12 April 2012 - 09:24 AM.

Back to top

#7 Suslik

Investigator

Member
500 posts

Posted 12 April 2012 - 09:26 AM

Galactygon, I doubt it is the most complicated issue we have ever faced in the infinity engine. Now that we know where to look, I think it's possible to address the problem to almighty Asc64. He even has already externalized bag search algorithm and and he is probably familiar with PartyHasItem() code. If he will not have enough time for such investigation, I can try and debug the executable myself, but it will take far more time to reverse the code and track the bug down.

Edited by Suslik, 12 April 2012 - 09:29 AM.

Howto: mage in Baldur's Gate. My lame guide.
List of issues I have encountered in BWP 10.3 tactical
Stutter-fixer for Infinity Engine
While you were reading this signature, I have probably edited the post above. Twice.

Back to top

#8 Suslik

Investigator

Member
500 posts

Posted 12 April 2012 - 11:38 AM

Well, I did figure out why killing *some* NPC's make the stuttering disappear. Following blocks in baldur.bcs keep firing endlessly:

IF
	!InParty(Player2)
	Global("USSCBloc2","GLOBAL",1)
THEN
	RESPONSE #100
		SetGlobal("USSCBloc2","GLOBAL",0)
END
//===========
IF
	Class(Player2,MAGE_THIEF)
	OR(2)
		Kit(Player2,WILDMAGE)
		Kit(Player2,TRUECLASS)
	Global("USSCBloc2","GLOBAL",0)
THEN
	RESPONSE #100
		SetGlobal("USSCBloc2","GLOBAL",1)
		ApplySpellRES("LI#BLMT",Player2) // No such index
END

Apparently it happens only when there's a mage/thief(Nalia) and they're Player2 and they're dead. When these blocks are fired, they prevent anything else below from being fired, so the stuttering code is never executed. When Sheena(Fighter/mage) is dead, this code is looped:

IF
	!InParty(Player5)
	Global("USSCBloc5","GLOBAL",1)
THEN
	RESPONSE #100
		SetGlobal("USSCBloc5","GLOBAL",0)
END
//======
IF
	Global("USSCBloc5","GLOBAL",0)
	OR(5)
		Class(Player5,MAGE_THIEF)
		Class(Player5,FIGHTER_MAGE)
		Class(Player5,CLERIC_MAGE)
		Class(Player5,FIGHTER_MAGE_THIEF)
		Class(Player5,FIGHTER_MAGE_CLERIC)
	!Kit(Player5,MAGESCHOOL_ILLUSIONIST)
THEN
	RESPONSE #100
		SetGlobal("USSCBloc5","GLOBAL",1)
		ApplySpellRES("LI#BLIL",Player5) // No such index
END

This code is from mod Refinements and although it does not add stuttering to the game, it is an error which prevents most of Baldur.bcs to be skipped under certain circumstances and led me to wrong conclusions :/

Howto: mage in Baldur's Gate. My lame guide.
List of issues I have encountered in BWP 10.3 tactical
Stutter-fixer for Infinity Engine
While you were reading this signature, I have probably edited the post above. Twice.

Back to top

#9 Suslik

Investigator

Member
500 posts

Posted 12 April 2012 - 11:48 AM

I am terribly sorry for this fountain of untested clues, but I hope to give as much information as I can:

Apparently the stuttering has nothing to do with those certain NPC's, because killing them only caused most of Baldur.bcs to be skipped. So problem is somewhere below those blocks.

According to bigg's advice here: http://www.shsforums...ing-stuttering/ I have uncovered that NO blocks are fired during the stuttering. Apparently the troublesome code is somewhere inside the IF- conditions, not inside the RESPONSE# blocks.

Howto: mage in Baldur's Gate. My lame guide.
List of issues I have encountered in BWP 10.3 tactical
Stutter-fixer for Infinity Engine
While you were reading this signature, I have probably edited the post above. Twice.

Back to top

#10 Sasha Al'Therin

Modder
615 posts

Posted 12 April 2012 - 01:12 PM

According to bigg's advice here: http://www.shsforums...ing-stuttering/ I have uncovered that NO blocks are fired during the stuttering. Apparently the troublesome code is somewhere inside the IF- conditions, not inside the RESPONSE# blocks.

how so? If any one of a block's conditions are false then that block is skipped. If all of the conditions are true then it will parse. Did you check for stuttering on all scripts? besides baldur.bcs there are the dplayer scripts which run constantly so long as party ai is on. see if your stuttering issue goes away with the party ai off...

Also note that the Bigg's code to prevent stuttering can also cause false stuttering itself, but that's due to the display string being placed at the start of the action list instead of the bottom. Can't place it at the bottom since some script blocks have multiple RESPOSNEs. but only one END.

Perhaps if you posted the script(s) that you suspect are the problem (place it inside the spoiler tag if you do) it'll be easier for anyone else to assist you.

My working mods:
an AI Party Script for BG2 game engine DOWNLOAD LINK ONLY!
Interactive Tweaks for BG series with some IWD support. DOWNLOAD LINK ONLY!
Rest For 8 Hours an IWD mod
-------------------------------------------
My contributions: BG1Fixpack, BG1Tweaks
On Hold: Solestia an NPC for SOA
-------------------------------------------
My website: http://sasha-altheri...s.com/index.htm

Back to top

#11 Suslik

Investigator

Member
500 posts

Posted 12 April 2012 - 01:38 PM

Perhaps if you posted the script(s) that you suspect are the problem (place it inside the spoiler tag if you do) it'll be easier for anyone else to assist you

Thanks for assistance, but I'm currently debugging Baldur.bcs which is about 40000 lines. I doubt anyone would want to debug it : D And I still have some ideas to investigate myself.

If any one of a block's conditions are false then that block is skipped. If all of the conditions are true then it will parse.

Checking conditions is not free. Some conditions like PartyHasItem() are quite expensive just to check. Even though this particular one is fixed in latest TobEx, there are still other possibly troublesome conditions.

besides baldur.bcs there are the dplayer scripts which run constantly so long as party ai is on

Problematic code is 100% in Baldur.bcs. Remeber that NPC bug I discussed earlier? Under certain conditions involving dead NPCS parsing of Baldur.bcs terminated at approx 1/4 of it and all stuttering completely went away. If I manually erase Baldur.bcs, stuttering is gone too.

I will investigate the issue further until I run out of ideas and then will ask for community's help. I do not want to overflow the forum with excess of untested ideas.

Howto: mage in Baldur's Gate. My lame guide.
List of issues I have encountered in BWP 10.3 tactical
Stutter-fixer for Infinity Engine
While you were reading this signature, I have probably edited the post above. Twice.

Back to top

#12 Sasha Al'Therin

Modder
615 posts

Posted 12 April 2012 - 01:58 PM

I'm currently debugging Baldur.bcs which is about 40000 lines

maybe that in and of itself is the problem. just too much to process at one time.

Checking conditions is not free. Some conditions like PartyHasItem() are quite expensive just to check. Even though this particular one is fixed in latest TobEx, there are still other possibly troublesome conditions.

I don't expect it to be free of processing time, just that it makes no sense for it to stutter if it isn't actually trying to make the NPC do something. You yourself brought up the NPCs as part of the problem. But does it stutter for every party member and creature on the screen? Humor me and turn your sounds off one by one.

Excessive ambient sounds can cause stuttering.

My working mods:
an AI Party Script for BG2 game engine DOWNLOAD LINK ONLY!
Interactive Tweaks for BG series with some IWD support. DOWNLOAD LINK ONLY!
Rest For 8 Hours an IWD mod
-------------------------------------------
My contributions: BG1Fixpack, BG1Tweaks
On Hold: Solestia an NPC for SOA
-------------------------------------------
My website: http://sasha-altheri...s.com/index.htm

Back to top

#13 Suslik

Investigator

Member
500 posts

Posted 12 April 2012 - 03:40 PM

Sorry for enormous amount of updates in this post.

maybe that in and of itself is the problem. just too much to process at one time.

The weirdest part is that the lags started somewhere in the middle of gameplay. There were no stuttering until a certain point(I cannot figure out when exactaly).

I don't expect it to be free of processing time, just that it makes no sense for it to stutter if it isn't actually trying to make the NPC do something. You yourself brought up the NPCs as part of the problem. But does it stutter for every party member and creature on the screen? Humor me and turn your sounds off one by one. Excessive ambient sounds can cause stuttering.

Take a look. I have managed to narrow down the search even further.

Take a look at this block:

IF
	Global("CbPlayer1Has_Soul_sucker","GLOBAL",5)
THEN
        RESPONSE #100
END

Then if I remove all code from Baldur.bcs and copy-paste this block ~1500 times, the game starts stuttering. I have only one completely stripped character(protagonist) in a location where no one else can be found, without any items/bags/containers, he slept a few days and I have manually erased cache/ folder. And still the game lags for about 1 second every second. Normal, you may say? 1500 checks is not that few? But how about the fact that if I load savegame with the same party, same playthrough but a little earlier(about 10 hours of gameplay all in all) there is no stuttering at all?

I have created two savegames: both with a single protagonist without a single item on the same location, and one game stutters while the other does not. My test baldur.bcs looks like this:

IF
	Global("CbPlayer1Has_Soul_sucker","GLOBAL",5)
THEN
        RESPONSE #100
END

IF
	Global("CbPlayer1Has_Soul_sucker","GLOBAL",5)
THEN
        RESPONSE #100
END

IF
	Global("CbPlayer1Has_Soul_sucker","GLOBAL",5)
THEN
        RESPONSE #100
END//about ~10000 lines of such copy-pasted code

The value "CbPlayer1Has_Soul_sucker" does not exist in both games, so none if these blocks gets ever fired. If I rename it to "CbPlayer1Has_Soul_sucke"(without "r" at the end), stuttering vanishes!. If I rename it to "CbPlayer1Has_Soul_sucker1"(with "1" at the end) stuttering remains! Assigning any value to this variable(except for 5, of course) does nothing. I have one question: WTF?!

Here's the link for these test savegames: http://dl.dropbox.co... test saves.zip

Edited by Suslik, 12 April 2012 - 04:39 PM.

Howto: mage in Baldur's Gate. My lame guide.
List of issues I have encountered in BWP 10.3 tactical
Stutter-fixer for Infinity Engine
While you were reading this signature, I have probably edited the post above. Twice.

Back to top

#14 Sasha Al'Therin

Modder
615 posts

Posted 12 April 2012 - 04:05 PM

If PartyHasItem checks are the culprit, I'm curious to know what specific mod(s) put in tons of such checks to lead you to this conclusion and to test it in such a radical manner. I wouldn't be surprised if parts of the script could be better served in other locations. There are some mods that just use baldur.bcs because it is the easy thing to do. Only reason to put stuff in baldur.bcs is if it can't be ran in a specific area, on a specific creature or container, etc.. and if it is undesireable to force player to have party ai on.

My working mods:
an AI Party Script for BG2 game engine DOWNLOAD LINK ONLY!
Interactive Tweaks for BG series with some IWD support. DOWNLOAD LINK ONLY!
Rest For 8 Hours an IWD mod
-------------------------------------------
My contributions: BG1Fixpack, BG1Tweaks
On Hold: Solestia an NPC for SOA
-------------------------------------------
My website: http://sasha-altheri...s.com/index.htm

Back to top

#15 Suslik

Investigator

Member
500 posts

Posted 12 April 2012 - 04:16 PM

Please forgive me for updating the previous post like 100500 times. Check the update: a simple Global("CbPlayer1Has_Soul_sucker","GLOBAL",5) stutters in one game and does not stutter in another! Even though the variable is uninitialized in both savegames, renaming it to something else does affect stuttering! More info is in previous post. Sorry again.

Edited by Suslik, 12 April 2012 - 04:19 PM.

Howto: mage in Baldur's Gate. My lame guide.
List of issues I have encountered in BWP 10.3 tactical
Stutter-fixer for Infinity Engine
While you were reading this signature, I have probably edited the post above. Twice.

Back to top

#16 William Imm

Obsessive Penguin Lover

Member
486 posts

Posted 12 April 2012 - 04:23 PM

CB = prefix for CBisson, who wrote Shadows over Solubar and Check the Bodies. One of these mods is causing the problem. Do a file search for "CbPlayer1Has_Soul_sucker" on the script files for both mods, and you can find which mod is causing the problem.

At this point, I'm not really doing much Baldur's Gate related. More focused on Skyrim modding and the Born of Legend tabletop roleplaying game. Don't expect much activity here.

Back to top

#17 Suslik

Investigator

Member
500 posts

Posted 12 April 2012 - 04:27 PM

Hah, it's not that easy! Not only checking "CbPlayer1Has_Soul_sucker" variable stutters. There are other variables which act the same way in different mods. Strange thing is that they do not always stutter, they start stuttering under unclear conditions. I think I have narrowed down the search as much as possible. I hope that info in post #13 is as clear as possible, i'm going to move it to the first post so no one has to read the whole thread.

Updated the first post.

Edited by Suslik, 12 April 2012 - 04:39 PM.

Howto: mage in Baldur's Gate. My lame guide.
List of issues I have encountered in BWP 10.3 tactical
Stutter-fixer for Infinity Engine
While you were reading this signature, I have probably edited the post above. Twice.

Back to top

#18 Suslik

Investigator

Member
500 posts

Posted 12 April 2012 - 05:16 PM

It seems that stuttering is affected by length of the variable name. I know it sounds weird, but if I use variables less than 20 symbols in the test case, the game does not stutter, while it always does with variables longer than 25 symbols. I suppose it is somehow related to using std::string in Infinity Engine's code, since short and long strings are allocated different ways in Visual Studio's stl.

Updated first post with info on variables I have tested.

Edited by Suslik, 12 April 2012 - 05:16 PM.

Howto: mage in Baldur's Gate. My lame guide.
List of issues I have encountered in BWP 10.3 tactical
Stutter-fixer for Infinity Engine
While you were reading this signature, I have probably edited the post above. Twice.

Back to top

#19 William Imm

Obsessive Penguin Lover

Member
486 posts

Posted 12 April 2012 - 06:01 PM

I have a feeling that there should be a TobEx component that allows for fixing this stutter problem - it's probably something to do with the EXE. If A64 wants to look into this, he sure can.

At this point, I'm not really doing much Baldur's Gate related. More focused on Skyrim modding and the Born of Legend tabletop roleplaying game. Don't expect much activity here.

Back to top

#20 Suslik

Investigator

Member
500 posts

Posted 12 April 2012 - 06:15 PM

I have a feeling that there should be a TobEx component that allows for fixing this stutter problem

Do you mean that there's already such component or that Asc64 should take a look at the problem and probably make one?

I have already reported the summary to Asc64. In case he is not going to investigate this issue I'll try to reverse the executable myself. I have a few ideas already on where to look.

I'm just curious why absolutely the same script with absolutely the same values stutters in one savegame and does not stutter in another. And if it's variable's name that matters, why does variable "aaaaaaaaaaaaaaaaaaaaaaa"(23 characters) still stutter?

Howto: mage in Baldur's Gate. My lame guide.
List of issues I have encountered in BWP 10.3 tactical
Stutter-fixer for Infinity Engine
While you were reading this signature, I have probably edited the post above. Twice.

Back to top

Next
»

Back to Mega Mod Help