26 replies to this topic

[Suslik]

As the investigation goes further, I will keep the first post updated as always. If you are new to this thread, you can just read the first post for a summary.

What I currently have. A test case:

There's a creature that has a script attached to it R#BREU01.cre/R#BREU01.bcs from Improved asylum, actually. The problem is the script is not executed. The most important proof of that is that the first block in R#BREU01.bcs.

IF
	PickPocketFailed([PC])
	Allegiance(Myself,NEUTRAL)
	General(Myself,HUMANOID)
	!StateCheck(Myself,CD_STATE_NOTVALID)
	See(LastTrigger)
	!Allegiance(LastTrigger,FAMILIAR)
THEN
	RESPONSE #100
		StartDialogOverride("RR#PICKP",LastTrigger)
END

Does not fire when I fail a pickpocket attempt. The character does not prebuff(prebuff code is in the same script) and does not initiate dialog with PC as supposed to. And the weirdest thing is that if I open his script in near-infinity and then save it even unchanged(or just export it as-is) into /override folder, the script executes normally, several times each second.

I have no idea how to reproduce the bug on other creatures/scripts, but this one seems to be perfectly reproducible for me - whenever I remove B#BREU01.bcs from /override and the engine takes it from game biffs, it stops being executed.

Any ideas? I hope we can fix it at the very least to ensure that the same problem will not get into BGEE D:

Upd:
Ascension64, I need basic function addresses that are responsible for executing creature scripts. Is this info reversed? Can you provide it to me? Actual questions are also in posts #10-#11

Edited by Suslik, 14 May 2012 - 03:22 PM.

[Suslik]

<Info in this post is not relevant anymore>

Edited by Suslik, 12 May 2012 - 03:20 PM.

[Sasha Al'Therin]

You could try the 'cast and attack' timer trick as in the following:

Spoiler

IF
		!Allegiance(Myself,ENEMY)
		CheckStatLT(Myself,5,STONESKINS)
		!GlobalTimerNotExpired("ab_CastASpell")
THEN
		RESPONSE #100
				SetGlobalTimer("ab_CastASpell","LOCALS",6)  //approx one round timer
				DisplayString(Myself,39326) // Stoneskin
				ApplySpell(Myself,WIZARD_STONE_SKIN)
				DisplayString(Myself,26306) // Protected by Spell Trap
				ApplySpell(Myself,WIZARD_SPELL_TRAP)
				DisplayString(Myself,14789) // Protected from Fire
				ApplySpell(Myself,WIZARD_PROTECTION_FROM_FIRE)
				DisplayString(Myself,180353) // Protected from Cold
				ApplySpell(Myself,WIZARD_PROTECTION_FROM_COLD)
				DisplayString(Myself,17400) // Protection from Evil
				ApplySpell(Myself,WIZARD_PROTECTION_FROM_EVIL)
END

At the very least it will space out the repeating...

Also the unmodified spwi408.spl has one of it's abilities casting only 4 skins instead of 5 (2nd one in the list according to NI). perhaps that particular ability is being triggered instead of one of the others.

[Suslik]

Forgive me for modifying the posts above more than 9000 times. It seems like the issue is deeper than some problem with stoneskins. Please check the first post again.

Also the unmodified spwi408.spl has one of it's abilities casting only 4 skins instead of 5 (2nd one in the list according to NI). perhaps that particular ability is being triggered instead of one of the others.

That was the problem with stoneskins, and I have reported it to the mod developer. Weird thing is that the script block does not always loop, which led me to a fact that the whole script is actually not always executed. For more proofs that the script is not executed at all, please refer to the first post.

Edited by Suslik, 12 May 2012 - 03:35 PM.

[Sasha Al'Therin]

perhaps the speed at which files are retrieved from the biffs is insufficient, especially in mega mod installs which have had all files biffed rather than just the larger file types of wav & tis

In a insane mega mod install (pretty much everything possible using the BWS beta) with everything biffed there are 506 bif files to be sorted thru. Even NI takes a long time to scan them for the creature which uses the script in question.

It does make sense that it will work if the file is in the override as it is checked there first before cycling thru the bif files.

If you have the disk space, try installing Improved Asylum alone with Generalized Biffing and biff everything just as was done in the mega mod install. If it works fine with less # of bifs then that may be evidence of some sort of slow down in bif getting by the engine.

[Suslik]

perhaps the speed at which files are retrieved from the biffs is insufficient, especially in mega mod installs which have had all files biffed rather than just the larger file types of wav & tis

The way that IE runs the scripts is determined and runs in one main thread. So if a script that is not loaded and should be executed, the whole game will stutter/lag until it loads, because script resources can not be loaded in background. It does not matter how many mods I have installed(i have a little more than 1000), script system must run exactly the same way. If a creature has a script, it must be executed each set period of time.

If you have the disk space, try installing Improved Asylum alone with Generalized Biffing and biff everything just as was done in the mega mod install

I have not tried that, but I'm pretty sure the script will run fine. It seems it has nothing to do with a particular mod of Improved Asylum and a poor particular creature R#BREU01, but it seems to have something to do with internal memory management for script code or for biff resources. If this is the case, any pertubation of resources will interfere.

I suppose I will tinker with this a little more then I will have to use my trusty Olly debugger.

Edited by Suslik, 12 May 2012 - 04:26 PM.

[aVENGER]

Due to the recent changes of the PickpocketFailed() trigger behavior within TobEx, you need to use both RR v4.48 (or higher) and TobEx 0023 (or higher) in order for that block to execute properly. Also, make sure that tobextweak.ini contains this: PickpocketFailed Trigger Only=1

Furthermore, the script containing that block needs to be assigned to the highest priority script slot of the creature (i.e. override) in order to fire reliably. The block should also be located at the top of the script. Other than that, it's possible that the creature's own scripting is interfering with the pickpocket block in some way.

Edited by aVENGER, 12 May 2012 - 08:04 PM.

[Ascension64]

What aVENGER said. PickpocketFailed() is broken in vanilla.

[Suslik]

Due to the recent changes of the PickpocketFailed() trigger behavior within TobEx, you need to use both RR v4.48 (or higher) and TobEx 0023 (or higher) in order for that block to execute properly. Also, make sure that tobextweak.ini contains this: PickpocketFailed Trigger Only=1

What aVENGER said. PickpocketFailed() is broken in vanilla.

PickpocketFailed() was only an indirect evidence of the script not being executed. The character discussed does not initiate dialog with me, does not prebuff and his battle scripts do not work.

Furthermore, the script containing that block needs to be assigned to the highest priority script slot of the creature (i.e. override) in order to fire reliably.

Script discussed is in "override" slot.

And how is that possible that if I put a copy of this script into /override, it starts working as supposed to?

Edited by Suslik, 13 May 2012 - 02:32 AM.

[Suslik]

I did some investigation on how the scripts are executed using currently present reversed data in TobEx. What we have:

There are two basic structures for the script blocks and a scripts which actually contains a list of blocks:

struct CScriptBlock { //Size 3Ch
//Constructor: see 0x41829F
	BOOL Evaluate(CTriggerList& triggers, CGameSprite& sprite);
	BOOL EvaluateTrigger(Trigger& t, CTriggerList& triggers, CGameSprite& sprite);

	CTriggerList m_triggers; //0h
	CResponseList m_responses; //1ch
};

struct CScript { //Size 28h
	ResRef name; //0h
	int u8;
	CScriptBlockList blocks; //ch
};

It also seems that there are those 7 script levels attached to each CGameSprite:

class CGameSprite : public CGameObject { //Size 3D4h
//Constructor: 0x476DED
public:
//...
	virtual void SetScript(int nScriptIdx, CScript& script) {} //v8c
//...
	CScript* pScriptOverride; //23ah
	CScript* pScript1; //23eh
	CScript* pScriptAreaSpecific; //242h
	CScript* pScriptClass; //246h
	CScript* pScriptRace; //24ah
	CScript* pScriptGeneral; //24eh
	CScript* pScriptDefault; //252h
}

But I did not manage to find the mechanisms how the engine actually assigns and runs each of these scripts. Under certain circumstances some entity should call pGameSprite->SetScript(id, pScriptParser->ParseScript()), and execute it later: pGameSprite->pScriptOverride->Execute(), but where does that happen? And I have failed to find any info on CScript calls - not even a constructor =(

I also wanted to debug the code by detouring SetScript method, but it proved to be a little tricky task since the method is virtual. How do I detour virtual methods?

Edited by Suslik, 13 May 2012 - 01:13 PM.

[Suslik]

I tinkered with the executable for a while. Discovered not much, but still:
__thiscall ExecuteScript_41769A(CTriggerList&, CGameSprite&), seems to execute a script and return a pointer to a structure that contains a number(pointer or index) of the block that was fired.
__thiscall RunScripts_48E744(int) which is a pretty heavy function that among other iterates through some scripts and executes them.

That last function is pretty solid indeed, and my awesome reversing skills are apparently far from being enough to reverse it myself. I need at least a little more hints on related function calls and structures.

By the way the bug is perfectly reproduced under debugger.

Since it is extremely difficult to reproduce the bug on any other setup than mine, maybe I can grant Ascension64 access to my computer via remote desktop, so that he investigates it? I can install any needed soft and all that stuff. Is this idea crazy but acceptable or just crazy? : D

Edited by Suslik, 13 May 2012 - 05:12 PM.

[i30817]

Suslik about the stutter fix; are you sure the optimization you did for only writing to 10% of the buckets is correct?
What if they are all filled? Rehash and recurse, sure, but the hashcode will not introduce new spaces on hashcodes that were already linear.
If by some reason there is a sequence of (natural) hashcodes with
1, 2... 10 in buckets 1, 2..., 10 and you try to add (another) key with hashcode 1...
changing the size of the array and rehashing will not change the sequence - so infinite cycle?
Does the hashcode take the table size into account in a non-linear fashion (ie: 2+t.size,3+t.size is linear; 2*t.size,3*t.size is not)?

You can verify if the table affects the hashcode somehow by checking the value of 2 hashcodes of the same key before and after a rehash (but not if they are linear, though there wouldn't be a point otherwise i guess).

Anyway, if it is safe on the write end, it will be safe to restrict on the read end too, so i don't see why that optimization couldn't be used.

Doh i keep forgetting it's not a fixed value of buckets, but 10% after resize. That should eventually end in a free bucket yeah. And i just thought a bit and the read "optimization" is irrelevant anyway. If the value is always in the first 10% the rest of the table is not checked.

Edited by i30817, 14 May 2012 - 01:00 AM.

[Suslik]

Uhh.. bump?

[Ascension64]

No time for to help yet, sorry.

[Suslik]

Oh well. Thanks for the reply. I will try to tinker with the thing a little more, but I think I will need help anyway.

[Suslik]

I will wait patiently until someone can help me investigate the issue. I have created a test save, test scene and ready to return to it any time, but unfortunately I cannot do it on my own. So if you have any info on such problem, or you have encountered something similar, or you have accidentally reversed script execution code(lol), please let me know.

[Ascension64]

If it is vanilla, you can just attach it and I can have a play when I get around to it.

[Suslik]

Even though the problem can affect vanilla, I have managed to reproduce it only once, on my current mega-mod setup. And how can I attach it, if the script is not executed normally, but starts being executed when I export it to /override from game biffs? That's why I'm offering a remote control over my PC so that you can ensure that it's indeed possible. Or alternatively I can make a repack of my setup and send it to you via torrent. Unfortunately I cannot see any other ways to reproduce it at your end.

[Ascension64]

Hmm, remote control isn't much of a possibility as I am a bit too busy with RL most of the time. I'm not sure how I can help.

[Suslik]

Ok, then there's only one option left - to debug it myself. But to do this, I'll need some function call addresses and data structures which as far as I know are not fully reversed by now. These are: CScript, CScriptParser and the entities responsible for executing the scripts which I do not know about. Do you have any info on these that is not included to TobEx yet?