164 replies to this topic

[Qwinn]

Trying to spend as little time as possible at prompts, here's what I got:

parsing .ids files			   0.015
parsing .tra files			   0.015
unmarshal BIFF				   0.016
EXTEND_BOTTOM					0.016
process .D files				 0.016
resolve strings				  0.016
unmarshal KEY					0.016
parsing .d files				 0.032
EXTEND_TOP					   0.032
Parsing TPA files				0.048
marshal BCS					  0.078
parsing .bcs files			   0.095
COPY							 0.108
READ_*						   0.123
marshal DLG					  0.217
unmarshal TLK					0.234
loading files					0.280
saving files					 0.298
unmarshal DLG					0.313
eval_pe						  0.575
COMPILE						  0.594
ACTION_READLN					0.969
process_patch2				   1.536
stuff not covered elsewhere	  2.186
marshal and save TLK			 4.156
TOTAL						   11.984

Of that, my macros -might- be involved in some fraction of:

COPY							 0.108
READ_*						   0.123
loading files					0.280
saving files					 0.298
eval_pe						  0.575
COMPILE						  0.594
process_patch2				   1.536
stuff not covered elsewhere	  2.186

... so you've got all my 12 uses of the process macro contained in less than 5.7 seconds of installation time. And that includes time spent at prompts and all the file saving/loading/etc. for all my nonmacro stuff.

I think this is sufficient grounds for me to reject, and get justifiably defensive about, claims that my macros have "dealbreaker" levels of runtime issues.

Qwinn

Edited by Qwinn, 10 June 2009 - 04:45 PM.

[Mike1072]

The function I posted above *is* a real example (I even tested it to make sure it worked).

What I meant was a real example of this:

plus with a little planning, you can have functions whose parameters get "reset" upon exiting without needing to copy/paste in a huge block of code at the end of your macro setting all values to their defaults.

You are calling Qwinn's macros (the whole business), including the "huge block of code at the end... setting all values to their defaults" which you said should not be necessary "with a little planning."

My example is not a replacement for any of Qwinn's stuff, it's just something to to make another part of the process easier. It does reset its own parameters, so if I called it a third time, adding an actor without specifying its orientation, that field would be not be set to 5, the orientation of the last actor added, but instead to the default of 0. If you wanted to use a macro/INCLUDE'd code block to do the same thing, you'd have to 1) initialise the default values of each parameter before calling the macro for the first time and 2) set each parameter to its default value at the bottom of the macro.

@Mike1072 -- I saw in your example that you set the used values inside the launch. Do you think that that would solve the issue I brought up over at PPG concerning FUNCTIONS and variables? If so, I'll give that a try and see how it works..

It will let you do what you want to do, but there remains a concern when trying to ensure parameters are reset. The points I brought up there are relevant to the discussion - if you give your parameters common names, there's a chance that a variable named the same as one of them will get set somewhere previously, making it so when that parameter is left unspecified, the variable takes on the random other value instead of the default one like you'd expect. This is what I meant by "a little planning" - if you use prefixes, things are safer in this regard, but I still think your proposed command would be useful (because the idea of prefixes on parameters seems silly).

Edited by Mike1072, 10 June 2009 - 06:36 PM.

[Miloch]

I think this is sufficient grounds for me to reject, and get justifiably defensive about, claims that my macros have "dealbreaker" levels of runtime issues.

No, it isn't. Why should I justify and keep repeating why I'm sticking with existing code that does the same thing in half the time? We've already established your UB mod doesn't add containers. Regardless of whether you think that's any different than what UB does do, it obviously is in my testing, where that eval_pe spikes up (unlike in UB). So before you keep repeating the same info and logs, why don't you test it yourself to add several containers then use A64's code to do the same and try to figure out why that's the case. If you're not willing to do that, that's fine, you can do what you like, but quit giving me a hard time for using existing code that works and is quicker.

Edited by Miloch, 10 June 2009 - 07:59 PM.

[Qwinn]

This is why.

While it might not seem significant for this one container, I'm thinking it'd increase runtime dramatically if I added several in multiple areas.

I'm not bitching that you should use my macros. You can use whatever you like. I'm bitching that you've publicly made a lot of unwarranted baseless criticisms of the macros... "you're not adding containers, which might yield different runtimes than adding other sections" (it can't, there is no container-specific code or task required, ask Sasha), "does your code even work, has it even been tested?" (it does and it had), "this InitVars at the end is redundant" (no, it isn't), "this is an insignificant increase in runtime but if we did it lots of times in a row I bet it'll really add up" (it doesn't), etc. etc. etc.

Look, I really don't care if you don't want to use my macros, I offered 'em and have been posting for seven pages trying to help people use 'em cause I think they might help people. I would hope that, in return for my efforts, you could at least -one- time say "Gee, I'm sorry I implied your code sucks/is redundant" after I point out or prove where your assumptions were wrong, before springing on to the next one.

Qwinn

[Miloch]

"does your code even work, has it even been tested?" (it does and it had)

This was specific to their use for containers, which I'm still unsure of (whether they'd been tested for containers). You had to edit your post at least once to get that code right (all 2 lines of it), which wouldn't've been the case if it was readily accessible. I know this because I got your initial post by email, which didn't work either. I realise it's not in the macro itself, but the invoking code is integral to it (and probably should be in the macro since it just repeats variables already set).

"this InitVars at the end is redundant" (no, it isn't), "this is an insignificant increase in runtime but if we did it lots of times in a row I bet it'll really add up" (it doesn't)

None of this has been disproven.

after I point out or prove where your assumptions were wrong

You have not, but I'll probably drop out of this thread if it's just going to degenerate to baseless defensiveness. I don't need or like the conflict any more than Sasha does. I really don't know why you can't concede that anything can be investigated and improved, regardless of what it does or how useful it may be.

[Qwinn]

None of this has been disproven.

WTF? Running it 12 times within 5.7 seconds (and probably a lot less - that 5.7 includes all copying, saving and loading for everything -besides- those macros) doesn't disprove that, when running your version one time took 1.9 seconds?

I give up. If I didn't prove that, then you've got some serious issues.

Qwinn

Edited by Qwinn, 10 June 2009 - 08:50 PM.

[Qwinn]

Actually, no, I'll make one more try to explain this to His Orcishness.

Sasha, since you understand the code... is there ANY logical way that containers could somehow magically take longer than adding actors? In fact, isn't my earlier conclusion that actually -actors- (which I add a ton of) would require the most work, because they come earliest in the area file and therefore require more offset changes pretty much indisputable? I ask for a mere yes or no if you would, please.

I realise it's not in the macro itself, but the invoking code is integral to it (and probably should be in the macro since it just repeats variables already set).

If you mean, the two lines that I told you were missing should be automated, not possible. What if I added a container with 5 vertexes and a trigger with 12 vertexes all in the same call to the macro? How's it supposed to know where the cutoff is in the 17 vertexes I just added?

Look, you said yourself that you didn't understand the code, that this is not your forte. It would behoove you, then, to spread your criticisms around with a little less of a superiority complex, because most of your objections are based on very false assumptions, and it's getting irritating proving that repeatedly and yet your response doesn't even acknowledge the rebuttals. It's absurd that you claim I haven't rebutted your false charge that multiple runs increases runtime "dramatically". I mean, hell, I already bet two pages ago that if you ran your macro 12 times it would take longer than my entire UB mod. (And that's SEQUENTIAL, like my 12 runs, not calling the exact same function 12 times to to create 12 identical containers in the same area)

Your refusing to even acknowledge the possibility that even a single one of your uninformed criticisms aren't "immortal gods set in stone" is getting more than a little insulting.

Lastly - dear Lord, if you can find 3 posts in this entire thread by me that weren't edited at least once, I'll give you a cookie. I don't think that proves much. Shit, I took the -time- to find out what you missed in your code, and gave you the solution within an hour of you posting the problem, and rather than say thanks for spotting what you missed, you grudgingly say "it appears to work now" three paragraphs later and give me shit cause I spotted and fixed an additional issue inside of 5 minutes? Sense of entitlement much?

This was specific to their use for containers, which I'm still unsure of (whether they'd been tested for containers).

Unsure despite the fact that I told you I'd tested it three times now. I suppose the fact that you're still unsure of the testing despite this repetition shouldn't be insulting either.

Qwinn

Edited by Qwinn, 10 June 2009 - 09:24 PM.

[Qwinn]

In fact, the more I review this the more insulting it is. This'll be the last input I bring to this discussion until there's an attitude change.

I ran my original patch several times and got an average of 1.95 runtime. Yours took 2-3 times as long.

SO. By your own metric, running a single container add with my code took somewhere between 3.9 and 5.85 seconds. I've proven that I can run the process 12 distinct times in under 5.7 seconds, and that's with the additional overhead coming from like 8 other mod components. And yet, I have not proven that:

I'm thinking it'd increase runtime dramatically if I added several in multiple areas.

is objectively false? Praytell, how AM I supposed to prove it then? (And for the fiftieth time, the fact that it's a container instead of an actor makes no difference whatsoever, or if it does, only to your benefit.)

In view of that, and all the other rebuttals that have simply been ignored (including, amazingly, the highly relevant information that any time spent sitting on the install prompt increases the debug's runtime calculation), I can only say two things:

1. This isn't an argument, it's just contradiction.
2. I can explain it to you, but I can't understand it for you.

Qwinn

[GeN1e]

I'm not Sasha, but I can't nail anything that would make containers to take longer time.

Shit, I took the -time- to find out what you missed in your code, and gave you the solution within an hour of you posting the problem

Hey, I did that within 20 minutes!

[Qwinn]

Gen1e, thanks kindly. I forgot to mention earlier, and I should have.... I think your library of writes from post #123 looks excellent, and should I ever need to add a section type I haven't already done before, I'd definitely use your work there to cut/paste/tweak for the writes in my code.

Incidentally, I don't really see the value of turning those writes into macros/functions. Having to look back and forth between such a macro/function and the code calling it in order to set up all the variables we'd have to be passing back and forth seems to me to add, rather than relieve, overhead. If someone is actually more comfortable setting variables than tweaking the write library in a cut and paste because the latter looks scarier, then of course they should feel free to go for it, but I don't think it objectively reduces workload.

Qwinn

Edited by Qwinn, 11 June 2009 - 10:07 AM.

[Qwinn]

Alright, I added the Game_Is code provided by Sasha, thanks Sasha.

I also added another macro, Q_ARE_ResetVars, which is identical to Q_ARE_InitVars except it doesn't include all the constant SET's of "offset-of-offset" locations and such. It is this shortened version that is called by the Process macro now. This produces no detectable difference whatsoever in runtime (UB install, for example, still takes a total ~10.9 installation time), but whatever.

EDIT: Actually, never mind on the latter part. I removed the Q_ARE_ResetVars macro. All it was doing was increasing the size, number and complexity of my macro definition file, for no discernible benefit whatsoever.

Qwinn

Edited by Qwinn, 11 June 2009 - 11:05 AM.

[Sasha Al'Therin]

Sasha, since you understand the code... is there ANY logical way that containers could somehow magically take longer than adding actors? In fact, isn't my earlier conclusion that actually -actors- (which I add a ton of) would require the most work, because they come earliest in the area file and therefore require more offset changes pretty much indisputable? I ask for a mere yes or no if you would, please.

As I understand it.... The time it takes for your macro process to run is about the same for any section. Why? The macro only adds the space. Where the time gets eaten up is in the process used to do the writes. If we use functions or other forms of external code where the writes are listed with variables that must be defined at some earlier point it will take more time to call these external files and process them.

I'm not sure of the exact code that Miloch has said is from Ascension64, but if it's like the code he did for me and the restoration of the hidden container in FAI for vanilla BG, then the macros are defined in an ALWAYS within the header and then called up later as needed. Perhaps, the time difference is because of the location where the code is being called from... I'm not really sure...

If I can locate again the code for the hidden container, I'll pull the written data and use it with your macros. I'll set up a single mod that has two subcomponents one using A64 code and the other using your macros + writes. I'll share the timing data from the bottom of the debug file. Then I guess we will know one way or the other.... Of course, I'll have to uninstall BG-Totsc and reinstall just BG to do this, but I'm willing to do so if it means that we can get to the bottom of this and finally put it behind us once and for all...

[Qwinn]

As I understand it.... The time it takes for your macro process to run is about the same for any section. Why?

The reason it's an issue is because all the tests I've done with my UB mod indicating a very short installation time are being discounted as irrelevant because my UB mod doesn't actually add containers specifically. It actually adds far more complex stuff that is a lot more WRITE-intensive, but that doesn't apparently matter, for some reason containers alone are supposed to create these timing issues.

Don't feel you need to do these tests on my account. Since I know containers are no different, the UB tests going quickly are more than enough evidence for me that the purported "dramatic" increase in runtime upon multiple uses does not exist.

Might it take longer for a single use? Sure, it's possible. In the order of a second or two, possibly, due to needing to compile those PFE's and stuff (though, taking a page from the handbook, I don't think the fact that it -does- take longer is conclusively established either, considering that time hanging on the "Install" prompt counts). But even if it did, it adds a ton of flexibility, and once compiled, additional uses add no meaningful runtime. As I've said, I actually believe my method would probably run faster for multiple uses, because my process is compiled once and can be used repeatedly, whereas with the other methods most of the code would have to be rewritten, compiled and executed sequentially.

Qwinn

Edited by Qwinn, 11 June 2009 - 02:28 PM.

[Sasha Al'Therin]

As I understand it.... The time it takes for your macro process to run is about the same for any section. Why?

The reason it's an issue is because all the tests I've done with my UB mod indicating a very short installation time are being discounted as irrelevant because my UB mod doesn't actually add containers specifically. It actually adds far more complex stuff that is a lot more WRITE-intensive, but that doesn't apparently matter, for some reason containers alone are supposed to create these timing issues.

Don't feel you need to do these tests on my account. Since I know containers are no different, the UB tests going quickly are more than enough evidence for me that the purported "dramatic" increase in runtime upon multiple uses does not exist.

Qwinn

I was going to convert it (the container add on) to use your macros anyway... I'm slowly going through all my area patches and converting to at least use your variables so I don't have to do any additional reading... While there, I've found various other types of files that would benefit from having a function or macro because I edit the same fields for several files. In fact the conversion is already done, I just need to uninstall and reinstall to test it.... No, I don't... I forgot. My computer suffers from the merged override contents syndrome. That particular area file is one of the bad files that need to be removed... I can just make sure that the fixpack is not installed and give the container add on mod a test run...

Be back with the results in a few....

...goes off to give it a test run...

...comes back huffing and puffing...

Using least amount of time waiting for input (i.e my finger was hovering over the number to press and another finger over the enter button)

When the container addition using Qwinn's macros are first in the subcomponent list:
A64's macro code in an external file -- total of 0.562 while Qwinn's macros in the header -- total of 3.485
A64's macro code in the header -- total of 0.485 while Qwinn's macros in an external file -- total of 3.031

When the container addition using A64's macro code is first in the subcomponent list:
A64's macro code in an external file -- total of 0.672 while Qwinn's macros in the header -- total of 3.047
A64's macro code in the header -- total of 0.563 while Qwinn's macros in an external file -- total of 3.312

Both do the same work i.e. add one container with one item and 5 associated vertex points

There are some notable differences
A64's code inserts the bytes and does the writes inside the tp2 code also the new entries are added to the beginning of the respective sections (this is where I had picked up that habit)
Qwinn's code inserts the bytes within the macro and the writes are in the tp2 code also the new entries are added to the end of the respective sections.

Qwinn's code uses two patch_for_each commands one to match the section(s) being added and the second to examine the other sections to see if they need updating. He uses the patch_for_each commands along with the pre-defined offset variables to have only one single block of update code. A64 uses the numerical offset values and update code for every section. A64's macro does not add the space and has to be called every time space is inserted.

It boils down to this...
If you are doing just one thing and you won't be using the macro later for anything else, go with something simple.
If you are doing multiple things and you will need to use the macro later for more stuff, go with the more versatile but 'longer' running code...

The total time for Qwinn's macros includes using one call to Q_ARE_InitVars, one call to Q_AREAdd_InitVars, one call to Q_AREAdd_Process which recalls Q_ARE_InitVars once for each section being added to. This multiple macro call is what allows it to insert bytes and update offsets for multiple sections in one shot.

The two cannot be properly compared for run time because of the difference in scope, abilities and limitations of each...

Hey, I quoted you pre-edit

EDIT:
Per a conversation with GeN1e via PM, I created an ACTION function that would allow the user to set all their values within the launch command. The function calls up Qwinn's macros and adds as many floor triggers (i.e traps and other proximity type triggers) as the user wants along with their associated vertex points. Due to the nature of the multi-use of the writes I needed to use PATCH_FOR_EACH to resolve the large number of variable names. When PATCH_FOR_EACH was added, the amount of time it took to process increased. I believe that PATCH_FOR_EACH is the culprit in the higher run time found with the use of Qwinn's macros.

So if that is indeed the case, I wonder if ACTION_FOR_EACH causes an increase in run time as well.... But does it really matter? We aren't talking about taking up hours of a players life to install a mod, just a few more seconds maybe a minute tops depending upon number of uses and how big the mod is...

:END EDIT

Edited by Sasha Al'Therin, 12 June 2009 - 02:35 PM.

[Sasha Al'Therin]

Alright, I added the Game_Is code provided by Sasha, thanks Sasha.

And it works! Yay! I finally got something right...

I also added another macro, Q_ARE_ResetVars, which is identical to Q_ARE_InitVars except it doesn't include all the constant SET's of "offset-of-offset" locations and such. It is this shortened version that is called by the Process macro now. This produces no detectable difference whatsoever in runtime (UB install, for example, still takes a total ~10.9 installation time), but whatever.

EDIT: Actually, never mind on the latter part. I removed the Q_ARE_ResetVars macro. All it was doing was increasing the size, number and complexity of my macro definition file, for no discernible benefit whatsoever.

I can see where you added it and then removed it cause the comment line that says that macro definition is ended is still in there rather than the comment line that said the Q_ARE_InitVars macro definition has ended, but no biggie its not worth an update...

I keep having problems when I try to do $some_name(%variable%) Weidu doesn't like it as a value to be written, it usually converts it to the name but not to the assigned value or it adds the underscore but doesn't even convert the variable inside the parenthesis. Don't know what I'm doing wrong, but I've been able to get around it by doing this for text

SPRINT ab_var ~%string2%%index%~
   SPRINT ab_var EVALUATE_BUFFER ~%%ab_var%%~

and this for numbers

SPRINT ab_var ~%string%_%index%~
   ab_var = EVALUATE_BUFFER ~%%ab_var%%~

But to do this (cause there was a lot of entries to do) I had to use a Patch_for_each. If I could have just used the array structure in the write, it would have been much simpler to do. I tried WRITE_SHORT (%new_loc% + 0x2a) $RT_VERTEX_PAIRS(%index%) but weidu couldn't convert RT_VERTEX_PAIRS_%index% to an integer. But %index% was being properly defined as it was coming from the for loop command... Any ideas? I looked into your macros to see how you handled the array structures and saw that you always set something else equal to the array structure. Is there no way that you have found to use the array structure in a write command?

[GeN1e]

Add quotes to (%index%) - ("%index%").

WRITE_SHORT (%new_loc% + 0x2a) $RT_VERTEX_PAIRS("%index%")

[Sasha Al'Therin]

Add quotes to (%index%) - ("%index%").

WRITE_SHORT (%new_loc% + 0x2a) $RT_VERTEX_PAIRS("%index%")

I'll give that a whirl... It'll have to be tomorrow evening though... Gotta do overtime tomorrow so I've got to get a little sleep in...

I'll send you (GeN1e) an updated add_floor_trigger function wrapper for Qwinn's macros after I get it recoded and working with this new info...

Thanks...

[Qwinn]

Gen1e's response strikes me as correct. It was similar issues that led me to pretty much standardize my consistent use of quotes around variables.

I believe that PATCH_FOR_EACH is the culprit in the higher run time found with the use of Qwinn's macros.

Sounds quite plausible to me. What that command is actually doing there is pretty complex. And yes, I suspect that is indeed what is going into the "eval_pe" category. Still, I think the usefulness of that command for the purpose is worth the price. I'll take a look at ACTION_FOR_EACH, if it's a significant difference, sure, I'll replace it. I don't expect it to be different though. Isn't the difference between an "ACTION_" command and a "PATCH_" command simply that you use the PATCH_ version within a COPY_EXISTING block, and ACTION_ outside of it? I could be wrong, but that's all I thought it was.

just a few more seconds maybe a minute tops depending upon number of uses and how big the mod is...

A very few seconds is pretty much the max, IMO. I think the additional time is part of the compile process, not the use of the macro itself, so the additional runtime is a one-shot deal only affecting the first use. This view is consistent with the results of my UB installation. Do note - my UB mod is already pretty darn large as far as non-fixpack mods go, adding something like 30 actors, 5 areas, at least half a dozen items, and bunches of other stuff.

Qwinn

Edited by Qwinn, 12 June 2009 - 08:30 PM.

[Sasha Al'Therin]

Add quotes to (%index%) - ("%index%").

WRITE_SHORT (%new_loc% + 0x2a) $RT_VERTEX_PAIRS("%index%")

This works great for integers but not for text...

I have tried: WRITE_ASCII (%new_loc% + 0) ~$RT_NAME("%index%")~ (32)
and the result is: $RT_NAME("%index%")
I have tried: WRITE_EVALUATED_ASCII (%new_loc% + 0) ~$RT_NAME("%index%")~ (32)
and the result is: $RT_NAME("0")
I have tried: WRITE_EVALUATED_ASCII (%new_loc% + 0) ~$RT_NAME(%index%)~ (32)
and the result is: $RT_NAME(0)
I have tried: WRITE_EVALUATED_ASCII (%new_loc% + 0) $RT_NAME("%index%") (32)
and the result is a weidu error on $
I have tried: WRITE_EVALUATED_ASCII (%new_loc% + 0) ~%$RT_NAME("%index%")%~ (32)
and the result is: %$RT_NAME("%index%")%
I even tried: WRITE_EVALUATED_ASCII (%new_loc% + 0) ~%RT_NAME_%index%%~ (32)
and the result is: %RT_NAME_%index%%

Does anyone know how to get the correct text value to be written while using the array structure?
Or must I use:
SPRINT RT_NAME ~RT_NAME_%index%~
SPRINT RT_NAME EVALUATE_BUFFER ~%%RT_NAME%%~
WRITE_EVALUATED_ASCII (%new_loc% + 0) ~%RT_NAME%~ (32)

Sounds quite plausible to me. What that command is actually doing there is pretty complex. And yes, I suspect that is indeed what is going into the "eval_pe" category. Still, I think the usefulness of that command for the purpose is worth the price. I'll take a look at ACTION_FOR_EACH, if it's a significant difference, sure, I'll replace it. I don't expect it to be different though. Isn't the difference between an "ACTION_" command and a "PATCH_" command simply that you use the PATCH_ version within a COPY_EXISTING block, and ACTION_ outside of it? I could be wrong, but that's all I thought it was.

That's the only difference that I know of...

[Mike1072]

Does anyone know how to get the correct text value to be written while using the array structure?

You don't put quotes around an entire array entry. You do put quotes around the indices: "%index%" versus %index% or index.

Instead of:

WRITE_ASCII (%new_loc% + 0) ~$RT_NAME("%index%")~ (32)

Use:

WRITE_ASCII (%new_loc% + 0) $RT_NAME("%index%") (32)