216 replies to this topic

[Lollorian]

Ok, final statistics (with no biffing and IA ):

BGII-SoA: 28.0 GB
Data: 14.1 GB
Override: 7.32 MB

tb#biffs: 210

And Statistics after CleanupX:

BGII-SoA: 24.6 GB
Data: 14.1 GB
Override: 2.8 MB

But as you can see, while the data\ folder did reduce by 300 MB, the size after the CleanupX is actually larger than the normal-BWPExpert-with-biffing after CleanupX ... and I did use the same Cleanup.bat Well, the biffmeister is kinda finished now All that remains is just refining the CleanupX.bat

@thebigg: Is the prod folder in the gen_biff (which contains the entire de-biffed override before biffing) needed for a changelog?? It's using up 5.45 GB right now

And reviewing the Cleanup.bat, I found it's not updated to include all the mods in BWP 8.2.8 Not even something that patches all over like aurora

Edited by Lollorian, 20 January 2010 - 12:26 AM.

[Lollorian]

Ok, I've included all the new mods (and some old ones that got forgotten like Yikari ) and the BGII-SoA is now a GB less

But there's a slight problem ... some mods have spaces in their folder names, "BWL Contest", "NSC Portraits", "Mod backups" (from Nathaniel) and the BWP install/fix/smoothpack folders ... any idea how they can be made to be processed by the FOR loop??

[the bigg]

@thebigg: Is the prod folder in the gen_biff (which contains the entire de-biffed override before biffing) needed for a changelog?? It's using up 5.45 GB right now

It's not needed for a changelog, but it's needed if you want to uninstall gen_biff (and consequently change anything in your install). Perhaps you can offer removing that as an option

[Lollorian]

Ah, cool But while an option might be useful, the CleanupX.bat wasn't meant to be used by the people using the BWP (mainly cause it needs robocopy.exe from the Win2k3 server Resource Kit ) I could probably provide the .exe (also available as a separate download here) but the last thing I want is some mail from Mr.Gates and Co. (if ya know what I mean )

EDIT: I've kinda made some optimizations to the CleanupX, added a separate option for gen_biff's backup (will need testing ) and put up an ugly workaround for the spaced-out-folders

I rename them to underscored versions, clean them up, and renmae them back to their spacey versions (was easy cause there are only three ... yet ) Oh, and it deletes the BWP Smoothpack and Fixpack folders now

Attached Files

•  biffmeister_rc1.7z   499.04K   235 downloads

Edited by Lollorian, 20 January 2010 - 07:04 PM.

[Lollorian]

I did what could possibly be the LAST test install for the biffmeister It does what it's meant to do now, and I've attached everything with the debug/log of my last install - which happened with NO errors (I corrected the rupert typo meself btw )

Statistics after CleanupX:

BGII-SoA: 17.5 GB
data: 13.5 GB
override: 5.37 kB

A change in the CleanupX is that there are some mods that don't make backups in "backup" folders

NeJ2Biffer-> in NeJ2\rules
Eldoth, Vanim, Teddy, Roar -> in eldoth\eldoth-backup etc

This one is now foolproof

I've done all I could, now if someone other than me could just use this and report whether it's working or not, it'll be safe to assume that it's ... safe The package includes robocopy (for the moment ... *sweats*)

If anyone wants to test,

• Allow BWS to extract all the mods/extract them by hand (DELETE the Exnem\ folder, Setup-Exnem.tp2 and exe!!! )
  
• Extract the biffmeister in that folder and allow it to overwrite
  
• Run Unpack-updater.bat (I always forget this )
  
• Run the BiG World Install.bat (it'd b e great if someone chooses Expert, but it should work with everything )
  
• Go to sleep
  
• After it finishes, check the size of the folders and note them down
  
• Run BiG World CleanupX.bat (it asks you whether you wanna clean gen_biff, reply what you wish )
  
• Note down new sizes of the folders
  
• Check whether the only .bat's in the folder are the change-log.bat and CleanupX.bat
  
• Run a change-log of anything you want using the change-log.bat
  
• check if it can produce one

PS: Due to the 512 k limit, the biffmeister is in the post above this one

Attached Files

•  WeiDU.log   79.35K   188 downloads
•  BiG World Debug.7z   266.5K   208 downloads

Edited by Lollorian, 20 January 2010 - 07:08 PM.

[Lollorian]

Oh my, just noticed a small line I forgot to delete If you delete line 110 of the CleanupX.bat, you'll save another 14 MB of space This one:

if exist "Mod backups" ren "Mod backups" "Mod_backups"

[Leomar]

At first an information by AVENGER:

I just noticed that BWP's cleanup.bat corrupts the installation directories of Rogue Rebalancing and aTweaks by deleting RR#EVAL.2DA. Although that file is 0 bytes in size, it gets filled up with information by several RR and aTweaks components at install time. Deleting it causes an install failure, so please don't do that.

If it helps, in the next version of aTweaks and RR I'll slightly increase the size of RR#EVAL.2DA by adding a header to the file. It won't be zero bytes then.

It would be nice if you can update your Clean-UpX.bat to consider it.



Like you know, I'm running some installation tests, because of the CtB/NTotSC problem. I used your batches to speed up my installations and it worked great. The no backup option of the override directories fasten it really up.

The reduced biffing works fine, too, but I'm not sure, if all is correct. I know, that you've looked for bif's which must be present for other mods and I don't know the whole biffing process of the big BG1 and BG2 mods. But you should test it carefully and look at each mod with the reduced biffing option, if all is correct. What does the mod do before biffing... What are the biffing options... Does the mod copies/compile all mods in the override directory or in an own self created directory, which will be biffed after that....
Otherwise it could be a new source of error, because of missing files. Only my thoughts.

What I've recovered so far and should be checked:
CtB creates CTB-* folders, like NTotSC (NTotSC1, etc.), but after the installation, some of the folders are still present and not deleted. All the folders are empty, so it looks fine, but the mod preparing folders should be removed.

The only thing I'm wondering about is SoS:
The difference between SoS and CtB, NTotSC, etc. is, that the preparing folders are present, too, but some of them contains files, which are not in the override directory and get not biffed with your biffmeister. So it could be, that these files will be missing later.

At last, thanks for your biffmeister, it speed up my tests.
And be sure, to double check the biffing, which you offer.

Greetings Leomar

[Lollorian]

[quote name='Leomar' date='21 January 2010 - 08:16 PM' timestamp='1264085209' post='476242'][quote]I just noticed that BWP's cleanup.bat corrupts the installation directories of Rogue Rebalancing and aTweaks by deleting RR#EVAL.2DA. Although that file is 0 bytes in size, it gets filled up with information by several RR and aTweaks components at install time. Deleting it causes an install failure, so please don't do that.

If it helps, in the next version of aTweaks and RR I'll slightly increase the size of RR#EVAL.2DA by adding a header to the file. It won't be zero bytes then.[/quote]It would be nice if you can update your Clean-UpX.bat to consider it. [/quote]The X.bat (as well as the original Cleanup.bat) isn't supposed to be run before installing anything

But, if it helps, the X.bat only deletes .2da stuff from the RR folder and not the override. The override will still have any 2da's (if they remain after being gobbled up by gen_biff that is ) ... so running both Cleanup.bat's AFTER installing in safe

[quote name='Leomar' date='21 January 2010 - 08:16 PM' timestamp='1264085209' post='476242']Like you know, I'm running some installation tests, because of the CtB/NTotSC problem. I used your batches to speed up my installations and it worked great. The no backup option of the override directories fasten it really up.

...

At last, thanks for your biffmeister, it speed up my tests. [/quote][/quote]YAY!!! <victory rain dance>

[quote name='Leomar' date='21 January 2010 - 08:16 PM' timestamp='1264085209' post='476242']The reduced biffing works fine, too, but I'm not sure, if all is correct. I know, that you've looked for bif's which must be present for other mods and I don't know the whole biffing process of the big BG1 and BG2 mods. But you should test it carefully and look at each mod with the reduced biffing option, if all is correct. What does the mod do before biffing... What are the biffing options... Does the mod copies/compile all mods in the override directory or in an own self created directory, which will be biffed after that....
Otherwise it could be a new source of error, because of missing files. Only my thoughts. [/quote]Ok, here's the step-by-step process of what happened before and what happens now

[u]Before:[/u]

• .tp2/.bat associated with mod creates biff container folders (TDD-CRE, NEJ-ARE, XXX-YYY etc)
  
• .tp2 copies decompiled stuff (baf/d) to said folders
  
• .tp2 copies itm/cre/sto stuff in said folders
  
• .tp2 compiles all decompiled stuff (.baf/.d) in said folders and creates bcs/dlg etc
  
• .tp2 finishes and starts .bat
  
• .bat unpacks all tiz/ogg's to said folders as tis/wav's
  
• .bat does --make-biff of all biffing folders (which only biffs files that BGII can use ... leaving all the .baf/d/tiz/ogg's in the folders)
  
• .bat deletes all biffing folders

[u]After:[/u]

• .tp2 creates ONLY folders that contain stuff that isn't patched (mve/mos/tis/wav etc)
  
• .tp2 copies decompiled stuff (baf/d) to override
  
• .tp2 copies itm/cre/sto stuff to override
  
• .tp2 compiles all decompiled stuff (.baf/.d) in override and creates bcs/dlg etc
  
• .tp2 finishes and starts .bat
  
• .bat unpacks all tiz/ogg's to biffing folders as tis/wav's
  
• .bat does --make-biff of all biffing folders
  
• .bat deletes all biffing folders and also all packed/decompiled stuff in override

[quote name='Leomar' date='21 January 2010 - 08:16 PM' timestamp='1264085209' post='476242']What I've recovered so far and should be checked:
CtB creates CTB-* folders, like NTotSC (NTotSC1, etc.), but after the installation, some of the folders are still present and not deleted. All the folders are empty, so it looks fine, but the mod preparing folders should be removed. [/quote]I've fixed that in the RC1 (stupid lol I know)

[quote name='Leomar' date='21 January 2010 - 08:16 PM' timestamp='1264085209' post='476242']The only thing I'm wondering about is SoS:
The difference between SoS and CtB, NTotSC, etc. is, that the preparing folders are present, too, but some of them contains files, which are not in the override directory and get not biffed with your biffmeister. So it could be, that these files will be missing later.[/quote]Fixed in RC1 (you're using the very first version of the biffmeister lol It has LOTS of changes now )

[quote name='Leomar' date='21 January 2010 - 08:16 PM' timestamp='1264085209' post='476242']And be sure, to double check the biffing, which you offer. [/quote]Man, if anyone wants, I can But methinks I now need someone else to do this with me ... Vista/Win7 users etc

[Leomar]

Oh, another thing.

If your Biffmeister package must be extracted after the extraction of the mods and before the execution of the Install.bat, then you must consider the BW Smoothpack. Otherwise the BW Smoothpack patches will not be installed, because the informations in the original batch-files are changed for your biffing purposes. You should go through the BW Smoothpack patches and add the changes to your batches, so they include them. Then it is not important if the BW Smoothpack cannot patch the files, because the changes are there.

Oh, and I see, that you deleted the override backup lines in the TP2s of the mods. That is not the correct way. Please comment them out in the same line like:

/* the first line you want to comment out
here is the last line */

Otherwise you've changed the content/lines of the TP2s and the patches of the BW Fixpack could get problems to patch those. For best patch results, you should not add or delete a line in the TP2.

Greetings Leomar

Edited by Leomar, 21 January 2010 - 07:19 AM.

[Lollorian]

If your Biffmeister package must be extracted after the extraction of the mods and before the execution of the Install.bat, then you must consider the BW Smoothpack.

Considered both smoothpack and fixpack during tests It works (dunno about textpack though )

Oh, and I see, that you deleted the override backup lines in the TP2s of the mods. That is not the correct way. Please comment them out in the same line.
Otherwise you've changed the content/lines of the TP2s and the patches of the BW Fixpack could get problems to patch those.

Of course (but it still doesn't interfere with the fixpack in the RC1 ) ... I'll probably need to reupload anyway (for minor typos and extremely negligible stuff ) But the current biffmeister is 100% compatible with the BWP (and yes, I did a total of 7 BWPExpert installs to get this right )

[the bigg]

Otherwise you've changed the content/lines of the TP2s and the patches of the BW Fixpack could get problems to patch those. For best patch results, you should not add or delete a line in the TP2.

diff -u (that the BW Fixpack uses) can deal with deleted lines, so this is a non-issue. Besides, is there any reason why this shouldn't become standard for the BWF (if not outright included with the mods)?

[Leomar]

But, if it helps, the X.bat only deletes .2da stuff from the RR folder and not the override. The override will still have any 2da's (if they remain after being gobbled up by gen_biff that is ) ... so running both Cleanup.bat's AFTER installing in safe

It is not fully correct. For example: If you want to change (or install) some aTweaks components after the whole installation, you can't do it anymore, because of the missing file (o.k., who wants to do it? But I think we should not delete this option).

Ok, here's the step-by-step process of what happened before and what happens now

Thanks for the explanation.

Fixed in RC1 (you're using the very first version of the biffmeister lol It has LOTS of changes now )

All right, I'll use the updated files of the RC1 package for my today's installment.


Greetings Leomar

[Lollorian]

WAIT Use the RC2 (damn, probably missed you )

Only changed the Cleanup.bat and ... Leomar was right (2 hunks for TDD wasn't getting patched sory bout that) added it in

And about the Cleanup to tweak, I always believed the objective of the cleanup was to clean the BGII-SoA (remove mod directories and stuff) AFTER the install was fully tweaked and you were satisfied with what you have In order for the installers to function after Cleanup, you (theoretically) shouldn't delete anything

Attached Files

•  biffmeister_rc2.7z   499.48K   259 downloads

[the bigg]

2 hunks for TDD wasn't getting patched sory bout that

I'm sure this is unrelated (E.G. the officially available version of TDD now has those hunks built in). Diff can deal with a couple of deleted lines

[Lollorian]

It wasn't the missing lines I'd changed the MOVE's destination to ~override~ and originally, it moved to ~TDD-ITM~ and ~TDD-SPL~ (which don't exist now )

And thanks for your patience with answering all of my (probably stupid and n00blar) questions You're teh awesomest!!1!

[Leomar]

Of course (but it still doesn't interfere with the fixpack in the RC1 ) ... I'll probably need to reupload anyway (for minor typos and extremely negligible stuff ) But the current biffmeister is 100% compatible with the BWP

Not fully correct and I don't know why exactly. I've checked your "BiG World Debug" and you have some HUNKs in there. So for the safe side it would be better to consider the BW Smoothpack patches in your batches and comment out the override backup lines in the TP2. So nothing could be going wrong.

And if you want to do that, that is helpful for us, too. Because of thebiggs respond:

Besides, is there any reason why this shouldn't become standard for the BWF (if not outright included with the mods)?

If you have all the needed corrections in the batches and/or TP2s it is easier for us to create patches for the BWP tools, if we want to include it into the BWP. But it will not be included in the BW Fixpack, because these are not needed fixes. So it is usefull for the whole installation and for a faster installation process and it would fit perfectly to the BW Smoothpack.

Greetings Leomar

[Leomar]

2 hunks for TDD wasn't getting patched sory bout that

I'm sure this is unrelated (E.G. the officially available version of TDD now has those hunks built in). Diff can deal with a couple of deleted lines

But we fix it with the BW Fixpack and the HUNK Lollorian has, comes from our BW Fixpack patch, which gets problems, because of Lollorians TP2 changes.

Greetings Leomar

[Lollorian]

Heya again

About the failed hunks, let's have a look There are 4 failures in my last install:

Bardsong_IWD -> it's updated, not cause of biffmeister
TDD -> *grumble* *grumble* (fixed in rc2)
Hf_HHG -> it's updated. the new .bat uses -f to oggdec and it isn't being noted by the patch (not modded by biffmeister )
SC#Hubelpot -> I have no idea why this is failing the lines checked aren't modded by the biffmeister and it doesn't check for the oggdec -Q line

Attached both the patch and the original .bat and the biffmeistered .bat for someone to check the problem

Attached Files

•  Biff_Setup-HubAudioInstall.ba.txt   122bytes   198 downloads
•  Original_Setup-HubAudioInstall.bat.txt   123bytes   189 downloads
•  Setup-HubAudioInstall.bat.patch.txt   257bytes   201 downloads

[Leomar]

SC#Hubelpot -> I have no idea why this is failing

But we know. It is our fault and we'll fix it. In your case ignore the hunk, all is fine.



[EDIT]

Ohh, I saw, that you're not using a recent TP2. You've the TP2 of CtB v1.11 instead of v1.12...


Greetings Leomar

Edited by Leomar, 22 January 2010 - 10:50 AM.

[Lollorian]

Oh damn Sorry Here's an rc3 ... with CtB1.12

Attached Files

•  biffmeister_rc3.7z   499.77K   274 downloads