216 replies to this topic

[Lollorian]

And finally, some good news!!! The install's now finished using the real biffmeister package

Earlier BWPExpert statistics from here:

Time - 6h 25m
BGII-SoA size - 27.5 GB
data size - 13.7 GB
override size - 24.2 MB

No. of gen_biff .bif's - 180

Now, presenting the latest BWPExpert intell-biffing edition statistics:

Time - 5h 35m
BGII-SoA size - 19.8 GB
data size - 9.55 GB
override size - 5.29 MB

No. of gen_biff .bif's - 87

Aside from the ~4 GB reduced data folder, I'm rather concerned about this 8 GB reduction of the BGII-SoA folder itself I'm not sure whether just the latest unpackers could've saved THAT much space

Gonna sleep now, after defragging ... Then I'll run the Cleanup.bat for moar space savings

EDIT: Statistics after CleanupX.bat:

BGII-SoA size - 15.6 GB
data size - 9.55 GB
override size - 1.03 MB

No. of gen_biff .bif's - 87

Edited by Lollorian, 12 January 2010 - 10:44 PM.

[Leomar]

Ok, these are the mods that make a backup of the entire override and chitin.key when installed (some of them are commented out in the .tp2 but I list them for completeness sake )

//These still make backups of the override, dialog.tlk & chitin.key

DrizztSaga
BGT
CtB
PlanarSphereMod
SoS
TDD
TS-BP

//These have the backup lines commented out

AzenMod
RevisedBattles
Vault

Btw, Revised Battles has 3 .tp2's within it, ImpBattles.tp2, ImprovedBattles.tp2 and NythrunFiXBGT.tp2 What are they used for??

The following mods backup the whole override folder, too:

- DSotSC
- NTotSC
- SoBH

For completeness sake.

Utility updater
◦After unpacking this to your BGII - SoA folder, run the Unpacker_updater.bat to update ALL your tisunpacks and oggdecs to the latest version (included)

Which new versions do you use? Can you give us the links?

Greetings Leomar

Edited by Leomar, 16 January 2010 - 12:46 PM.

[Lollorian]

I'm just pretty slow sometimes is all Thanks!!! Here's an updated biffmeister 2000 with all the goodness and greatness of no backups, better biffing and ... stuff I forgot And an updated backup culprit list ...

//These still make backups of the override, dialog.tlk & chitin.key

DrizztSaga
BGT
CtB
PlanarSphereMod
RoT
SoS
TDD
TS-BP
DSotSC
NTotSC
BoneHill

//These make copies of just the keyfile and dialog.tlk

BP-BGT-Worldmap
BoneHill245
NTotSC171
AzenMod
RevisedBattles
Vault
ImpBattles      //Revised Battles
ImprovedBattles //Revised Battles
NythrunFixBGT   //Revised Battles

//And this guy copies the keyfile, dialog.tlk, dialogf.tlk and just AR0700 and AR5500.bcs

ToA

... goes withour saying that none of them do ANY backing up in the package

Which new versions do you use? Can you give us the links?

The attached package has the latest versions called X's But these are where I got them from (last modified date)

tisunpack.exe - bg1ub\tis\ (20/6/2007)
oggdec.exe - wheels\ (3/1/2009)

There probably are better ones, but I found none on google

Cheers,
Lol

Attached Files

•  biffmeister 2000.7z   336.28K   351 downloads

[Miloch]

The latest oggdec is something like 1.9.6. But I found no noticeable speed difference between using that and the slimmer 1.0.1. There is only 1 version of tisunpack that I know of (0.91) - hasn't been updated in ages. I could be wrong, but I think you might be just wasting time by copying newer versions into all the mod folders. On the other hand, it might make sense to delete all the duplicate files and just launch them all from a central 'tools' folder or the like, but that would involve changing a lot of mod batch files. If you're going to go that far, you may as well convert all .ogg files to .wavc instead of .wav and then biff those, because you'll probably get significant space savings.

[Lollorian]

ABout the common tools thing, Helborn's doing something along those lines

And about wavc conversion, I found a wavc.exe here, but dunno what formats it can convert from

And, if possible, is there a readymade way to do the ogg->wavc conversion??

Those oggdec/tisunpack stuf you linked to are different and older than what I got Why are mine newer (and smaller)??

[Miloch]

And about wavc conversion, I found a wavc.exe here, but dunno what formats it can convert from

Try wav2wavc, also packaged in Sam's psgui. It only converts WAVs, but you could ask him to include an OGG conversion tool if there isn't one already (he just added an MP3 one I believe). Or you could use Audacity to convert OGGs to WAVs and then PS Gui to convert those to WAVCs, hopefully without too much loss (OGGs are already lossy though).

Those oggdec/tisunpack stuf you linked to are different and older than what I got Why are mine newer (and smaller)??

The only reason they appear "newer" is because someone copied or archived them with the current date set (rather than the original date). Just type oggdec at the command prompt or tispack -V to see the versions. They're 1.01 and 0.91, the same versions as packaged with Aurora. Like I said, I didn't see a reason to use a higher version of oggdec, but you could test it I suppose... maybe there's a marginal speed difference. Could be slower even, who knows.

Someone should maybe look at converting what you've done to a patch, so you don't overwrite a bunch of mod files, and you probably don't want to copy (or patch) them at all if they're not there. Once again though, if you're biffing BAMs before Infinity Animations is installed, that's a bad idea.

[Lollorian]

It only converts WAVs, but you could ask him to include an OGG conversion tool if there isn't one already (he just added an MP3 one I believe). Or you could use Audacity to convert OGGs to WAVs and then PS Gui to convert those to WAVCs, hopefully without too much loss (OGGs are already lossy though).

I didn't know PSGUI supplied the unpackers too

And about Audacity ... won't that practically mean we should give away wavc'ed files instead?? (since the BWP can't possibly expect Audacity to be present in every system ) Download size bloatage matey??

The versions in the biffmeister are 1.01 and 0.91 too (but just to be on the safe side, I'm using the PSGUI ones now )

Someone should maybe look at converting what you've done to a patch, so you don't overwrite a bunch of mod files, and you probably don't want to copy (or patch) them at all if they're not there.

That'd be awesome

Once again though, if you're biffing BAMs before Infinity Animations is installed, that's a bad idea.

Next on the agenda - no BAM biffing (should be easy now, since we've isolated all the .bat's that DO biff )

[Miloch]

And about Audacity ... won't that practically mean we should give away wavc'ed files instead?? (since the BWP can't possibly expect Audacity to be present in every system ) Download size bloatage matey??

I'm not saying you should include Audacity, especially not if there's a decent command line OGG converter (I'm sure there is somewhere). The difference between OGGs and properly converted WAVCs is fairly small, and what you might be gaining in download size, you're saving in total install size and time because OGGs get extracted to WAVs which are larger than WAVCs. Plus it takes time to deogg them, whereas WAVCs can be copied as-is.

[Lollorian]

About converting ogg's to wavc's, methinks it'd be better to ask someone to create an ogg2wavc.exe (if it's possible) Does Sam. take requests??

Anyway, here's da biffmeister 3000:

• Doesn't biff BAMs: Has the advantage of letting item bams be overwritten by the mods that give them new (different) bams and overall blatant (and fantastically marvellous ) support for IA
  
• Stable unpackers: The default unpackers are the ones that PS_GUI provides (but I included the old ones as _old.exe's since the PS_GUI oggdec seems very large compared to the one I used )

(reuploaded cause the PS_GUI oggdec didn't use -q So that proves that atleast the oggdec got changed )

Attached Files

•  biffmeister 3001.7z   472.47K   277 downloads

Edited by Lollorian, 18 January 2010 - 04:48 AM.

[Lollorian]

DO NOT use the biffmeister I just discovered the cause of the miraculous reduction in space and time for my last install

Ok, I discovered I just accidently slipped in Exnem's in my latest install, so I'll have to do this stuff again

Anyway, there's something stupid going on with the oggdecs ...

The version I used (earlier from Wheels of Prophecy) was smaller and was the newest version of the file I found. It has a silent tag (-Q) and stuff (which the PS_GUI oggdec doesn't seem to have) ... Now thing is, the WoP oggdec can't process wildcards So anything that reads:

oggdec -Q *.exe

returns an Invalid Argument error

Will probably need to use a FOR loop for this now But, if possible, we could use some kinda ogg->wavc stuff right now (that can use wildcards )

EDIT: This biffmeister should work for all installs now (it uses PS_GUI's oggdec so it isn't silent) but I thought it'd be better than editing some 72 files to add in a for loop (some of which have different types of wildcard usage) Sorry for the trouble!!

Since the attachment system isn't working (along with my pm system)

Edited by Lollorian, 18 January 2010 - 08:03 AM.

[Sam.]

Okay, here is my official 2 cents. The TISpack and TISunpack that are included with PS gui are the newest version available (v0.91) according to jcompton who is the official IE Community contact for Per Olofsson. There are mods, however, that ship with an older version. Classic Adventures uses an older version of TISunpack, for instance. The difference that I notice is the version CA uses does not have the -e (Do not halt on errors) and -V (Print version number and exit) parameters. The TISpack and TISunpack shipped in Cuv's My Lil' Toolset are also really old versions.

I use OggDec v1.9.6 (2008-05-30) with PS gui because it is the newest version I could find and has a wider selection of parameters that offer much more control over the conversion. These are the optional parameters:

OGGDEC.EXE - Copyright © 2002-2008 John Edwards

 Usage: oggdec [options] input.ogg [...]

 OPTIONS
 -h, --help 	Prints this help information.
 -a, --album 	Use ReplayGain Album/Audiophile gain settings, OR
 -r, --radio 	Use ReplayGain Track/Radio gain settings.
 -m, --downmix	Downmix multi-channel vorbis to stereo output.
 -d, --dither X Dither output, where X =.
 	0 for 	dither OFF (default).
 	1 for 	dither without Noise Shaping.
 	2 for 	dither with Light Noise Shaping.
 	3 for 	dither with Medium Noise Shaping.
 	4 for 	dither with Heavy Noise Shaping.
 -p, --play 	Plays ogg files thru the soundcard using Windows audio, OR
 -s, --shuffle	Shuffles and then plays ogg files using Windows audio.
 -o, --stdout 	Writes output to stdout.
 -w, --wavout name.wav Writes output to file 'name.wav'.
 	(Only valid for single file input.)
 PLAYBACK PRIORITY OPTIONS
 -c, --class X	Set playback priority class, where X =
 	0	for 	NORMAL priority.
 	1	for 	HIGH priority (default).
 	2	for 	REALTIME priority.
 FORMAT OPTIONS
 -b, --bits X	Set output sample format, where X =
 	1	for 	Unsigned 8 bit PCM data.
 	2	for 	16 bit PCM data (default).
 	3	for 	24 bit PCM data.
 	4	for 	32 bit PCM data.
 	5	for 	32 bit floats.

 INPUT FILES
 OggDec input files must be Ogg Vorbis I files with
 a sample rate of 48000Hz, 44100Hz, 32000Hz, 24000Hz,
 22050Hz, 16000Hz, 12000Hz, 11025Hz or 8000Hz.
 Wildcards (?, *) can be used in the filename.

As you can see, it supports wildcards.
There is also a OggDec 1.0.1 which is a much older version which does not support wildcards, but does have a "quiet mode":

OggDec 1.0.1
Usage: oggdec [flags] file1.ogg [file2.ogg ... fileN.ogg]

Supported flags:
 --quiet, -Q 	Quiet mode. No console output.
 --help, -h 	Produce this help message.
 --version, -v	Print out version number.
 --bits, -b 	Bit depth for output (8 and 16 supported)
 --endianness, -e Output endianness for 16 bit output. 0 for
 	little endian (default), 1 for big endian
 --sign, -s 	Sign for output PCM, 0 for unsigned, 1 for
 	signed (default 1)
 --raw, -R 	Raw (headerless) output.
 --output, -o 	Output to given filename. May only be used
 	if there is only one input file

I could add the older version of OggDec to PS gui and have a settable option in the config that determines which version to display the parameters for and run. However, if the intention here is to convert all of the OGG files to WAVC and then repackage them so that no conversion is needed at install-time (which is what should ultimately be done), then I don't see why you need to use wildcards at all. You can select and convert just about as many files as you want at once with PS gui. (I say "just about" because I tried doing over 7000 files at once and something clogged. Haven't figured out what the limiting factor was yet.)

As for an ogg2wavc.exe, I could add one to PS gui, but it wouldn't be a one step deal (by that I mean the conversion wouldn't go strait from .OGG to .WAVC). I could make it a one step conversion (in the gui) with all the defaults set to be as quick and quiet as possible, but it would require using OggDec.exe to go from .OGG to .WAV then wavc.exe to go from .WAV to .ACM then my code to go from .ACM to .WAVC. It would save a good bit of time, regardless.

[Miloch]

I think the quiet mode is more useful than any of those newer options. Not sure why they removed it even. It'd be one thing if those options let you change the sample rate (not sample format), but why would you want "dithering" (whatever that even means for sounds)? Wildcards can be useful, but if you're converting your own mod's sounds, you know what those are and it's quite easy to generate a batch file that does them all.

Edit: the only concern I have with converting OGGs to WAVCs (even if it takes several steps) is if theres's any (further) loss. If it's not noticeable, that's one thing, but if the clips sound degraded after all that conversion it may not be worth it. Also, they'd need to be 22khz as I previously mentioned.

Edited by Miloch, 18 January 2010 - 04:09 PM.

[Lollorian]

Just finished another test (original) biffing install without the biffmeister

Statistics:

Time: 5h 33m

BGII-SoA: 29.7 GB
Data: 14.4 GB
Override: 9.8 MB

tb#biffs: 207

And Statistics after CleanupX:

BGII-SoA: 23.7 GB
Data: 14.4 GB
Override: 5.37 MB

I installed IA with this one, so that's the 2 GB bloat and all the errors I got were IDS errors

Attached BWP debug.txt and WeiDU.log

Attached Files

•  BiG World Debug.7z   378.3K   270 downloads
•  WeiDU.7z   20.85K   185 downloads

Edited by Lollorian, 18 January 2010 - 06:46 PM.

[Sam.]

I think the quiet mode is more useful than any of those newer options. Not sure why they removed it even.

I'll add the option to use the older version in PS gui then.

It'd be one thing if those options let you change the sample rate (not sample format), but why would you want "dithering" (whatever that even means for sounds)?

Wikipedia.

Edit: the only concern I have with converting OGGs to WAVCs (even if it takes several steps) is if theres's any (further) loss. If it's not noticeable, that's one thing, but if the clips sound degraded after all that conversion it may not be worth it. Also, they'd need to be 22khz as I previously mentioned.

I had assumed the "original" sound files were no longer accessible and that the Ogg files were already in the correct format under the assumption that they had been converted from WAV files with the end use of being added to the IE. As far as quality loss, this is what ABel's Sound Comparator reports: (the 1st file is always the unmodified original)

Max. abs. value in the 1st file: 32758
Max. abs. value in the 2nd file: 32324
Max. error: 4401
Mean square error: 444300
Root mean square error: 666.558

Max. abs. value in the 1st file: 32758
Max. abs. value in the 2nd file: 32237
Max. error: 6348
Mean square error: 455904
Root mean square error: 675.207

Max. abs. value in the 1st file: 32758
Max. abs. value in the 2nd file: 32733
Max. error: 6154
Mean square error: 15304.4
Root mean square error: 123.711

[Miloch]

I had assumed the "original" sound files were no longer accessible and that the Ogg files were already in the correct format under the assumption that they had been converted from WAV files with the end use of being added to the IE.

Well most mods probably only have the OGGs. Even Aurora, where we *should* have lossless source files, but I think most folks actually recorded their clips to OGGs.

As far as quality loss, this is what ABel's Sound Comparator reports

What does all that mean in practical terms? Is the loss difference significant? Can you hear it if you play both WAV and WAVC in NI?

[Lollorian]

Sorry for not replying about the oggdec's earlier

However, if the intention here is to convert all of the OGG files to WAVC and then repackage them so that no conversion is needed at install-time (which is what should ultimately be done), then I don't see why you need to use wildcards at all. You can select and convert just about as many files as you want at once with PS gui. (I say "just about" because I tried doing over 7000 files at once and something clogged. Haven't figured out what the limiting factor was yet.)

We (well, me at the least ) don't intend to repackage anything I just wanna edit the existing code (.tp2, .bat) to make the process take place according to the original megamod package, but a tad faster and bloat-free

So, ideally, I need an cmd-runnable ogg2wavc.exe OR an oggdec that can go silent and use wildcards as well (the latter seems favorable looking at Miloch's quality-loss argument )

In other news, I started the non-biffing install some 2 hours ago and its still running pretty well (no errors except the rupert typo yet and it's now finishing with the NPC mods ... I can see it oggdecing [while saying that -Q is an invalid argument ] )

Oh, and this one has IA too

And another thing I forgot to mention (from my first test install ) is that I'm using a slightly modified install.bat

• Removed crefixer and tob_hacks from BWPFixpack folder since they've been updated
  
• using the latest hotfixes for spell_rev's _copy folder
  
• removed redundant patches for CtB and Bardsongs_IWD from Fixpack
  
• moved all hotfixes from after gen_biff to before gen_biff
  
• added IA just after scsII, before L1NPCs with Miloch's recommended components
  
• removed Exnem's (cause it borked REEVOR) and NMR-Relationships (cause it pwned the dialog.tlk last time I played )

Dunno if it'll affect anything (didn't do anything yet ) but I thought I'd just mention this

Edited by Lollorian, 19 January 2010 - 08:15 AM.

[Lollorian]

Ok, something went horribly wrong I'd reached aurora and suddenly, the system started giving LOTS of continous beeps (the ones that come directly from the motherboard) ... the cmd window froze, I panicked and killed setup-aurora.exe, through the task manager and the beeps stopped (google says its a memory issue ... but with 2 GB RAM and a 4 GB swap, I seriously doubt that )

I now have an incomplete BWPExpert (which I won't delete) with an incomplete Aurora DEBUG ... But it'd help lots if someone could provide insight on wtf happened just now And most interestingly, the end of the BWPDebug,txt seems garbled up Is this my system saying no more biffjobs??

(The Debug.txt doesn't even compress properly and ended up 1.91 MB big Here's a mediafire link)

I tried restarting the install.bat minus the Fixpack and all the mods before surora and it's the same error again ... the cmd window shows garbled text and beeps start (the task manager shows 2 cmd.exe's and the setup-aurora, all of which I need to kill to stop the incessant beeps )

I'm really pissed off now!! WTF?!? (and for the record, the biffmeister doesn't modify any of aurora's files )

EDIT: Ok, having found some cool ... the problem seems to come from this block:

//! Sounds (Tutu, BG2 and BGT) //////////////////////////////////////////////
ACTION_IF (~%WEIDU_OS%~ STRING_EQUAL_CASE ~win32~ = 1) BEGIN //if Windows
  AT_NOW                   ~aurora/agkarain.bat~ //install
  AT_INTERACTIVE_UNINSTALL ~aurora/agkaraun.bat~ //uninstall
END ELSE BEGIN //if Linux or OS X
  AT_NOW                   ~aurora/agkarain.sh~ //install
  AT_INTERACTIVE_UNINSTALL ~aurora/agkaraun.sh~ //uninstall
END

where agkarain.bat is:

@echo off
IF NOT EXIST "override\agka001.wav" goto install
goto end

:install
aurora\oggdec.exe -o override\agka001.wav aurora\taudio\agka001.ogg
aurora\oggdec.exe -o override\agka002.wav aurora\taudio\agka002.ogg
aurora\oggdec.exe -o override\agka003.wav aurora\taudio\agka003.ogg
aurora\oggdec.exe -o override\agka004.wav aurora\taudio\agka004.ogg
aurora\oggdec.exe -o override\agka005.wav aurora\taudio\agka005.ogg
aurora\oggdec.exe -o override\agka006.wav aurora\taudio\agka006.ogg
aurora\oggdec.exe -o override\agka007.wav aurora\taudio\agka007.ogg
aurora\oggdec.exe -o override\agka008.wav aurora\taudio\agka008.ogg
aurora\oggdec.exe -o override\agka009.wav aurora\taudio\agka009.ogg
aurora\oggdec.exe -o override\agka010.wav aurora\taudio\agka010.ogg
aurora\oggdec.exe -o override\agka011.wav aurora\taudio\agka011.ogg
aurora\oggdec.exe -o override\agka012.wav aurora\taudio\agka012.ogg
aurora\oggdec.exe -o override\agka013.wav aurora\taudio\agka013.ogg

:end
::::

And the oggdec in aurora IS overwritten by PS_GUI's oggdec but it can use -o as a parameter ...

(bless thebigg for the Adblock tip )

Attached Files

•  SETUP-AURORA.7z   13.28K   194 downloads

Edited by Lollorian, 19 January 2010 - 09:29 AM.

[the bigg]

Are you sure that the -o does exactly what you mean? It's possible that it's writing to the standard output instead (which is a valid explanation for both the beeps and the incomprimible and garbled debug file).

[Lollorian]

Damn that's right Sam.'s first post in this page says that:

-o, --stdout Writes output to stdout.

... I didn't nab any oggdec.exe's in my search for stuff to silence I just searched for "oggdec "

So, it isn't my system acting up then??

EDIT: Ok, so that's fixed The biffmeister's gonna include aurora's .bat wth -w instead of -o

Meanwhile, in testingland, the install's resumed but now, I probably won't be able to tell ya the time cause of my ... panicky-ness-ification (and what's the big idea naming something that affects BGT, agtutuin.bat?? )

EDIT2: So while aurora finally installs, a proposition, BGTTweaks has this thing that "Removes duplicate scripts" and proceeds to do some stuff that looks good to read

Does this do what I think it does?? Does it remove duplicate blocks of the same script?? If yes, it'd be awesome to make it a standalone component and run it like the crefixer or something at the end

Edited by Lollorian, 19 January 2010 - 10:20 AM.

[Sam.]

As far as quality loss, this is what ABel's Sound Comparator reports

What does all that mean in practical terms? Is the loss difference significant? Can you hear it if you play both WAV and WAVC in NI?

Personally, I don't notice a difference. The quality loss that occurs in the Ogg conversion is probably mostly overlapped by the quality loss in the ACM conversion, so the end result is probably not very different.