(I also prefer detecting other people's mods via markers because then I'm not hostage to their changes of TP2 structure.)
Not to keep nitpicking this technique, but this bugged me a bit. So you'd rather be hostage to a mod's marker files than its (hopefully) hardcoded component numbers? And what if the marker filenames changed with every version increase (as the bigg seemed to be suggesting was the case for some mod earlier)?
I don't think you're nitpicking - it's a perfectly reasonable question, but it shows that I'm not managing to explain what's going on here properly. So here's another go:
Component numbers are
not hardcoded, either in theory or (in complicated mods) in practice. The component number structure of
SCS and SCSII have changed at least three times in their four-year history (you can tell when this happens, it's the times when I tell people "you need to uninstall and reinstall"). Sometimes that's because of infelicities in my own organisation of components that I later recognise. More often, it's because I want to rearrange the order in which I present components to the player - for instance, I split the tactics part of both mods into "Tactical challenges" and "AI improvements" a year or two ago, which required the order to change; subsequently, I reclassified a few components ("harder spiders", notably) from one section to another, which again changed the order.
Now,
WEIDU component numbers need to be presented in order. (Or at least, not doing so is pretty dangerous at best.) So any time you rearrange so that component A occurs before component B rather than after it, necessarily you'll renumber at least one, and possibly both. So you cannot rely on hardcoded numbers to reidentify components on rearrangement.
For most mods, this isn't an issue. Ascension has six components; Wheels of Prophecy has one. But SCSII has about ninety, and it's crucial to organise them in ways that make them accessible and convenient for installers, and sometimes to change that organisation. That means that the only way I can use MOD_IS_INSTALLED is if I commit to changing all those numbers whenever I change the order. I'm very averse to doing that, because it's an absolute recipe for making mistakes.
Here's a slightly more computer-science way of putting it: the marker approach gets the logical structure of the situation right. Suppose that (say) Improved d'Arnise Keep requires Improved General AI to be installed. The marker approach checks, directly, whether that component is installed. The MOD_IS_INSTALLED approach checks something different: whether component 6000 is installed. Any time I change "Improved General AI" so that it is no longer component 6000, that requires a change in the MOD_IS_INSTALLED requirement even though the actual logic of the task hasn't changed at all.
Now, to be sure: component number changing isn't an everyday occurrence, and I try to do it as little as possible (partly to avoid breaking people's installs part-way through, partly out of a desire not to mess up automated install programs). And so if there really was a relevant downside to the marker approach, I'd find another way around (either by faking up my own version of
weidu.log, or by just being very careful updating all the various MOD_IS_INSTALLED lines). But since there really isn't any such downside (of which more below), I don't see the need.
As for detecting other people's mods: well, in general I don't trust
either MOD_IS_INSTALLED,
or other people's markers, and I try to find a more direct way to check. The concrete case I was thinking of was Spell Revisions, where the mod author has a strong interest in achieving
SCS compatibility and added the marker specifically for my convenience.
I'm not sure if there's a good enough argument here for me (or any other modder) to start installing .xxx files with their components if they don't already.
There isn't any such argument. If MOD_IS_INSTALLED works for you in your mods, that's fine with me; if I want to check compatibility between your mod and mine and I can't do it in a way that's satisfactory to us both, we can then liaise. (In the case of IR, I had a faithful promise that component 0 would remain component 0 even unto the end of time; that's fine too.)
Picking up more generally on the install-time issues, what I take from the discussion is that there is a slowdown proportional to the number of files in the override (whether they're markers or not). If that slowdown is bothering you, feel free to BIFF the override, markers and all.
The issue of things like .baf files in the override is slightly more interesting, as these aren't harmless if someone does something hugely stupid (i.e., try to COMPILE the override). My own view, frankly, is rather like Miloch's on creating folders in the override: namely, if your mod does that, it deserves to fail. But I'm somewhat more persuadable here if someone can give me a good reason (bearing in mind that fixing it in
SCS would (a) take a nontrivial amount of work, and (b) create a nontrivial risk of introducing bugs into
SCS itself). In any case, if install speed is what worries you, then again: just BIFF them. Harmless files sitting in your BIF files won't make any material difference to anything (there are more than a few such Bioware files, as I recall).
Getting back (somewhat) to the OT, there seems to be other rubbish in the override beyond the marker files, which could point to actual mod install issues.
The dw#*.mrk and dw#*.xxx are my marker files (I changed from xxx in
SCS to mrk in SCSII because xxx was triggering spam filters). Other dw#* files are bits of dialog and script that I build and compile in the override; they're all harmless (unless someone compiles the override) but equally, they don't serve any function.
Well, I do now realize how the EasyTutu is(and probably the IwD-in-BG2 are both) made, it's made by first copying the original game(BG1) content, then reindexing the files to the proper sets so the new game/engine can use them... but it also means that the new engine must overwrite or just ignore a whole lot of files because the originals are on the wrong format... which adds to the space bloat and loading time(because you have bigger mass to load the files from).
This isn't an issue. Neither
IWD-in-
BG2 (for sure) nor TUTU (I believe) use any engine-accessible file as the destination for their copied-over pre-modification files. And in
IWD-in-
BG2, at least, things basically only get copied into the override if that's their final destination. Files that are destined to be BIFFed get put in dedicated folders pre-biffing.
And considering the file and disc fragmentation is bad thing on any game, this just adds to it... I for one don't want to go and manually retreave the 1 byte from the Saturns third moon to calculate the 1+1=? equation.
I have no clue what this means.
And the originality of the situation was question if one should .bif the game files or not? And since you already have .bifs from some of the files, why not make one from all of them... after all the modifying is done
"Why not?" is an incredibly bad design principle in writing half-way complicated computer code. It leads to code bloat, confusion of the internal logic, run-time delays, and general chaos. The right question, always, is "why?" - i.e., before you spend time and effort coding some particular outcome, check if that outcome is actually part of what your code is trying to achieve.
(But since this thread is for
BWP brainstorming, and
IWD-in-
BG2 is one of the few mods which is guaranteed-completely-incompatible with
BWP, I'd better not digreess on its design much longer!)