Jump to content


Photo

Mass file change log tool

Started by Sasha Al'Therin, Apr 28 2012 11:17 AM

  • Please log in to reply
12 replies to this topic

#1 Sasha Al'Therin

Sasha Al'Therin
  • Modder
  • 615 posts

Posted 28 April 2012 - 11:17 AM

Sometimes I like to whip up something just to see if it is possible....

The change-log processes noted here was the basis for this idea. While its a useful tool it does require knowing how to edit a file and how to use that file. I've had the pleasure to run across some individuals who did not understand what to do.

This tool, therefore, is designed with those individuals in mind who just can't figure it out.

There are two options
1) Run Change Log on Override Directory Contents

This option launches a batch file which will scan the contents of your override directory and pass the file name into Weidu's command line --change-log routine. It only works with those files in your override directory. If your game has been biffed you will miss out on many modified resources. It takes a long time. Any files that have been modified have the corresponding backed up versions copied and numbered to match the change-log text entries. Any files that have not been modified have an empty text entry appended to the change-log text file.


This option is not for you if any of the following apply:

a) low drive space

b) lack of time


Example of a file of each type (comments in bold not part of output):

Mods affecting 25ELM14.WAV: <--This file exists but had no mods affecting it. It may be pre-existing or mod added



Mods affecting 25SPELL.DLG: <--This file was affected by 4 different mods and would have 4 matching files
00000: /* from game biffs */ ~BG2FIXPACK/SETUP-BG2FIXPACK.TP2~ 0 0 // BG2 Fixpack - Core Fixes: v9.01
00001: ~SETUP-BONEHILLV275.TP2~ 0 0 // Secret of BoneHill (Requires BGT or Tutu): v2.75a
00002: /* from game biffs */ ~SETUP-KIVAN.TP2~ 0 7 // Install ToB Portion of the Mod: v13
00003: ~BG2_TWEAKS/SETUP-BG2_TWEAKS.TP2~ 0 130 // Force All Dialogue to Pause Game: v9


2) Run Change Log on Selected File(s)

This is the best choice. It will work with any game setup including those that have been biffed. The user is asked to input the file name including extension. A change-log is then processed. If there were any changes, then the backed up versions are copied and numbered to match the change-log text entries. If there were no changes or the text entered for the file name is garbage, then only an empty text entry is appended to the change-log text.


Example of a file of each type (comments in bold not part of output):

Mods affecting FRESD.K.AILKA: <--This is a non-existant file, therefore nothing affected it.



Mods affecting WORLDMAP.MOS: <--This file was affected by only one mod and would have a matching file
00000: /* acted upon in an indetectable manner */~SETUP-BP-BGT-WORLDMAP.TP2~ 0 0 // Worldmap for Baldur's Gate - including colored Baldur's Gate map icons: v9.0.1


When finished text is displayed that tells where the change-log file is located and where the mod backup versions (if any) were stored. User is also asked if they wish to display the change-log text at that time. If yes, notepad is opened and the contents can be viewed. Recommendation is to not view the contents when doing the override directory. Selected file(s) option has a much typically smaller size and is safe to view in notepad.

Things to note:
  • Uses NO_LOG_RECORD which means you can interrupt the process or run it as many times as you want. However, this also means that any backup files that get recorded during the change log process will remain indefinitely. It is recommended to delete those files prior to doing another change log run. If NO_LOG_RECORD is not used then it will be treated as a regular mod and if you forgot that it was ran and added additional mods afterwards, when running it again it would uninstall all the mods following thus defeating the purpose of running the change log.
  • It is designed for Windows OS. If someone wishes to have a version capable of operating on a different OS, they will need to adapt the tool.
  • Sam pointed a few things out that I had missed. While working to solve that, I ran across other issues. Everything has been addressed and it fully functions as is intended. It remains as BETA simply due to it's 'working theory' nature
Download link:
MassFileChangeLog Beta 2

Beta 2 Changes:
-- learned how to make the whole override directory change-log work properly - reimplemented
-- simplified the one file at a time option. Earlier additions made it too complicated and limited its functionality.
Beta 1.5 Changes:
-- component 1 was dropped due to issues where the tool would not recover if an existing file is too large to be change-log'd by weidu.
-- added in additional exit option during the file name entry request stage.
-- added in additional recovery routines should input data not meet expected form.

Edited by Sasha Al'Therin, 30 April 2012 - 12:53 PM.

My working mods:
an AI Party Script for BG2 game engine DOWNLOAD LINK ONLY!
Interactive Tweaks for BG series with some IWD support. DOWNLOAD LINK ONLY!
Rest For 8 Hours an IWD mod
-------------------------------------------
My contributions: BG1Fixpack, BG1Tweaks
On Hold: Solestia an NPC for SOA
-------------------------------------------
My website: http://sasha-altheri...s.com/index.htm

#2 Sam.

Sam.
  • Administrator
  • 1335 posts

Posted 29 April 2012 - 11:42 AM

Interesting. Have you seen Mod Tracker? Does yours operate on different or the same basic principles?

Edit: And in my opinion, this should probably be under IE Modding Tools, not Misc Released Mods... :)

Edited by Sam., 29 April 2012 - 11:45 AM.

"Ok, I've just about had my FILL of riddle asking, quest assigning, insult throwing, pun hurling, hostage taking, iron mongering, smart-arsed fools, freaks, and felons that continually test my will, mettle, strength, intelligence, and most of all, patience! If you've got a straight answer ANYWHERE in that bent little head of yours, I want to hear it pretty damn quick or I'm going to take a large blunt object roughly the size of Elminster AND his hat, and stuff it lengthwise into a crevice of your being so seldom seen that even the denizens of the nine hells themselves wouldn't touch it with a twenty-foot rusty halberd! Have I MADE myself perfectly CLEAR?!"

--<CHARNAME> to Portalbendarwinden

--------------------

post-10485-0-15080600-1348188745.jpg
___________Old pen and paper modules of the 70s and 80s.___________

CA Forums CA Homepage

Spoiler

#3 Sasha Al'Therin

Sasha Al'Therin
  • Modder
  • 615 posts

Posted 29 April 2012 - 12:04 PM

Interesting. Have you seen Mod Tracker? Does yours operate on different or the same basic principles?

Edit: And in my opinion, this should probably be under IE Modding Tools, not Misc Released Mods... :)

I don't know about Modding Tools... doesn't help with modding per se. it helps more with tracking down problems. ^^

Never heard of mod tracker, but I'll look at it. All this does is run weidu's --change-log feature on the given files within a weidu enviornment.

After looking at it, I can't even get Mod Tracker to work. Can't locate all the mod folders cause some mods don't use a folder name that matches the tp2/exe name. And it won't let you change specific folder entries. It's all or none, so that tool is a fail in my book...


But if moderators wish to change the locale, that's their call ^^

My working mods:
an AI Party Script for BG2 game engine DOWNLOAD LINK ONLY!
Interactive Tweaks for BG series with some IWD support. DOWNLOAD LINK ONLY!
Rest For 8 Hours an IWD mod
-------------------------------------------
My contributions: BG1Fixpack, BG1Tweaks
On Hold: Solestia an NPC for SOA
-------------------------------------------
My website: http://sasha-altheri...s.com/index.htm

#4 Sam.

Sam.
  • Administrator
  • 1335 posts

Posted 29 April 2012 - 03:09 PM

And it won't let you change specific folder entries. It's all or none, so that tool is a fail in my book...

Granted, it has been a while since I have run the thing, but

However, you can edit MTCUSTOM.ModTracker to specify overrides, the format is based on
"MODNAME:MODFOLDER/BACKUP" eg "BP-BGT-WORLDMAP:BP-BGT_Worldmap/BACKUP"

sounds exactly like changing specific folder entries to me. ;)

Edit: I just tried to run it and think I see what you mean... Give me a minute...
Edit2: If I paste the game path in front of the entries in the "Mod Folder" column, it works fine. Interestingly, I don't remember having to do that before. I'll try yours next and see how it compares :)

Edited by Sam., 29 April 2012 - 03:25 PM.

"Ok, I've just about had my FILL of riddle asking, quest assigning, insult throwing, pun hurling, hostage taking, iron mongering, smart-arsed fools, freaks, and felons that continually test my will, mettle, strength, intelligence, and most of all, patience! If you've got a straight answer ANYWHERE in that bent little head of yours, I want to hear it pretty damn quick or I'm going to take a large blunt object roughly the size of Elminster AND his hat, and stuff it lengthwise into a crevice of your being so seldom seen that even the denizens of the nine hells themselves wouldn't touch it with a twenty-foot rusty halberd! Have I MADE myself perfectly CLEAR?!"

--<CHARNAME> to Portalbendarwinden

--------------------

post-10485-0-15080600-1348188745.jpg
___________Old pen and paper modules of the 70s and 80s.___________

CA Forums CA Homepage

Spoiler

#5 Sam.

Sam.
  • Administrator
  • 1335 posts

Posted 29 April 2012 - 03:37 PM

AB_MASSFILECHANGELOG failed to install.

Spoiler


"Ok, I've just about had my FILL of riddle asking, quest assigning, insult throwing, pun hurling, hostage taking, iron mongering, smart-arsed fools, freaks, and felons that continually test my will, mettle, strength, intelligence, and most of all, patience! If you've got a straight answer ANYWHERE in that bent little head of yours, I want to hear it pretty damn quick or I'm going to take a large blunt object roughly the size of Elminster AND his hat, and stuff it lengthwise into a crevice of your being so seldom seen that even the denizens of the nine hells themselves wouldn't touch it with a twenty-foot rusty halberd! Have I MADE myself perfectly CLEAR?!"

--<CHARNAME> to Portalbendarwinden

--------------------

post-10485-0-15080600-1348188745.jpg
___________Old pen and paper modules of the 70s and 80s.___________

CA Forums CA Homepage

Spoiler

#6 Sasha Al'Therin

Sasha Al'Therin
  • Modder
  • 615 posts

Posted 29 April 2012 - 04:10 PM

I'll admit. I just downloaded the file and tried to run it. didn't look for any specific instructions

still after taking a good while to correct all the entries it couldn't find (there being 11 of 'em on a fairly lite BWP install). it would error out on the BWP related directories cause there was no uninstall information, they are merely dummy entries into the weidu.log so that there is a 'record' that something was done. The info is crammed together making it difficult to read etc... If someone took the time to get it to function with their setup, then it would be better than the option 1 in my little tool here.

now in my BWP setup that's got everything it would allow together all at once.... had to add an additional 40+ mods to the list... and it fails to work because of mods that have spaces or other special characters in the directory names. with all that time spent hunting up backup directories and modifying the tracker file for the mod tracker tool to work. I'd have to say that it's not worth the hassle, just to find out that you have to go back and rename mod directories just for the tool to work... then you gotta remember to change the directory names back so that the mods themselves will work if you need to uninstall for any reason.

even with it's limitations the change-log batch file setup or this change log mod of mine would be preferable.

My working mods:
an AI Party Script for BG2 game engine DOWNLOAD LINK ONLY!
Interactive Tweaks for BG series with some IWD support. DOWNLOAD LINK ONLY!
Rest For 8 Hours an IWD mod
-------------------------------------------
My contributions: BG1Fixpack, BG1Tweaks
On Hold: Solestia an NPC for SOA
-------------------------------------------
My website: http://sasha-altheri...s.com/index.htm

#7 Sasha Al'Therin

Sasha Al'Therin
  • Modder
  • 615 posts

Posted 29 April 2012 - 04:12 PM

AB_MASSFILECHANGELOG failed to install.

Spoiler

actually it worked. did you look in the location for the output file?
It uses NO_LOG_RECORD so there won't be an entry in the weidu.log
highlighted the successful install marker in that debug quote

*************
If anything failed it was Miloch for making t-1000.mos too big for weidu to copy and get the name from so it could not be used in the change-log feature LOL

Edited by Sasha Al'Therin, 29 April 2012 - 04:19 PM.

My working mods:
an AI Party Script for BG2 game engine DOWNLOAD LINK ONLY!
Interactive Tweaks for BG series with some IWD support. DOWNLOAD LINK ONLY!
Rest For 8 Hours an IWD mod
-------------------------------------------
My contributions: BG1Fixpack, BG1Tweaks
On Hold: Solestia an NPC for SOA
-------------------------------------------
My website: http://sasha-altheri...s.com/index.htm

#8 Sam.

Sam.
  • Administrator
  • 1335 posts

Posted 29 April 2012 - 04:19 PM

Did it not choke and stop analyzing files when it hit the one that was too big for WeiDU?

Examining what mods if any affected t-1000.mos ....
ERROR: [override/T-1000.tis] has size 18841624: TOO BIG FOR WEIDU (max 16777211)
ERROR: error loading [override/T-1000.tis]


"Ok, I've just about had my FILL of riddle asking, quest assigning, insult throwing, pun hurling, hostage taking, iron mongering, smart-arsed fools, freaks, and felons that continually test my will, mettle, strength, intelligence, and most of all, patience! If you've got a straight answer ANYWHERE in that bent little head of yours, I want to hear it pretty damn quick or I'm going to take a large blunt object roughly the size of Elminster AND his hat, and stuff it lengthwise into a crevice of your being so seldom seen that even the denizens of the nine hells themselves wouldn't touch it with a twenty-foot rusty halberd! Have I MADE myself perfectly CLEAR?!"

--<CHARNAME> to Portalbendarwinden

--------------------

post-10485-0-15080600-1348188745.jpg
___________Old pen and paper modules of the 70s and 80s.___________

CA Forums CA Homepage

Spoiler

#9 Sasha Al'Therin

Sasha Al'Therin
  • Modder
  • 615 posts

Posted 29 April 2012 - 04:27 PM

Did it not choke and stop analyzing files when it hit the one that was too big for WeiDU?

Examining what mods if any affected t-1000.mos ....
ERROR: [override/T-1000.tis] has size 18841624: TOO BIG FOR WEIDU (max 16777211)
ERROR: error loading [override/T-1000.tis]

I guess so

but check this out

Installing [Run Change Log on selected files] [BETA 1]
Copying 1 file ...
Copied [inlined_blank.txt] to [ab_MassFileChangeLog\change-log.txt]

Please type in the file you wish to run a change log on.
Example: somefile.ext
User answer: "t-1000.are"
[./override/t-1000.are] loaded, 20540 bytes
[./override/t-1000.are] loaded, 20540 bytes
[./override/t-1000.are] loaded, 20540 bytes
Copying and patching 1 file ...
[./override/T-1000.ARE] loaded, 20540 bytes
[./override/T-1000.ARE] loaded, 20540 bytes

Examining what mods if any affected T-1000.ARE ....

Do you wish to run a change log on another file?
[0] No
[1] Yes
User answer: "1"

Please type in the file you wish to run a change log on.
Example: somefile.ext
User answer: "t-1000.mos"
[./override/t-1000.mos] loaded, 173624 bytes
[./override/t-1000.mos] loaded, 173624 bytes
[./override/t-1000.mos] loaded, 173624 bytes
Copying and patching 1 file ...
[./override/T-1000.MOS] loaded, 173624 bytes
[./override/T-1000.MOS] loaded, 173624 bytes

Examining what mods if any affected T-1000.MOS ....

Do you wish to run a change log on another file?
[0] No
[1] Yes
User answer: "0"

and my output

Mods affecting T-1000.ARE:
00000: /* from game biffs */ ~XVARTS/SETUP-XVARTS.TP2~ 0 100 // Xvart Caverns: Beta2



Mods affecting T-1000.MOS:
00000: /* from game biffs */ ~XVARTS/SETUP-XVARTS.TP2~ 0 100 // Xvart Caverns: Beta2

It worked locally... course I have xvarts beta 2 installed instead of beta 1 that might be a factor....


*********************************
I do see some adjustments I could make... I'd copied the override method.... where it has checks etc to make sure it is a game file and not some marker file... since the inputs method makes sure the file already exists.. I can bypass that and go straight to the change-log

Edited by Sasha Al'Therin, 29 April 2012 - 04:31 PM.

My working mods:
an AI Party Script for BG2 game engine DOWNLOAD LINK ONLY!
Interactive Tweaks for BG series with some IWD support. DOWNLOAD LINK ONLY!
Rest For 8 Hours an IWD mod
-------------------------------------------
My contributions: BG1Fixpack, BG1Tweaks
On Hold: Solestia an NPC for SOA
-------------------------------------------
My website: http://sasha-altheri...s.com/index.htm

#10 Sasha Al'Therin

Sasha Al'Therin
  • Modder
  • 615 posts

Posted 29 April 2012 - 05:07 PM

even changing the code to call the change-log directly after typing in a valid game file results in ERROR output for too large files. However the tool recovers and goes on.
Spoiler
Going back to the available version of this tool... it also ERRORs on too large files but recovers and goes on. If you want weidu to change-log larger files then you'll have to ask the bigg to modify weidu. doubt he will do so tho.

To get the result you had... oh i see now, you went with all files in the override... I may consider dropping that part as it can only be run without biffing, etc... but i'm looking into a way to detect files that are too large

My working mods:
an AI Party Script for BG2 game engine DOWNLOAD LINK ONLY!
Interactive Tweaks for BG series with some IWD support. DOWNLOAD LINK ONLY!
Rest For 8 Hours an IWD mod
-------------------------------------------
My contributions: BG1Fixpack, BG1Tweaks
On Hold: Solestia an NPC for SOA
-------------------------------------------
My website: http://sasha-altheri...s.com/index.htm

#11 Sasha Al'Therin

Sasha Al'Therin
  • Modder
  • 615 posts

Posted 29 April 2012 - 06:24 PM

updated. see 1st post for changes and reason why

My working mods:
an AI Party Script for BG2 game engine DOWNLOAD LINK ONLY!
Interactive Tweaks for BG series with some IWD support. DOWNLOAD LINK ONLY!
Rest For 8 Hours an IWD mod
-------------------------------------------
My contributions: BG1Fixpack, BG1Tweaks
On Hold: Solestia an NPC for SOA
-------------------------------------------
My website: http://sasha-altheri...s.com/index.htm

#12 Sasha Al'Therin

Sasha Al'Therin
  • Modder
  • 615 posts

Posted 29 April 2012 - 11:45 PM

muwhahahaha finally figured it out.

--change-log itself on weidu's command line can handle large files
the COPY command in weidu's tp2 language cannot, in tp2 language COPY is the only way to get a file name into a variable. This is why the override as a whole option failed when a large file was reached.

In the one by one option, even tho the COPY command was used, the variable was preset so the command line action could still take place even when it said there was ERROR due to file size.

On the whole override option. With a little help from google, I have a series of inlined batch files that will get each file name and pass that on to the weidu --change-log line. Only drawbacks I see is that for such a large amount of files scanned by --change-log, the output copies of the changed files could possibly over stuff someone's drive :( I'll have to find a way to turn off the copying of the backup files in this particular scenario.

I've toyed with the idea of IF the Mod Tracker is present to modify or create the rule file for the backup directory location (its tedious doing it by hand, plus it craps out on certain characters). Cause for large masses of files, it does do a better & faster job. but right now such a thing is beyond my understanding.

still testing...

oh learned a previously unknown fact (to me at any rate)...
in tp2 language
FILE_EXISTS has no size limit
FILE_EXISTS_IN_GAME is restricted to the size limits impose upon COPY

My working mods:
an AI Party Script for BG2 game engine DOWNLOAD LINK ONLY!
Interactive Tweaks for BG series with some IWD support. DOWNLOAD LINK ONLY!
Rest For 8 Hours an IWD mod
-------------------------------------------
My contributions: BG1Fixpack, BG1Tweaks
On Hold: Solestia an NPC for SOA
-------------------------------------------
My website: http://sasha-altheri...s.com/index.htm

#13 Sasha Al'Therin

Sasha Al'Therin
  • Modder
  • 615 posts

Posted 30 April 2012 - 10:04 AM

Simpler is better... all issues solved.

mass change log of override contents can be done. It launches a batch file which gets the files in the override and processes it thru the Weidu change-log.

Individual files can be typed in and change-log made. Weidu change-log doesn't care whether the file entered exists or not. It compares the text against those listed as modified by the mods. If nothing is found an empty data set of "Mods affecting FILENAME" is appended to the output text.

While the output text may be 'bloated' due to files that had nothing done to them or due to user input of invalid file names, there will be no backup file output in such cases. Only time backup files will be created & numbered to match listed entries is when a file actually had some work done on it.

In other words, all the fancy checking and examining within tp2 to check whether entered text matched a file name was hurting more than helping.

Will update first post & the link once I get examples of output to demonstrate the results... (could be a while as testing install has TONS of content in the override)

My working mods:
an AI Party Script for BG2 game engine DOWNLOAD LINK ONLY!
Interactive Tweaks for BG series with some IWD support. DOWNLOAD LINK ONLY!
Rest For 8 Hours an IWD mod
-------------------------------------------
My contributions: BG1Fixpack, BG1Tweaks
On Hold: Solestia an NPC for SOA
-------------------------------------------
My website: http://sasha-altheri...s.com/index.htm

Back to Miscellaneous Released Mods


  1. Spellhold Studios
  2. → Modding
  3. → Miscellaneous Released Mods
  4. Guidelines