TAC compression via de-duplication

[Fishbiter]

I had a bit of spare time this evening, so I scratched an itch I've had for a while.

As you may realise, there's a ton of duplicated files inside the TAC files (because they contain the images of the separate disks from the original, which had to hold copies of the same data).

Since the TAD is just a map of hashes to offsets it occurred to me that you could just point the offset for a duplicate file to the original, saving a bit of space. I wrote this python file to do it, and it seems to work (nb. I've done very little testing!). It'll shave about 1.2gb off the install size if you run it for the disk_xxx tacs.

The command line is:
CompressTAC.py <source with no extension> <destination with no extension>

e.g.
CompressTAC.py E:\SteamLibrary\steamapps\common\SMLaunch\sm2\archives\dx11\data\disk_5b7bcbc7 disk_5b7bcbc7

It will not work if source and destination are the same...

Er... I doubt anyone really cares about saving a mere gig, but I just thought I'd put it out there...!

 

[ner0]

I haven't had the time to test this, but the idea itself is very good.
I wonder why D3T didn't consider doing this during production, it's probably so easy to streamline it into their build process... mainly considering consoles which tend to be a little more limited in volume size, it would be worth it. And are you kidding?! 1.2GB doesn't sound like much, but considering that the game (PC install) amounts to 15.1GB, then we're talking ~8% saved space from redundant data (unless I took a wrong turn somewhere).

I do care, applaud the ingenuity, and hope that D3T would take notes, as they should, despite the great work they're already doing. C'mon people, there's always room for improvement!

 

[ner0]

I was attempting to build a binary from your script, because first I got an issue because of only having Python 3.4. Apparently the file() function isn't present in Python3. Then I installed Python 2.7 and tried getting it to install a bunch of dependencies, one of them being PyInstaller, but they all failed. I couldn't install one single dependency through pip, and I had trouble accessing a lot of sites that hosted the resources. Long story short... my ISP, my router, IPv6 and Python is the formula for mayhem. Took me almost 2 hours to figure out what was wrong, or shall I say a workaround (disabling IPv6), to be able to use pip. Phew...

 

[Fishbiter]

Ha! Yeah, sorry I should have mentioned it was Python 2.7... I'm one of those people who never made the move to 3.x