wc3edit.net
https://forum.wc3edit.net/

Frost MPQ Alpha Release
https://forum.wc3edit.net/useful-tools-f69/frost-mpq-alpha-release-t37511.html		 Page 1 of 2
Author:  owner123 [ August 7th, 2020, 12:51 am ]
Post subject:  Frost MPQ Alpha Release
Frost

Frost is a multi-platform MPQ editing tool which can:

- Extract
- Import
- Fix MPQ corruption
- And more!

This tool is meant to mimic the implementation of Storm as much as possible. There is no such thing as a "read only" MPQ for Frost. This tool can bypass PG protection and Spazzler and import files to corrupted maps.

This tool is very much in an alpha state. It will be improved going forward, but it is usable right now. Please report all bugs through either GitHub issues or the release post. Feel free to make any suggestions or ask questions as well!

It is open source: https://github.com/zach-cloud/Frost , built and maintained by me.

Features

- MPQ Opening
- Add your own Listfile
- Extract files
- Extract all files
- Import files
- Rebuild archive

User Guide

You need Java installed to use this.

Double click run.bat (rename to run.sh for Linux based machines, such as Macs)

A command line interface will open. and prompt you to select a file. Write the name of the file you want to import. This file should be in the same location as the Frost jar file.

After the file opens up successfully, it will give you a menu of operations to choose. This is what each of the operations do:

1: Extract.

The extract action will prompt you for a file name to extract. This should be the exact file name as it appears in the MPQ Archive. It will tell you if it doesn't exist.

If the file exists, it will extract it to the "out" folder.

2: List

This will provide you a list of all known filenames in the archive.

It first attempts to use the internal listfile. If that listfile doesn't exist or isn't complete, then it will try to use an external listfile. If there is a file named "listfile.txt", it will use that. If there isn't, it will prompt you for a listfile to use. If you don't enter a listfile, it'll continue without it.

After finding all known files, it will give you a report with all file names.

3: ExtractAllKnown

This will extract all known files to the "out" directory.

It will not extract files that it does not know the name of.

It first attempts to use the internal listfile. If that listfile doesn't exist or isn't complete, then it will try to use an external listfile. If there is a file named "listfile.txt", it will use that. If there isn't, it will prompt you for a listfile to use. If you don't enter a listfile, it'll continue without it.

4: Import

Imports a file from disk into the archive. The file should exist in the same directory as the Frost jar, and should be the same filename as you want to put into the archive.

If the file already exists, it will delete it first.

If the hash table is filled up, it will refuse to add new files.

It adds the file as a single entity, uncompressed.

5: Delete

Deletes the file name from the archive.

6: Save

Saves the MPQ archive with modifications. It will prompt you for the name to save as (will not overwrite by default). Before saving, the tool will delete the (attributes) file

Note that the mpq WILL NOT BE SAVED if you don't use this command. If you import a file and don't press save after, it will not be imported!

7: Quit

Quits the program.

Limitations

Currently, the tool is limited in the following ways. I plan to improve it going forward.

- I do not support all compression types. It's possible the tool will find a compression type that it doesn't know how to decompress, in which case it will refuse to extract the file. These types were not found during my testing.
- All files are only added as an uncompressed, single entity. This means the map size will increase when you replace files.
- It's not the most user friendly tool yet. It is command-line only and lacks very good error reporting.
- The Extended Hash Table, Extended Block Table, Weak Signature, and Strong Signature are not supported. I have not seen any maps using these fields in my tests.

Attachment:
Frost0.1.zip


This is the PG-Protected map I tested with: https://drive.google.com/file/d/1E2JZL3 ... uG4RF/view
Author:  Vegas [ August 7th, 2020, 11:15 am ]
Post subject:  Re: Frost MPQ Alpha Release
Great work! I'll take a look this weekend.

After fixing corruption, would the map not be easier to deprotect?
Author:  nuzamacuxe [ August 7th, 2020, 9:01 pm ]
Post subject:  Re: Frost MPQ Alpha Release
I tried to open a pg protected map with your program but it just throws a lot of errors and then closes itself:
Spoiler:
Image


MAP:
fulfilled-requests-f75/naruto-rpg-0-49fix1-w3x-t36397.html#p201945
Author:  owner123 [ August 8th, 2020, 7:33 pm ]
Post subject:  Re: Frost MPQ Alpha Release
I will test it out - looks like there were multiple versions of PG

I'm not sure if tools like xdep will be able to deprotect it after saving. The best bet is probably still to rebuild the map. I may be interested in looking into deprotection in the future sometime, but it's something that is totally undocumented and not too useful anyways. We'll see.
Author:  Vegas [ August 8th, 2020, 8:50 pm ]
Post subject:  Re: Frost MPQ Alpha Release
There are a couple more deprotect tools that have worked out in the past. There are several requests for deprotected maps. Some ask for the Triggers which is not possible, but some want to be able to load, edit, and save in W.E. This can allow for novice wc3 people to edit maps, mostly abandoned many years ago, and make new updated versions without needing any coding skills. Editing models, upgrades, new abilities, as well as translating maps can be done extremely easy on a deprotected map.

Just think of the possibilities with your own website with just one tool to do it all.


http://www.owner123.edit.org

:D
Author:  owner123 [ August 8th, 2020, 11:00 pm ]
Post subject:  Re: Frost MPQ Alpha Release
Haha, I don't think I'd use owner123 :D I made this name when I was like.. 12 years old, when I joined this site. Just a random name.

I am a bit torn on this topic. On one hand, I think the most proper way to do things is to educate people on the proper way to edit protected WC3 maps. But on the other hand I can see how it may be too difficult for many users. I think I will at least look into deprotection after I get a decent version of my MPQ tool working. I feel like I can probably make something to do it, given enough time and research.

I did look into the PG protection and this map has a lot more corrupted stuff than the previous one I tested. A bunch of block/hash table entries that my tool interprets as negative values. They also added a ton of useless data into the map (for example, they added the entire WarCraft III executable to the map, presumably just to make it a huge archive), that stuff should be easy enough to remove. Looks like there are so many hash table entries that it actually overflow my program. There's also a lot of design stuff I would have done differently if I knew how building this tool was going to be. I'll try to get something published that works for all maps as soon as possible, though I don't know how long it'll take. So much stuff I want to make but so little time :(

First I'm going to make the tool trim the stupid data off of the file and then fix the invalid flags and such, then investigate some of the weird values I'm seeing. After I figure this stuff out, I plan on publishing a real MPQ file spec that details all of the ways a map can be corrupted.
Author:  Vegas [ August 9th, 2020, 3:04 am ]
Post subject:  Re: Frost MPQ Alpha Release
Did you click the link to your website?
Author:  nuzamacuxe [ August 10th, 2020, 5:44 am ]
Post subject:  Re: Frost MPQ Alpha Release
Vegas wrote:
http://www.owner123.edit.org

:D


:lol:


owner123 wrote:
Haha, I don't think I'd use owner123 :D I made this name when I was like.. 12 years old, when I joined this site. Just a random name.

I am a bit torn on this topic. On one hand, I think the most proper way to do things is to educate people on the proper way to edit protected WC3 maps. But on the other hand I can see how it may be too difficult for many users. I think I will at least look into deprotection after I get a decent version of my MPQ tool working. I feel like I can probably make something to do it, given enough time and research.

I did look into the PG protection and this map has a lot more corrupted stuff than the previous one I tested. A bunch of block/hash table entries that my tool interprets as negative values. They also added a ton of useless data into the map (for example, they added the entire WarCraft III executable to the map, presumably just to make it a huge archive), that stuff should be easy enough to remove. Looks like there are so many hash table entries that it actually overflow my program. There's also a lot of design stuff I would have done differently if I knew how building this tool was going to be. I'll try to get something published that works for all maps as soon as possible, though I don't know how long it'll take. So much stuff I want to make but so little time :(

First I'm going to make the tool trim the stupid data off of the file and then fix the invalid flags and such, then investigate some of the weird values I'm seeing. After I figure this stuff out, I plan on publishing a real MPQ file spec that details all of the ways a map can be corrupted.


I see. Take as much time as you need, don't need to rush.
Author:  owner123 [ August 12th, 2020, 10:20 pm ]
Post subject:  Re: Frost MPQ Alpha Release
No of course I didn't click it :) That website won't exist yet anyways, I don't need to click it!

I found some interesting stuff. The header of the file says "PG2", which makes me think that the version I experimented with was PG v1 instead.

It added a bunch of garbage entries to the block table, but it's relatively easy to find the actual entries. However the entries specify that they are compressed, but block size is -1 which is a violation of the MPQ format. Very interesting that these get accepted anyways. I think it assumes that it's not actually compressed. Or that compressed size equals regular file size.

It makes the hash table massive and adds garbage entries there as well. Each file also has 4 collisions. The collisions are no big deal, but the strange part is that none of the hash table entries actually appear valid. I hashed a known file name, and no entries contain that hash. That's where I'm stuck right now; it's actually very curious. I think there's some key concept that I am missing and once it clicks, I can break it. If I can't figure it out myself, I can always look at Stormlib which is open source! There are some other things I want to do in the meantime though.

I believe the root cause is an invalid hash/block table offset. They are both set to 0, which doesn't make much sense. Perhaps the table itself is compressed.
Author:  Bertix [ November 15th, 2020, 7:17 am ]
Post subject:  Re: Frost MPQ Alpha Release
Will be here another update for Frost MPQ? I think i find map with PG 3 and i cant find war3map.j with it also crashes on maps with Somj
Map name DisRPG 2.46
Map link https://mega.nz/#F!uk0wyLqa!9i9WZ4gjBB-htPoZ5kgLbw
Page 1 of 2 All times are UTC
Powered by phpBB® Forum Software © phpBB Group
http://www.phpbb.com/