Army Builder file format support

[IBBoard]

IIRC I've been waiting for about six years and counting on a response from GW Legal I got the auto-responder saying they had a backlog, but no reply. In the end I sent another email saying "I haven't had a response for a year, this is what I think is legal and what I will do, if you still have a problem with this then please contact me. If you don't contact me I'll assume I have now satisfied your requirements." I never heard from them and so I'm assuming they're happy with our Rollcall files

AB3 automatically unpacks .ab files, so if you already have the files in use with Army Builder then you will just be able to copy the data files. The ideal would be to extract the files from the .ab files because that means we don't force people to install/use the Army Builder app first, but it does rely on someone cracking the compression format

I've taken a bit of a break and done other things for the past week (because I wasn't getting anywhere and thought a fresh look at it might help things) but I'll try to start a proper "Army Builder plugin" tomorrow that just loads the data files for now.

[IBBoard]

I've just had a quick look at the AB2 Construction Kit and its readme. The parts of the readme I saw with data in them implied something like an INI file for data format, but the Construction Kit interface is, to some degree, awful. There are settings and values in the readme that I don't know where they go, like version numbers!

Hopefully I'll be able to work something out about the Army Builder 2 file formats as well.

[IBBoard]

The file formats don't seem to have been as bad as I thought they were, but we do seem to be screwed on the "extracting files for you" front.

I gave up on getting a response by email and posted on their forums. As I posted on the wiki a few days ago, the official response basically says "Standard compression - what's that? We went for something slightly proprietary instead".

Now to see what I can do with the raw, pre-extracted files (which you can potentially convince the authors to release in a separate Zip anyway).

[IBBoard]

Odd. I don't know what I've done, but my install of Army Builder 2 won't recognise the files from that Yahoo group! Then I try it with a simple little test file I made and it recognises it, but crashes with "out of memory" type errors when I try to create a new army! Looks like I'll need to try and fix it so I can get at the actual data files.

[snowblizz]

Weird indeed.

One thing to note however. The "current" AB2.x is 2.2c, earlier datafiles apparently do not work with that or some such (or rather the other way around I'd say). So the files from anarchistica requires the use of the AB 2.2c update. But that should be available from the list... if not I should have it.

[IBBoard]

I think I had the app from the group. The one problem I did find was that I couldn't check, though, because Army Builder has a stupid design where it forces you to start an army or import, meaning it was crashing out on me before I could do a Help > About!

It sounds like shoddy programming if they've broken things that majorly between minor or even micro updates. I can understand that 3.x files wouldn't work in 2.x, but if the files from 2.2 can't gracefully degrade to be used in 2.1 then you've a) made a mess up at the start and b) potentially done some somewhat larger changes!

Keep an eye out for comments from darpified, as he has said he is looking at the compression format

[IBBoard]

It looks like I annoyed the developers slightly by saying they'd picked the wrong compression (which, IMO, they have if their documentation says it is PKZip compatible but the data is in a proprietary format), but one of their comments about being "the only compression engine that allowed us to properly identify the archive as being an AB import file" does sound like confirmation that they've used Greenleaf's custom compression.

Lets hope that darpified can make some progress on decompressing the data. After all, decompression is the least secret part of compression - decompressing DDS image files is easy, but getting good quality compression when making the files is where the real challenge is.

On the plus side, I've just tried WarFoundry again and this time it wants to import the WH data file, so at least I've got the raw data files for it now and can tinker

[snowblizz]

Yeah the 2.2c seems to be a "major" release despite the minor version change. I know for sure that only the 2.2c can run 2.2c ab-files.

Oh yeah, THAT has annoyed me for years. It exits as soon as there's any error. At one point I was playing around with the datafiles updating some armies after the "official" updates had stopped and if I made any mistake I'd get an error on load and then the whole thing jsut crashed.

You always have to be able to load a gamesystem to do anything. Sometimes it defaulted to the Tutorial game system or if you had loaded datafiles it would pick the first army from the gamesystem.

Definitely rooting for Darpified. Hope you can find a way to "hack" the datafiles.

Edit: read the thread you linked. You definitely wounded someone's pride there. Programmer cat-fight, not quite as interesting as the more common version but entertaining nonetheless.

[IBBoard]

Greenleaf Software do give some minor hints about their compression, so hopefully if darpified knows a bit about pulling apart compression formats he'll be able to get somewhere If not then we'll have to fall back to "you have to have them pre-extracted" and talk to some of the file developers about releasing the source and compiled versions at the same time.

It does seem like the dev has a case of "wounded pride" there, but as I tried to point out in my last post I did have all the facts and there was a logical basis for my statements, not just "rash assumptions".

[darpified]

It appears that Greenleaf's proprietary compression format is a slightly modified
LZW, same as zlib.

Modified just enough to make it impossible to use stock zlib routines.

I'm slowly picking out the details. Much slower than I expected, going to
try and peck away at it this weekend, will keep everyone posted.

rp.

[IBBoard]

So as the dev said, it's a custom LZW format. At least if it is based on a fairly standard compression it'll hopefully be a bit easier to work out than a completely unknown compression. I've put up a simple little compression wiki page for you to record any useful details on.

Good luck and happy hunting

As a side note, someone in that other thread has tried to say my logic was wrong because I assumed the documentation was telling the truth! Yes, you can't assume it is PKZip compatible if you are told it isn't, but if the document says "these files will use compression that is PKZip compatible" then it is perfectly reasonable to assume they're PKZip compatible!

[darpified]

ok, many dead ends and false starts later.

I'm going to chip away at it from the other side and see
if that helps to get it along.

will keep updating...

RP.

[IBBoard]

The other side? Not quite sure what that would be, but hopefully it works out Proprietary formats sure are a pain in the arse.

[stfas]

i think he meant the "other side" in a pictural sense and not in a literal sense...

[darpified]

Heh, actually I meant placing different text as placeholders in odd positions in the to be encrypted side and checked out what comes out the other end in the file format.


Just another way of attacking the problem.

Plus gives me a chance to ease the frustration level.

RP.