Army Builder file format support

[IBBoard]

Hopefully it works out. I'm not sure how much freedom you get with ArmyBuilder files as to how much you can insert diagnostic-type files (specially constructed text with minor variations to identify what it does in certain situations). I guess the AB3 XML files might be useful there as you can use <!-- XML comments --> and not affect the content in a way that'll upset the AB app.

[Edit] darpified - the thread on the Army Builder site has unearthed something useful (at last!). According to this post, the files are similar to .ARJ files. I don't know if that helps at all

[darpified]

I believe that someone is talking out of their ass.

"http://datacompression.info/ArchiveFormats/arj.txt"

The format doesn't bear any resemblance to ARJ, it was one of the
formats that I first considered, given the timeframe that Lone Wolf
had specified.

Anyways, I'm about to sink a few hours into it right now. Will hopefully
make more progress.

RP.

[IBBoard]

I take it the files don't use something approximating ARJ's compression algorithm either then, even if they do potentially us a different wrapper? That page seems to cover the headers but not the compression. I'm just doing a bit of hunting at the moment to see if I can find any hints via the names of the creators of the Archivelib library.

[61bfe0ab023c143d]

I believe that someone is talking out of their ass.

Your lack of faith disturbs me.

1) Download the dearj source code.
2) Extract the bit that actually does the decompression.
3) Run it on the compressed chunks from the ab file.
4) Thank your lucky stars that the coders at Lone Wolf aren't the sharpest tools in the box and didn't put more effort into obfuscating their data.

Or, if you want to use the code I have written, email me. It's not particularly pretty, but it works.

$ tar -ztf abunpack.tar.gz
mini/
mini/arjdecomp.h
mini/abunpack.cc
mini/read_endian.h
mini/Makefile
mini/abformat.h
$ ls -al abunpack.tar.gz
-rw-r--r-- 1 user user 9115 2010-04-28 19:37 abunpack.tar.gz

Code: Select all

// read a compressed .ab file

#include <fstream>
#include <iostream>
#include <zlib.h>
#include "abformat.h"

int main(int argc, char *argv[])
{
   for(int fiter = 1; fiter < argc; fiter++)
   {
      std::cerr << "Unpacking  " << argv[fiter] << "\n";
      
      abfile fin(argv[fiter]);

      std::string dest = "poop";
      if (fin.details.size() >= 4)
         dest = fin.details[3];
   
      std::cerr << "Writing to directory " << dest << "\n";
      system(("mkdir " + dest + " 2>&1").c_str());
      dest += "/";

      for(size_t i = 0; i < fin.size(); i++)
      {
         const file_header &fh = fin[i];

         std::cerr << "Name=\"" << fh.name << "\" ";
         std::cerr << "Time=" << ctime((time_t *)&fh.date);
//         std::cerr << "Position of file is pos=0x" << std::hex << fh.pos << " size=0x" << fh.size <<
//            " " << "original size=0x" << fh.orig_size << "\n";
   //      std::cerr << "stored crc = 0x" << std::hex << std::setw(8) << std::setfill('0') << fh.crc << "\n";
         
   //      std::cerr << "unknowns " << std::hex << " 0x" << fh.unk0 <<
   //      " 0x" << fh.unk1 << " 0x" <<  int(fh.unk2) << " 0x" << int(fh.unk3) << " 0x" << fh.unk4 <<
   //      " 0x" << fh.unk5 << " 0x"   << fh.unk6 << "\n";
   //      lpos = fh.pos + fh.size;

         unsigned char *data = fin.getFile(fh);
      
         std::string fn = dest + std::string(fh.name);
         
         bool present = false;
         {
            std::ifstream fin(fn.c_str());
            if (fin) present = true;
         }
         
//         if (present) std::cerr << "Over writing " << fn << "\n";
         
         std::ofstream fout(fn.c_str());
         fout.write((char *)data, fh.orig_size);
         
         delete [] data;
      }
   }
}


[IBBoard]

So it is the ARJ compression, rather than the ARJ format. Thanks for the heads up. Based on other info I've found, I guess it is just the same (or a sufficiently compatible) algorithm behind ARJ and the GreenLeaf ArchiveLib library that they used.

Thanks for the pointer. Now to check licenses and see what we can do

[61bfe0ab023c143d]

Or, if you want to use the code I have written, email me. It's not particularly pretty, but it works.

A few people have taken me up on this offer but, after further consideration, I've decided not to distribute my code.

I am disgusted by Lone Wolf's recent actions over trademarking and have decided that I don't want to do anything that might help promote, however inadvertently, the sale of software created by such a XXXXX bunch of XXXXX

For a full exposition of what I'm talking about read this nonsense http://forums.wolflair.com/showthread.php?t=9753

If the staggering XXXXX of lone wolf's spokesman doesn't make you sick, then you are a better man than I.

To reiterate, what I did wasn't hard and, given what else has been mentioned on this thread, it wouldn't be hard to reproduce it.

(Self censored to try and protect myself from Lone Wolf's over excitable and XXXX legal department)

[IBBoard]

Fair enough, but the way that I see it is that if we can support the file format of a proprietary (and, TBH, hideously ugly and inflexible) app then we can move people away from it and lose them customers. It's the same thing that apps like OpenOffice do - take the dominant standard and support it, but also provide their own, better standard.

Thanks for the original offer, though. I'll have to look into the open source implementation and port the bits we need to C#. Hopefully you can stick around and help out in other ways

[snowblizz]

I have to agree here. If you really want to screw 'em over, and I must say sympathize with that idea after their draconian exploits, then enabling users of the-program-that-shall-not-be-named to easily migrate to an alternative will do them more harm.

It should be important to note that the datafiles are produced by and for the users, in a sense (legally speaking completely inaccurate) they are "our" property, being able to use these in an OSS program strengthens the users' position considerably.

So if you can make sense of the datafiles I would very much appreciate being able to make use of the datafiles from that-other-program. I just don't have the time and availability to produce datafiles for all games and systems.

[IBBoard]

I think the "other app" intentionally uses the "made by users" cover as a legal defence. I've had GW legal give me a warning over the old Rollcall data files*, so the fact that they've had the links up for so long must be in part due to the fact that they're only responsible for the app and not for the data. If we can get co-operation from some of the groups like AB40K to make use of their data in our app then all the better

I promise I will look at the data files, including extracting them from the .ab archives, but my first priority is getting the v0.1 final releases for Linux and Windows


* a much politer one than the recent and potentially baseless trademark threat, it must be said - they basically said "would you mind not doing that, as we feel it devalues our codex books" rather than "stop it within 72 hours or we're setting throwing our big bad group of lawyers at you"

[karlthepagan]

I recently got into the hobby. I am a java / flash(as3) dev. I'll probably be able to replicate the arj findings when I have time. I may or may not have time to finish an army list builder product based on my skillset but I will make my code GPL and put it on Google code.

Now as a software dev who earns a living from my company's IP and as a user of GPL greyware software like LAME or free codec packs I was instantly wary of the $40 product which uses ab40k files. And then doubly wary of paying $40 for greyware as I learned about the history of that product.

No doubt my apathy will overcome my outrage and I'll buy it some day.

Also isn't ab40k.org infringing on LW's trademark?

[IBBoard]

It'd be great if you could have a look at some of the C# code and see what you can contribute. My day job is in Java and I do C# in the evenings, and I don't notice too big a difference. Once you start to use a couple of languages then picking up some more isn't normally that difficult

Following 61bfe0ab023c143d's comments, I found the official version and a FOSS rebuild of some deARJ apps. I emailed the official version about making a port and never got a response (their license was unclear on whether I could port it or just use the code in my app) but there obviously won't be that issue with the FOSS version.

as a user of GPL greyware software like LAME or free codec pack

I think they're only "greyware" in the US. AFAIK things like LAME normally do patented stuff, but Europe doesn't officially support software patents the last I read (although some companies were trying, of course).

No doubt my apathy will overcome my outrage and I'll buy it some day.

If it's a "one day" thing then wait for WarFoundry to be finished. It'll be a lot cheaper on your pocket and leave you with a warm fuzzy feeling from the FOSS

I was instantly wary of the $40 product which uses ab40k files

Yeah, given their reliance on the IP of others then it is a bit of a dubious situation, you'd have thought, but they seem to have made it last this far.

Also isn't ab40k.org infringing on LW's trademark?

Yep, in the same way that GW won't let you use their names in your domains (so "Warhammer Archive" wouldn't be allowed). However, since LW can't make money without data files, and since making those data files directly is even more legally dubious for them, and since AB40K covers one of the larger markets then I guess they let them be.

[IBBoard]

Just for reference, the OSS version of ARJ that I found is here. Hopefully it includes what we need, since it is GPLed code and so the license is fine if we want to port it to .Net

[karlthepagan]

It'd be great if you could have a look at some of the C# code and see what you can contribute. My day job is in Java and I do C# in the evenings, and I don't notice too big a difference.

Yeah, that wouldn't be a problem - I'll pitch in where I can but by all means do not rely on me as a regular contributor. It's going to be a busy year for me.

I think they're only "greyware" in the US. AFAIK things like LAME normally do patented stuff, but Europe doesn't officially support software patents the last I read (although some companies were trying, of course).

Yes (and I'm a Yankee), but the same principle essentially - in the realm of commercial software patents, copyright, trademark are all despicable tools used solely to regulate the price of software. That said FOSS / contracted software dev as labor is the only real way out (and the way of the future IMO). Compared to other indie software devs out there LW is really fleecing their users and especially so when they both enforce and violate IP laws to make their product work.

since LW can't make money without data files, and since making those data files directly is even more legally dubious for them, and since AB40K covers one of the larger markets then I guess they let them be.

It is grounds to invalidate the trademark if they do not enforce it. At the least it is grounds that anyone can use the "AB" acronym.

[snowblizz]

It is grounds to invalidate the trademark if they do not enforce it. At the least it is grounds that anyone can use the "AB" acronym.

In Swedish "AB" is short for "corporation", good luck to LW to try and force 2 countries to changed that...

[karlthepagan]

The use of "AB" in this context refers to "Army Builder" - that is the context in which it would be interpreted by the courts.

Of course I meant anyone could use "AB" to indicate an "Army Builder like maker" product or "Army Builder" compatible product.

</pedant>