Initial file specs

[IBBoard]

As promised, I've thrown together a quick package of the current DTDs for the data files. They're still incomplete (for example they don't have any way of recording "restrictions" beyond the core numeric limits) and the army one hasn't been tested properly (because I've not got to saving data out), but that's what I'm working from for now. As mentioned in the other post, they will be replaced with schemas at some point as they're more powerful, flexible and expressive, but I'm more familiar with DTDs so they were the first pass.

There are three files, one for each data file type.

• "Army" is for a saved army file and specifies units selected, unit size and equipment taken.
• "Race" is for army book/codex/[insert name of army list for other game] data and specifies unit types, available equipment, restrictions, etc.
• "System" is the definition of a wargaming system, e.g. Warhammer, 40K, Lord of the Rings, Babylon 5, and any of the others that I can't think of at the moment. It's the smallest of the files, but specifies the stats amongst other things (because not all games use the old WH 4th Edition list that Rollcall constrains you to)

I'm also hoping to get the app compatible with things like Gorkamorka and Necromunda, so we'll need to expand it to cover skills and skill points as well.

Initial DTD download

[Brother Tober]

Do you have some kind of data model where these DTDs are derived from?
If not, I would strongly suggest to construct one. This would make discussions and changes much easier.

[IBBoard]

I've got a scribbled diagram of what has what (e.g. each race has many unit types and each unit type is from one army) but I've not got anything digital. The files themselves were the basis for the data model - there's more in the DTDs then there necessarily is coded up in the classes.

[Brother Tober]

Maybe you could scan that paper? I could draw a digital one out of it.

[IBBoard]

I'll try to make a digital copy tonight, but it's my night to look after my son, so if he wakes up from his nap then I might not get much chance! I'll have to double check it as well as it was my original plan (after several re-drafts) that tried to be generic enough for most game systems, but the DTDs and coding might have strayed a bit as I found things they couldn't do that I hadn't thought of.


[edit] Nope, half hour nap for the kid, no time to make the document. I'll re-check it on paper later and then make a digital version tomorrow. It's vaguely UML or relational database design like but not strictly anything in the sketch.

[IBBoard]

Okay, here's the initial diagram that's similar to what I had sketched with some minor changes. It was made using Dia (because it's the best diagramming tool I know of in Linux and there is a Windows version available). The source .dia file is available if you want it.



I've tried to annotate the lines so that they make sense. There are probably still some entities missing from our earlier discussion. Now that I've uploaded it I've also noticed that there are a few links missing (like Unit Type -> Unit Type for "champion type" and Unit -> Unit for "champion", but I had thought that could be made more generic).

Game System, Race and Army are the main files, with the others being elements within the files. Units may be exportable to their own file (as Rollcall has) but that would just be for import rather than WarFoundry opening them directly.

To explain a couple of "odd" bits:

• both Game System and Race define Categories because a Game System will define the defaults but some races (e.g. Chaos in Warhammer 5th, I think) then define their own categories.
• An army contains army categories which map to a single category to handle containership - an Army Category is an instantiation of the abstract concept of a category in the same way that a Unit is an instantiation of a Unit Type.
• Abilities are a way to record special rules. We might end up needing something similar with its own 'points value' system to handle skills purchased with experience (e.g. campaigns or games like Gorkamorka and Mordheim)
• I've forgotten to include stats in there - they'll be in tomorrow (if I get a chance) and are defined by the game system (multiple ones, referenced by the Unit Type, to allow for things like Warhammer 40,000 where infantry has one stat line, vehicles another and walkers a third). That'll make for a messy diagram!

If any of the pluralities look wrong then give me a shout and I'll fix it. Hopefully that explains the core of the data model that needs expanding on.

[Brother Tober]

Thanks for providing the diagram.
Here are a few questions that came up at the first glance:

1. Why does a Game System define a default Race?
2. Could you please explain the relationships between Category, Army Category and Army in more detail?
3. Why are there relationships between Ability and Race as well as between Ability and Unit Type? If a Race defines an Unit Type, then the first relationship seems to be somehow redundant.
4. Could you please explain the concept of Requirements in more detail?

Cheers
Brother Tober

[IBBoard]

1) Game System defines a default race as a way of handling things like Warhammer Siege. The idea is that a Race will define a number of other Races that are its allies, which it can then take units from. The default race was then a way of having a central point for system-wide things like siege towers without making each race include them. There may be some unworkable constraints in terms of wanting to have system-wide equipment or abilities, in which case it might need re-working.

2) A Category is the classification (or classifications in the case of the latest Warhammer) of a Unit Type (e.g. HQ, Elite, Troops, etc for 40K and Lord, Hero, etc for WH). The Army then has one Army Category per Category defined in the Game System (or in the Race, if it has an override). If a Unit is created whose Unit Type is from the "HQ" category then they aren't added directly to the Army but are added to the "HQ" Army Category.

It is possibly more implementation level than design, but the way it works is that it keeps all Units from a Category together in one collection that can also provide additional information (such as "percentage of allowable points used" or "number of slots used"). By making it an object that is distinct from Army and Category it stops the Army trying to do all of the work, simplifies some of the modelling (because you can throw a collection of Army Category objects at a tree to show the unit selection, for example) and keeps it a bit more "one blob, one job".

3) The difference is that the Race defines the Ability (e.g. "They Shall Know No Fear") and then the Unit Type references it to say "I have this ability". The idea is to save duplication, which can occur quite a lot in Rollcall files and some Codex books.

4) Requirement is potentially extensible, but covers what we were discussing in the Unit and equipment requirements thread. It can be "must have at least one of X to take any number of this", "must be army general", "can't be taken on own" (i.e. "is a champion"), or any number of other conditions and constraints.

Hopefully that clears things up a bit. I'll try to expand on the unit stats tonight, but the in-laws are visiting so it might have to wait until Saturday.

[Brother Tober]

Thanks. That made things clear.

[IBBoard]

Okay, it's now been through an update to include stats and unit containership (a more flexible "champions" system). I've gone for a more UML-like format and differentiated between between optional and required links.

The updated diagram has got wider, so it's now got to be linked:


Hopefully it all still makes sense.

[IBBoard]

I've been thinking about the file spec recently as I'm getting on to some of the more specialised and less core areas of the tooling.

Requirements need to be very general so that we can cover just about anything (e.g. a Lord of the Rings plugin that adds the option for "no more than 50% of troops with missile weapons" requirement). That means we'll have requirements at the Race level, and I think what were classically "unit requirements" should now be moved up to the Race level. Unit Types can then have their own requirements that are specific to a single Unit (e.g. numbers of types of troops or forcing one of a collection of champion types or non-global weapon limits). The move would also make it easier to enforce selection structures (e.g. W40K force organisation charts) through a plugin that adds requirements on the number of units of a given category.

Hopefully that makes sense. I'm not sure what anyone else thinks of the idea.

[IBBoard]

The more I'm thinking about how to handle requirements, the more I'm wondering whether having Min/Max Choices and Base/Increment values and amounts in the data of categories is the best way to go.

Percentages are the only universal measure of army composition, even if it's "between 0% and 100%" for each category. Choices (40K 3rd/4th Edition style) and Base/Increment (Warhammer 7th Edition style) are quite specific and you're not likely to use both at once, which leaves you with a spare set of attributes. On the other hand a plug-in could add additional requirements that implement the restrictions without storing it in the data.

I think the plug-in method is cleaner and better structured, so unless anyone can think why it isn't a good idea then I'll remove it when I move the unit requirements.

[IBBoard]

The schemas are in progress now (ticket 32 is for the migration and revision 50 added the first schema) and the Systems schema has dropped the 40K and Warhammer specific attributes.

The one thing I'm thinking now is that both of the game systems have special conditions beyond just "unit counts for X in this category". While I could make it part of the category list (UnitType has a "main category" and a list of all categories, which was originally for the WH-style "takes a Lord and two Hero slot" units), I'm inclined to drop that along with the simplification of the categories in the Game System. We'd still need to store the important data somewhere, and it'd have to be extensible, but the reading of units from data will be fixed in code.

What I'm thinking of at the moment but need to refine is an additional part in the Race file that holds "extra data". That data is there purely as an extension mechanism for plugins, but one that is held in a simple associative array/get by key mechanism. That would then let plugins get the data when they needed it, let anyone add what they wanted in there, and let it be just about any format, all without the file loading ever having to know what to do with it. First, though, I've got to work out how to sensibly name it...

[IBBoard]

The schemas have now been completed and are visible online at http://dev.ibboard.co.uk/projects/warfo ... trunk/dtds. There are still a few bits I want to clear up (including some places where I was trying to use a custom restriction of a data type but the schema validation failed, so I've not been able to constrain values as much as they should be) but overall the schemas should hopefully be about done now.

[IBBoard]

Okay, the schemas are now up and working. I've fixed the problems with the custom restrictions and we now have schemas that should properly validate the values in a file to make sure they're legal (although nothing stops them being stupid!)