So our final list for our little experiment is comprised of the following 15 units:
- Skullcrushers of Khorne [Ensorcelled weapons]
- K'daai Destroyer
- White Lions of Chrace [Banner of the World Dragon]
- Chaos Warriors [MoN, Halberds]
- Arachnarok Spider
- Demigryph Knights
- Mournfang Cavalry [GW, Heavy Armour]
- Black Guard of Naggarond
- Witch Elves
- Har Ganeth Executioners
- Hell Pit Abomination
- Beasts of Nurgle
- Savage Orc Big'Uns [AHW]
- Chaos Trolls
- Soul Grinder [MoN, Daemonebone Claw]
I have finally settled on format for our analysis. Why not let the best duke it out for themselves. Each unit will be pitted against every other unit on the list. The results will tabulated in the following table:
View attachment 80294
View attachment 80295
Procedural Rules:
- Combats will be treated as ongoing. So no charge bonuses for either side.
- Fear tests will be assumed as being passed
- Unit sizes will be adjusted to create the fairest possible balance in terms of points (obviously, single entity monster vs. monster matchups will be uneven)
- Each unit will be stuck using their overall best loadout, even if it is sub-optimal in a particular matchup. For example, Ensorcelled weapons are the best option for Skullcrushers in most situations, but not against the Banner of the World Dragon due to ensorcelled weapons being magical. While Skullcrushers would be better off not upgrading to Ensorcelled Weapons in this matchup, they will be forced to do so. This is is partially to avoid problems of counter picking, and the resultant doubling of the number of match calculations I would have to make.
- Unit formations will be optimized within the normal/typical range we find them on the battlefield. No unit will be narrower than 5 models wide and no unit will be wider than 10 models wide (horde).
- Partial wounds (for monster models... and maybe for monstrous models as well... haven't decided) will be carried forward from one round of combat to another (i.e. a Hell Pit Abomination could be sitting on 4.2 wounds going into a subsequent round of combat), for other models, 0.4 wounds will be rounded down to 0 = death.
- For the purpose of calculating combat results, partial wounds of 0.4 or lower will be rounded down and 0.5 or higher will be rounded up.
Questions remaining... PLEASE PROVIDE FEEDBACK!
How should break tests be handled?
- ignored? We go until a unit is completely wiped out! And if so, how is this counter-balanced with crumbling?
- assumed to be passed at 50% or greater probability of success and assumed to be failed at lower than 50% probability of success? And if so, do we assume the breaking unit has lost.. i.e. wiped out (even though they might not be caught in pursuit)?
- some other idea or method?
How do we balance points cost?
- in the case of a monster vs. multi-model unit it is straight forward, the unit size of the multi-model unit must be adjusted to meet the monster's points cost
- in a monster vs. monster matchup, one side will have a slight advantage. Can't do much about it.
- the problem arises with multi-model unit vs. multi-model unit matchup. There might exist different optimal unit sizes. For example trolls work very well in a horde formation, but Demigryph knights do not.
Anything else?
- any other ideas, potential pitfalls, concerns, etc.
Once we settle on all the rules/procedures, I can begin with the matchup analysis. Obviously, they will be completed one or a few at a time and I will post the results and updated table for discussion. To keep things consistent, I'll do all the matchups myself, but I'll rely on you fine folks to check things over incase I bungle anything up!
By the end of analysis we'll have:
- completed colour-coded table of results
- a Win-Loss record for each unit: (M. Win - Win - Draw - Loss - M. Loss)
- I will calculate a single numerical score for each unit by adding their results together [massive win = 2, win =1, draw =0, loss = -1, massive loss = -2)
- using the numerical result from the step above, we'll be able to order the list from the best (of the best) to the worst (of the best)

Thoughts?
First off, i would like to say that i like how much effort you have put into this mini project, i really like the idea! I think it is very fair that you let every unit fight each other unit, in that way outliers (like the White Lions with the Banner of the World Dragon heavily countering Skullcrushers) do not matter that much.
On the issues you still encounter, I think to recreate situations that most commonly arise upon the tabletop, i think you want to limit the amount of turns a combat goes on to around 3. Combats that go on longer than that are rare, sides often rout before that, or other units join the fray to heavily influence the outcome.
Considering point costs, i would define normalized unit sizes for each of the unit competing in this competition. Then i would define a point value (this number does not actually matter that much), to weigh the results to. You can then compare the weighed wounds they did to each other, and then give that a point value to use in the table. As an example:
Unit X is usually taken in a unit of 20, this unit costs 250 points as a unit of 20.
Unit Y is usually taken in a unit of 6, this unit costs 400 points as a unit of 6.
Let's say we weigh to 500. Unit X does on average 10 wounds over 3 turns. Unit Y does 15 wounds over 3 turns. From your initial assessment, you could conclude that Unit Y won, but that is unfair, seeing as they cost more than Unit X. When weighing the wounds done, you would 500/250=2, 2*10= 20 weighed wounds for Unit X. Unit Y gets 500/400=1.25. 1.25*10=18,75 weighed wounds for Unit Y, meaning that considering point costs, Unit X has won this head to head with a 1.25 wound difference.
An even better way to calculate this difference would not to pit these units against each other, but against the field. To do this, you could create a number of categories in which the units could compete. These could be, for example:
- Light infantry (Imperial Halberdiers)
- Heavy infantry (Chaos Warriors)
- Light cavalry (Dark Riders)
- Heavy cavalry (Imperial Knights)
- Monsters (Stegadon)
You could check how many weighed wounds you would do for each unit against every category, you could even check how much damage you got back, and calculate a score based on that, using the method i described before.
To cut a long story short, you have 2 main choices:
1. Calculate how strong the units are against each other.
2. Calculate how strong the units are against the field.
When doing this, you would have to just set a few parameters, like the amount of turns you are fighting and how many points each unit is, but as long as you perform enough tests this should normalize and have no real impact on the results. Would be interested to hear what you would like to test, I think both are interesting options.