Salamanders, Salmanders and more Salamanders!!! They will do massive amounts of damage against a unit of Sword Masters. Toughness 3, poor armour save, small bases (and he even conveniently fields them in a target friendly hoard formation). Salamanders for the win.
Dwellers Below will also work very nicely. It is the bane of Elves. Remember that Dwellers allows no saves of any kind, so the Banner of the World Dragon is useless against it.
Alternatively, you can probably hold the unit up with a defensively built stubborn Oldblood cowboy (just make sure you don't equip him with a magic weapon). This works best if backed up by a life Slann to heal the odd wound that may slip through (though that combo is only feasible in larger games).