imo stats are the biggest bonus to damage
weather its dex for bow dmg str for sword dmg ect
at some point it will cost you more to buy 10 str than it will to buy 1 level of boosters which is 10 str at that point if you are chasing str it is wiser to buy the booster to get the 10 str in the end tho it is only 10 str there is no invisible dmg mod
your weapon skills will also raise damage and at some point its likely that raising a weapon skill is cheaper than buying more str and in that you will get more out of spending points in the weapon but at the beginning id highly reccomend stats over skills at least up to 70