Hmm.. not too sure on that, probably if you want to be more spell orientated than it should be more, or else you could use the pts instead for melee skills.
For me my favor was 100+ and fortitude was bout 80+
Haha for me because i didn't really have other skills to spend on besides two-handed, evasion and fortitude... i didn't mind using 3 pts. Unless of course you find there are other skills u intend to work on, maybe its wiser to use for those instead.