Hroldar the Strange (minor spoilers)

[Coot]

The only solution to the problem this Hroldar poses seems to be punching him (with your bare hands) until he leaves. He doesn't fight back at all.
I find this solution really distasteful. I feel like a guy working at McDonalds who's kicking a confused homeless guy again and again to get him away from the door.
Also, Hroldar even has a valid point about what Ebonheart Company is doing there.
Is there any other way to solve this? My personality and speechcraft are really high but that didnt have any effect. There are no other dialogue options.