Because goodness requires effort to understand others.
And you can truly understand others only when you have truly understood yourself.
And understanding yourself means making deliberate, persistent effort to spend alone time with yourself, no matter how much the world tries to provoke into not doing so.
Thus, in order to truly love others you have to risk being quietly hated by them so that you could nurture your love for yourself, which you are eventually going to use to love them. Ah, the irony.
This requires an insane level of self-love and selflessness, something not everyone can afford.