Do you think verbal racism is a petty & childish thing to get upset over?

As long as nobodies rights are violated & as long as someone doesn't amass any type of power, influence or following because of their message. Then shouldn't they just be ignored? Inherently no different than calling someone an azzhole?