Mate, using your logic “black” isn’t a race. This is also technically true, but not really how the term “race” is used. Taking your logic and applying it to this term that is widely accepted caloquially as a race, you end up being able to say that it’s “impossible to be racist against black people” which would be seen as a racist remark.