If you fear that defending your boundaries is being controlling, don't worry. There is a difference between being controlling and having boundaries. Controlling people is about telling them what to do. Setting boundaries is about saying what you do or do not want to happen to you.