back down

verb

1. move backwards from a certain position"The bully had to back down"
2. remove oneself from an obligation"He bowed out when he heard how much work was involved"