fall by the wayside

drop out give up drop by the wayside throw in throw in the towel quit chuck up the sponge