Toribash
They are saved in the autosave directory, not in the replay directory, so it doesn't mess up your replays.
Is it really a bother ?
Well each replay takes up on average 50kb, so after awhile yes. I could see the autosave feature becoming a bit of a bother.

Take Hamster for example; 13,000 total matches.
If all autosaves still there. (averaging on 50kb a match ( sometimes more sometimes less)
13,000 x 50 = 650,000 KB, which is 650 MB, which may not seem like alot. but that space can be easily missed in the future D:
Hmm.. yeah, we should have an menu option to review replays in the autosave directory and clean it up afterwards.