I want to edit a save, but I need it to not appear anywhere on the by date page so it doesn't look like I'm just shamelessly republishing old saves for upvotes.
There's no direct way to do this. All saves bump to the top of "By Date" unless you already bumped it within the last 3 hours.
One option is to resave it, but unpublished. Then after a day or so has passed, publish it without resaving (you can do this directly in the save browser). It will still show up in By Date, but it will be buried a few pages deep.
If it's not many saves I wouldn't worry though. Bumping is normal.