My pick would have to be the A Whole Nother Story trilogy, in which (spoiler warning) you can only travel back in time. But because time is in a loop of sorts, if you go before the beginning of time, you will be at the end of time. From there you can go back to any time you want to. And time paradoxes cannot be produced. Plus, your memories from the previous timeline exist as well as the memories from the new one.
The End of Eternity by Asimov. Very unique take on the structure of time. If you like the setting for the Loki series it is basically the same thing but IMO way better done.