Think Back to Look Ahead