First, go to the album and tap the 3-dot icon on the top-right. This will open the album's options menu. Tap Change album cover:
Now it’s time to select the photo that will replace the album's cover image. Tap the photo you’d like to set as the cover for that album. The change will be immediate, and you’ll be able to see the new cover appear in the album screen and My albums.
And that's all there is to it!