When a new season begins and I release a young player (no long term contract), do I still have to pay his salary?

Yes. Unless the player is a free agent you don't sign, or an arb-eligible player you release before arbitration. If there's a figure in the $ column of the Roster page, you are on the hook for it. Try trading him first. If that doesn't work, put him on waivers. If someone claims him, they take on his salary. If no one claims him, use him in the minors smewhere, since you're paying him anyway.

So players should be released at the end of the season. If not, I have them for the next season. Right?

Yes. Most of the time it makes more sense to keep the player around until he retires — you  have to pay someone to provide depth throughout the minors anyway, and don't want a useful prospect sitting on the bench instead of playing at a lower level. I don't recall ever releasing a player other than to avoid paying him through arb.

Releasing minor leaguers seldom makes sense.