Help me with this Riddle... I can't figure it out.

There's 5 people hung on crosses behind one another, 2 of those crosses are painted blue on the back side, 3 of them red. If they find out what color their own cross is, they get set free, else they get killed. Who would be the guy who can know for sure which color his cross is.