The diversity of building up the quiescent sequence at redshift z~1