Should we Only Have One Teacher?