In many ways. The caste system established a society where it was nearly impossible to break the caste you were in once you were in it. This meant that even if there was untapped potential in lower castes, those people's talents couldn't benefit the population since they couldn't do anything with those talents unless they were working or living in the caste they were born into. This also meant that many families probably married into each other as marriages and generally interactions between castes were not common.