Why didn't the Eastern Jin government levy reconsideration on the expatriates who went south from the north?

At the beginning of the Eastern Jin Dynasty, the residents in the north and south were registered as "yellow race" and "white race", and no tax was levied on white people. In the late Eastern Jin Dynasty and the Southern Dynasties, the government from time to time incorporated the permanent residence into the household registration of the counties where they lived, and the whites gradually became indigenous, which was conducive to expanding the scope and promoting the development of land ownership.