There is no need to remove the old ground cover (bark, mulch, pinestraw). As the material breaks down, it will add organic matter and nutrients into the soil. It is like a slow release fertilizer.
When applying new ground cover, you want to add enough material so that the mulch is no deeper than 3 inches around shrubs and trees or 2 inches around flowers. Adding more than that can actually be harmful to the plants.
Before adding new material it is always a good idea to cultivate the old material first breaking up any matting that has occured.
