It will take time to oust die-hard remnants of Saddam Hussein's deposed regime in Iraq, U.S.Defense Secretary Donald H. Rumsfeld said Tuesday.