The Home Office has promised to fast-track deportation reforms in the wake of a shocking knife incident in Edinburgh involving an immigrant suspect.