Ex-judge 'fears for the safety' of colleagues being attacked by Trump