A Portage man is going to prison for his part in a scheme that defrauded 1,500-plus elderly people of more than $560,000.