Hundreds of protesters and refugee rights advocates have called for the government to bring the refugees on Manus Island to Australia.