One of the attackers held a gun to the head of one of the refugees while the other two were armed with bush knives.
The stolen items included mobile phones, laptops and speakers.
Refugee Action Coalition spokesman Ian Rintoul said no one was injured in the attack at the Koki Apartments, but the refugees had been further traumatised.
He said refugees were warned to stay in their rooms, especially during the election period, as they had become routine targets of violent assault in Port Moresby.
Fourteen refugees are housed at the complex where the robbery took place, and the attack has renewed calls for the remaining 100 refugees and asylum seekers being held in PNG by the Australian government to be evacuated to Australia.
Rintoul said PNG had never been safe for refugees, who had been held for more than nine years with no prospect of safe settlement there.
"Many of them have been accepted to the United States and Canada but have heard nothing from those countries for years," he said.
"They should be given safety in Australia."