Nauru, which is home to an Australian immigration detention centre, is hosting this year's leaders' summit, which will get under way on September 1.
Amnesty International has co-signed a joint open letter representing a coalition of 84 NGOs, which addresses Forum leaders and observers about conditions for refugees on Nauru, and also PNG's Manus Island.
Amnesty's Pacific researcher, Roshika Deo, said the system is clearly broken if children on Nauru are considering self-harm.
The letter calls on the Australian government to immediately end offshore processing, and move refugees and asylum seekers to Australia or another safe country.
"While it is hard to call out close neighbours and allies, the Pacific Leaders Forum is the correct space for this urgent discussion. It is important that Pacific Island countries hold Australia and each other accountable to human rights obligations for all refugees and people seeking asylum," the letter said.
Amnesty International said leaders must take urgent action to stop the "escalating health crisis" for refugee children on Nauru.
There are 119 children being held on Nauru, with their families.
Australia's new Foreign Affairs Minister Marise Payne will be attending the Forum instead of Prime Minister Scott Morrison - it will be her first international engagement in the role.