The shark bit the leg of the surfer, aged between 50 and 60, off Kingscliff, 800km (500 miles) north of Sydney.
Several "heroic" boarders tried to help the man by fighting off the shark but although he was brought to the beach, he died there of his wounds.
This is the third fatal shark attack in Australian waters this year.
The Sydney Morning Herald said that although police gunmen were deployed to find the shark, they were unable to kill it and it left the area after several hours.
Police are licensed to kill sharks if they are considered a threat to human life but no shots were fired in this case.
The attack took place off Salt Beach at about 10:00 local time on Sunday (midnight GMT Saturday). The shark bit the man's thigh, then circled those who came to his aid, reportedly ramming one of their boards.
The surfer, who the Herald said was from the Gold Coast, died at about 10:40.
"He was rendered first aid for serious injuries to his left leg but died at the scene," a police statement said.
Paramedic Terry Savage praised the men who brought the surfer ashore, saying: "It was nothing short of heroic to get him to the beach.
"Unfortunately there was nothing else we could do," he told ABC.
People were ordered out of the water, with jet skis used to bring some in.
Salt Surf Life Saving Club member, James Owen, told the Herald: "It's a bit of a shock for everyone at the moment. There's a very sombre mood here. I don't know of a previous fatal shark incident in this area."
White sharks are said to be active in the area at this time of year.
This is the third fatality from shark attacks this year.
In April, a 23-year-old Queensland ranger was killed in an attack on the Great Barrier Reef.
In January, a 57-year-old diver was killed off Western Australia.
No deaths were recorded last year.