Police say they suspect the 27-year-old mother of killing the children before attempting to take her own life at a train station in nearby Düsseldorf.
Few details have been provided, with no information about the cause of death.
Emergency services were called to the residential block in the Hasseldelle area of the city on Thursday afternoon.
Responding to call at about 13:45 local time (11:45 GMT), police said they arrived at the apartment building in Solingen, in North Rhine-Westphalia state, to discover the bodies of five children - three girls and two boys - aged from one to eight.
A sixth child, reportedly an 11-year-old boy, was said to have survived.
The children's grandmother, who lives 60km (37 miles) away in the city of Mönchengladbach, had alerted the emergency services, the German news website Bild reported.
Police spokesman Stefan Weiand said the children's mother had been "seriously injured" after throwing herself in front of a train in Düsseldorf and was being treated in hospital under police guard.
"Background and further details are not known at this point and that is what we are trying to find out," Mr Weiand told journalists, adding that police investigators were at the scene "in full force".
Police said they were hoping to learn more about the "incredibly tragic occurrence" after speaking with the mother.
The entrance to the block of flats in Solingen has been sealed off and images show police cars and ambulances lining the streets, with forensic officers also at the scene.
The mayor of Solingen, Tim Kurzbach, wrote on Facebook that he had visited the apartment block where "this terrible act took place" after hearing the news.
"For me it is still incomprehensible," he wrote in the post, adding: "Today is a day of mourning for all of Solingen."
Later on Thursday, residents began leaving flowers and candles at the entrance to the building as a tribute.