Keyakizaka46 went on stage in Yokohama on 22 October wearing black capes and caps resembling those of SS officers
Social media users were quick to point out similarities with the uniforms of Hitler's brutal paramilitary force.
Sony Music Entertainment (Japan) said in a statement: "We express our heartfelt apology for causing offence."
It blamed "our lack of understanding" for the incident, which had prompted the Simon Wiesenthal Centre to demand an apology.
The band is a sister act to the country's popular AKB48 super group. Both are produced by Japanese hit-maker Yasushi Akimoto, an executive board member of the Tokyo 2020 Olympics committee.
In a statement on the band's website he also apologised for his "lack of oversight", saying he had not been aware of the clothing choice before the performance and that he did not think the designers had intended the Nazi parallels. Staff will be better educated and outfits will be checked in future, he said.
With Japan determined to make a good impression in 2020 despite spiralling costs and scandal, the girls' outfits prompted some to hope (in Japanese) that the Games' opening ceremony was being prepared with a bit more care.
Ignorance of the offensiveness of Nazi iconography is surprisingly widespread in Japan and across Asia, and commentators have suggested the mostly teenage pop stars themselves were unaware of the significance of their striking look - one that Sony in Japan has had to apologise for before, when boy band Kishidan appeared on TV in similar uniforms in 2011.