The local council in Kakogawa placed a piano inside the area's main railway station in November.
Authorities hoped residents would respond to a global trend that has seen a huge rise in street pianos.
But officials were disappointed by apparent rule-breaking, such as people playing for too long or singing.
In law-abiding Japan, some directives were laid out for the piano's use. Officials insisted that users should disinfect their hands before playing, that performances should be kept to 10 minutes and that people should avoid voice accompaniment.
But they decided to pull the plug on the grounds that too many people used the piano for longer, or sang loudly while playing.
Some budding musicians stand accused of practicing the same sounds, over and over - for up to an hour - while others provoked complaints by continuing to play during station announcements.
Officials said they had issued warnings, but saw no improvement.
But there is hope for local music lovers. Officials say they might place the offending piano in a different public location, away from the station loudspeakers.