A man was arrested and "the situation is now contained", police said.
The incident was not believed to be terror-related, said Victoria Police Acting Commissioner Stuart Bateson.
The incident happened at the Bourke St Mall, a busy shopping area, just before 14:00 (03:00 GMT).
Mr Bateson told reporters that police believe the driver "deliberately drove into the crowd".
Footage broadcast on Australian media showed a maroon car driving erratically in circles in front of nearby Flinders Street railway station.
Overturned pram