Basuki Tjahja Purnama, a Christian, is the first ethnic Chinese to hold the governor's post in the capital of majority Muslim Indonesia.
Protesters gathered at the Istiqlal Mosque and are heading towards the presidential palace.
Police are bracing for the possibility of religious and racial tensions.
In 1998, a wave of anti-Chinese sentiment led to mobs looting and burning Chinese-owned shops and houses. Ethnic Chinese make up about 1% of Indonesia's population of 250 million people.