Up 16-3 at halftime, the Pumas tackled their hearts out in the second half to take the win.
Nicolas Sanchez scored all of Argentina's points with a try, six penalties and a conversion, while the Pumas held off a fierce late onslaught from the All Blacks to secure a famous victory in their first test for more than a year.
It was Argentina's first defeat of the All Blacks in 30 attempts going back to 1985.
The Argentinians were given almost no chance of registering their maiden victory over the All Blacks in their tournament opener after being kept off the pitch for almost the entire season by the global health crisis.
Sanchez was a clear Man of the Match but it was a 23-man effort as the Pumas tackled frenetically, rucked like demons and ran the ball with confidence throughout the 80 minutes in front of 9063 fans enjoying the late afternoon Sydney sun.
"It's a big game for Argentina rugby today," said captain Pablo Matera.
"There's a lot of guys who have pulled this jersey on before us and this allowed us to get here. We're used to (adversity), it's always tough for Argentina. We just wanted to show our people that if you work hard, you get your rewards."
The All Blacks were looking to impose themselves on the competition after losing 24-22 to Australia last week in Brisbane, but instead fell to back-to-back test defeats for the first time since August 2011.
Captain Sam Cane finally forced his way over the Argentina line in the 53rd minute and winger Caleb Clarke followed after the hooter but it was too little, too late for the All Blacks.
"It was very tough, full credit to Argentina, they came out with more intent than we did and they thoroughly deserved their victory tonight," said Cane.
"They just kept coming, a lot of errors from us."
New Zealand remain top of the standings with six points ahead of Australia and Argentina, who both have four.