Namibia led at half-time in a World Cup game for the first time after two penalties by Theuns Kotze.
Georgia were guilty of too many handling errors in a first half that lasted 68 minutes in real time.
However, they scored two tries after the break through Mamuka Gorgodze and Lasha Malaguradze to help seal victory before Kotze went over for Namibia.
While that try earned the Africans their first ever World Cup point, the result lifts Georgia, who have eight points, above Tonga into third place in Pool C.