He died close to his childhood home in Gladstone, Oregon, on Saturday due to complications from diabetes.
James was best known for appearing alongside Sir Roger Moore in the Bond films Live and Let Die and The Man with the Golden Gun during the 1970s.
His daughter Lynn said: "He was the most outgoing person, beloved by everybody."
She added: "I don't think the man had an enemy. We were incredibly blessed to have had him in our lives."
In 1973 James played Louisiana sheriff JW Pepper in Live and Let Die, in which he made a memorable appearance in a chaotic boat chase sequence.
His character proved so popular he was asked to reprise the role in 1974's The Man with the Golden Gun, involving another car chase, in Thailand, and a scene where he gets pushed into water by a baby elephant.
Paying tribute on Twitter, Sir Roger wrote: "Terribly said to hear Clifton James has left us. As JW Pepper he gave my first two Bond films a great, fun character."
You only appear twice
While many James Bond characters either feature once or have recurring roles in the series, Sherriff Pepper is among a select few characters to have featured in just two different titles.
Here are four others:
Sylvia Trench: A love interest of Bond played by Eunice Gayson. She appeared alongside Sir Sean Connery in the 1962 film Dr No, as well as 1963's From Russia with Love.
Jaws: One of Bond's most famous villains, Jaws was played by the late Richard Kiel. The character appeared with Sir Roger Moore in the 1977 film The Spy Who Loved Me and later in 1979's Moonraker.
Valentin Zukovsky: An ex-KGB agent turned Russian mafia head who was portrayed by Robbie Coltrane. He featured with Pierce Brosnan in the 1995 film GoldenEye and 1999's The World is Not Enough.
Rene Mathis: A French intelligence operative played by Giancarlo Giannini. He appeared alongside Daniel Craig in the 2006 film Casino Royale and 2008's Quantum of Solace.
James grew up just outside Portland during the heart of the Great Depression, in which his family lost all their money.
He served as a soldier with the US Army in the South Pacific during World War Two, for which he was awarded two Purple Hearts, a Silver Star and a Bronze star.
His acting career spanned five decades and included stints on stage, TV and film.
Other credits include appearing in the TV series Dallas and films Superman II and The Bonfire of the Vanities.
His last film credit was a 2006 comedy, Raising Flagg but he had also been cast to star in an upcoming independent film called Old Soldiers, according to IMDB.