The Brazilian authorities stopped Gunnar Bentz and Jack Conger from boarding the flight at Rio de Janeiro airport, bound for the US, for questioning late on Wednesday.
Two teammates - Ryan Lochte and James Feigen - were barred from leaving the country, but Lochte had already left.
Police have queried their accounts of a robbery they reported in Rio on Sunday.
The four swimmers said they had been robbed at gunpoint in a taxi in Rio.
But police said their account of when they arrived at the Olympic Village did not square with CCTV recordings.