Heirs of Gaye's co-writer, Ed Townsend, argue that Sheeran, Warner Music Group and Sony Music Publishing owe them money for allegedly stealing the song.
As the case opened, a lawyer called Sheerin's use of Gaye's lyrics at his concerts a "smoking gun".
The Grammy winner, 32, testified that he did not copy from Gaye's 1973 hit.
Asked by lawyer Keisha Rice about another song he wrote, Take it Back, which contains the lyrics "plagiarism is hidden", Sheeran confirmed that he had written the words.
"Those are my lyrics, yep," he said, adding: "Can I give some context to them?"
She said that if she wanted any context she would ask for it, and went on to ask him about concert footage showing him mixing lyrics from Gaye's song with Thinking Out Loud.
Earlier, another lawyer for the family - civil rights advocate Ben Crump - told the court that the concert video amounted to a "smoking gun" confession.
Sheeran responded that he sometimes mixed together songs with similar chords at his performances, and appeared to grow frustrated when Ms Rice cut him off.
"I feel like you don't want me to answer because you know that what I'm going to say is actually going to make quite a lot of sense," he said.
"You could go from Let it Be to 'No Woman, No Cry and switch back," Sheeran continued under oath, referring to the Beatles and Bob Marley classics.
"If I had done what you're accusing me of doing, I'd be a quite an idiot to stand on a stage in front of 20,000 people and do that."
As the trial began on Tuesday, US District Judge Louis Stanton warned the seven-member jury that despite the fact that music will be played in court: "We don't allow dancing."
The trial is expected to last at least one week. If the jury finds the pop star liable for copyright infringement, the trial will enter a second phase to determine how much he owes.
The court case comes as the singer prepares to launch a North American stadium tour and release a new album.