Blackfishing is a word used to accuse someone of pretending to be black or mixed-race.
Nicki, who features on the song, said on Instagram: "Y'all gotta stop."
She added on Twitter: "If you know someone has been suicidal from bullying in the past, why try to get a bunch of people to bully them again about something else?"
The pair went on Instagram live to promote the single.
Jesy said she was "in a group with two women of colour for nine years" and the subject was never brought up until they recorded her last video with the group, Sweet Melody.
As Jesy began to explain about being messaged by one of her ex-bandmates, Nicki interjected saying people should "focus on your own energy."
Boyz, which samples Diddy's song Bad Boy for Life, is Jesy Nelson's first solo release since leaving Little Mix last year.
In the video, Jesy is heavily tanned and her hair is styled with wigs and braids. She wears basketball shorts and gold teeth, and sings about wanting a boy who's "so hood, so good, so damn taboo".
Some people say it is wrong for white people to profit from imitating stereotypically black characteristics - when black people themselves have been held back for having the very same characteristics.
But Nicki Minaj said: "There's a lot of women out here in the United States that tan, get bigger lips.
"I wear straight blonde hair when I want to."
Nicki went on to say Jesy's bandmates were only "calling [her] out" to help their "personal vendetta" against her.
She said: "Don't wait a decade after you've made millions with the person."
Jesy laughed as Nicki Minaj referred to her ex-bandmates as "clowns". So far, there's been no response from Leigh-Anne or the rest of the band.
Jesy later said: "My intention is never to offend people of colour with this video and my song.
"When I was in the video with [Nicki], I didn't even have any fake tan on. I'd been in Antigua prior to that for three weeks.
"I'm just really lucky that as a white girl, when I'm in the sun I tan so dark.
"My hair's naturally curly, I've always had curly hair. I wanted to get a wig that emulated the same texture as my hair, I genuinely didn't think I was doing anything wrong."
In a recent interview with Vulture, Jesy said: "I'm very aware that I'm a white British woman; I've never said that I wasn't."
Newsbeat has asked Jesy Nelson's label Polydor to comment about the video but so far, there's been no response.
Some fans have been asking how this video was approved, and why the artists decided to discuss it on Instagram live.
George Griffiths is a freelance pop music critic, and he was tuned in last night. He told Newsbeat he found it "really chaotic".
"I know everything is managed by PR and management to a certain degree but that did not look managed at all.