Brown, 35, won the TV baking competition in 2016 and married boyfriend Liam Macaulay in France in 2018. But this week it was revealed the couple have split after less than two years of marriage.
Macauley told MailOnline: “There is nobody else involved. We have just sadly decided it wasn’t working.”
But the tree surgeon admitted the photograph of Brown and Bake Off judge Hollywood sharing an “intimate kiss”, published in The Sun newspaper in 2017, had caused tensions in their relationship.
He said: “I’d not long asked Candice to marry me and I had to read reports that my new fiancee was seen kissing Hollywood and getting close to him.
“That was just a kiss on her cheek and there was nothing else to it. But we had only been engaged a few days. It was a lot to have to deal with.
“Sometimes your mind plays tricks on you. It can tell you you’re no good, that it’s all hopeless.
“But remember this; you are loved, and important and you bring to this world things no one else can. So hold on.”
Macauley said he had moved out of the flat he and Brown shared above the pub they own in Milton Keynes.
As news of the split was made public, Brown tweeted: “Think I’m going to eat a lot of biscuits today.”
Last October she opened up to White Wine Question Time about her fertility struggles.
She said of being asked about plans to start a family: “People have got no idea that sometimes your body doesn’t work the way you want it to and that’s really hard, but people don’t understand that.