I thought leaving home ain't easy was about leaving your wife. Though I guess it can fit both. It's just I kinda think the lyrics fit better for this explanation.
'I won't lay no blame
I won't call you names
Cause i've made my break
And i won't look back
I've turned my back
On those endless games'
No?
It does sound a bit like Freddie to me, but if Brian says it's Brian then it's Brian I suppose (though he has remembered stuff wrong before).
Anyway I've certainly never meet a woman who sounds like that! I'd be worried if I did..