He joins label president Jason Iley and Clive Cawley - who has been promoted from label manager at imprint Vertigo to MD of Mercury, reporting into Iley.
Elsewhere, Elliot Taylor from Universal's central sales team becomes commercial director for Mercury, while Rachael Paley returns to the label as senior marketing manager after two years away at Rhino and Warner Music.
Smith will start his new role in September.
The exec left Columbia in April after six years at the company as MD. He signed the likes of Calvin Harris and Ryan Adams to the Sony label.
At EMI Music Publishing (1992-2006), he signed the likes of Supergrass, Arctic Monkeys, Kasabian and Blur - who he also signed when working for MCA in the early '90s.
"I was proud and pleased that immediately after the news got out that I'd left Columbia, David [Joseph, Universal Music UK chairman and CEO] and Jason were on the phone straight away," Smith told Music Week. "Mercury do an incredible job on so many levels but I'll be judged ultimately on the acts the UK A&R department sign and break around the world. I've no doubt I've got 100% support from Jason and David in doing that job."
"It's an absolute coup for Mercury to have Mike working with us," Iley told Music Week. "Speak to anyone in the industry: they'd agree with that. He's incredible with artists and his knowledge of music is phenomenal."
Pick up the new Music Week magazine today for the full story and reaction from both camps.