The popstar who has appeared on the show a number of times as a performer, could be the judging panel's new 'top dog' if Cowell decides to take a back seat to focus on the US version of the show.
It is thought that the Take That star's cheeky sense of humour and outspokenness would make him the perfect replacement for the music mogul.
An insider told The Daily Star: 'Robbie is a huge name so would naturally be a massive ratings hit. Not only that, he is also straight-talking and comical.
'We all understand that right now America is where Simon's focus has to be. There aren't many people that can fill his shoes, but Robbie just might be able to do it.'
However, TV bosses are also thought to be considering Gary Barlow for a role on the judging panel, which could bring up some old rivalries between the two singers.
Speaking to The Sun after the National Television Awards last week, Cowell said that none of last year's judging panel have been given new contracts yet.
'I think there are going to be a few surprises. I genuinely do. I don't think there's one set way of doing it. I am currently in conversations with ITV and Fox,' he added.