updated 03/16/2011 AT 4:00 PM ET
•originally published 03/17/2011 AT 12:05 PM ET
Simon Cowell has picked a music mogul to be a judge on The X Factor.
Island Def Jam chairman Antonio “L.A.” Reid will sit alongside Cowell on the panel, a source confirms to PEOPLE.
While he doesn’t mention X Factor, Reid sent out an email to his staff on Thursday announcing, “I have decided to leave my position as chairman of the Island Def Jam Music Group. I have always thrived on growth and the next great challenge, and I look forward with much enthusiasm to what the future holds.”
“We always wanted to work with L.A., whether it was now or in the future,” Cowell recently told PEOPLE when asked who’s in the running for the judging slots. “Because in terms of his credibility and what he s achieved, he s always somebody who we would love to work with … He s had so much success. And he s also a good friend of mine.”
The X Factor, which is set to premiere on Fox in the U.S. in the fall, is holding auditions in Miami, New York, Seattle, Los Angeles, Dallas and Chicago. There’s a $5 million recording contract up for grabs for the winner of the show.
In addition to Cowell and Reid, two more celebrity judges are expected to be named. While there’s been a lot of speculation about who will join the show, Cowell told PEOPLE he’s still meeting with candidates.
“We’ve met with Paula [Abdul],” he says. “What I’ve always loved about Paula is that she actually has got good taste in music.”
Fox had no comment.
For more on Simon Cowell’s thoughts on American Idol and The X Factor, pick up this week’s issue of PEOPLE on newsstands Friday