Matthew Perry is getting candid in his new memoir Friends, Lovers, and the Big Terrible Thing. In an excerpt published by The Times, he admits he broke up with Julia Roberts in the 1990s because he felt he would “never be enough” for her.

The Fools Rush In actor said he and Roberts began dating after she did a cameo on Friends. However, Perry had trouble believing it would last. “Dating Julia Roberts had been too much for me. I had been constantly certain that she was going to break up with me,” he wrote in the excerpt, adding that he felt like the Pretty Woman actress was “slumming it” with him.

Battling multiple addictions at the time, Perry continued, “Why would she not? I was not enough; I could never be enough; I was broken, bent, unlovable. So instead of facing the inevitable agony of losing her, I broke up with the beautiful and brilliant Julia Roberts.”

The 17 Again star said he couldn’t “begin to describe the look of confusion on her face,” when he ended things.