Not when your metals are held in an IRA. Your physical assets owned by an IRA must be held by a qualified custodian and stored in a secure, IRS-approved depository. Many depositories offer segregated or non-segregated storage. Segregated storage means that your personal metals are identified and stored under your name.