If it's press fit and not coming off there's a decent chance the latch is siezed. You might want to thread some breaking fluid in, if you can. Or take a padded hammer and tap it very carefully and just kind of work it lose. Maybe a hair drier to heat it up a bit (not a heat gun). My guess would be that it's a collection of that brown dust/rust/oil crud endemic to old guns.