Yes I would definitely anneal the neck/shoulder area as brass does get brittle with age even when not being fired and worked in dies. Annealing does no harm even if not required immediately, problem is you won't know whether required or not until you seat a bullet or fire the loaded cartridges, then it's too late if they split.

However not all is lost with scarce or valuable cases if you do get some neck splits, I have devised a method to successfully silver solder splits in several of my 404 Jeffery cases that split when I got a bit lazy with annealing or in the case of my early Kynoch factory ammo where some split the neck when first fired. Have even soldered a split from neck to mouth on one case.
After annealing and polishing the repaired cases it is near impossible to see the repair and I carry on sizing and using them for cast bullet loads trying to remember to anneal them every second firing.