Yes I agree. 165 to 220.
Bullet construction is FAR more important then bullet weight.

If you go as light as 165 please do yourself the service of using a Barnes or Swift bullet, or perhaps one of the bonded core bullets available these days. Standard construction of 165 gr 30 cal bullets is NOT up to the task with the 300 Weatherby.
But a 300 Weatherby with a good 165gr bullet is an outstanding long range arm. Good to any range you are capable of making hits.
(It's still 98% the hunter and 2% what he uses in all cases)

There are a lot of good 180s to choose from. The 200 and 220s are great killers on big animals.

I am a believer in exit wounds all the time. I have been a hunter and a guide for many many years, and I reject the idea that a bullet should stay in the animal. I want an exit every time.

With that said, and throwing my personal "bent" into the mix, I'd go for a Nosler, Swift or Barnes bullet in the 200-220 grain range if i were doing what you are going to do.