Lactose after fermentation

The bitterness could be from other things, such as hops of course… But also, 3 oz of vanilla is too much IMO and can add bitterness in high amounts in my experience.

I would not add more than 1 lb lactose in 5 gallons, total.  You might already be there.

Perhaps just give the brew more time to mellow.  Maybe needs 4-6 months or more.