VoiceOver's volume fluctuates significantly when connected to bluetooth speakers

I noticed this when using AirPlay to play media from my iPad to an Apple TV. As you said, it's just when first beginning to play something, or resuming after something was paused. It affects both speech and sound effects, and I think it's caused by differences in volume settings between the iOS device and the paired speaker. I don't know of any way to fix it, but for me it's infrequent enough that I can deal with it.

I have dealt with this. You are not crazy or alone. Apple is aware of the issue & is working on a fix.
I have found a workaround though, but it's not ideal. If you're listening to media, turn off speech while connected to bluetooth. The sounds will still fluctuate, so it's not a 100% fix, but it's better than nothing. And I've noticed the sounds don't do it as often as before. And if you need to do something then what I do, is disconnect the ipad from whatever bluetooth device it's connected to then turn speech back on.
Only issue I'm running into now, is that sometimes when I'm turning speech on/off, I accidentally turn on screen curtain, and don't realize it! And if you have 2 get sighted help, then, well... you can guess the results. I hope this helps until it can be fully fixed. I'm sorry I can't give you a work around that helps 100%. But know that you are not alone. I have dealt with this too, and it's very annoying especially @ 2 in the morning if I have 2 pause my Bard book so I can get up & do whatever I need to do @ night.