If the ads were silent banners changing the aspect ratio of the videos, I wouldn't mind them.
This is an interesting idea. It is already used to some extent in live football matches broadcast on TV. The sidelines have their ads replaced. I think your idea is a good one and deserves some testing by Google/YouTube.