First off. No matter what you post, it wont suit everyones tastes.
This being said, i like it, i think it works for comedy, its very early, i dont imagine you will have lines for arms in the final. I specifically dont think it would work for serious stuff though. But that’s A ok. As a general rule of thumb it would be better when dealing with such short animations to deal with comedy subject matter. In that situation the raw audio is an asset, blending character and player is an asset. not necessarily flowing in real time is an asset. it kind of all works together.
If you were doing serious business moments from shows raw audio from the show might be more difficult to work with. The stop and start in the action would need to be handled carefully so as not to work against the animation. It would be a very different animal altogether to work with and it would go either way as to whether it would be better presented with all aspects of the meta /player discussion stripped or included.