add some sort of penalty to using brown note on large groups of people

[Erev]

There's already a hard counter - the bible. 

And no you don't need to put it under the brown noter for it to work. You place it under yourself. The game will not biblegib you, it will biblegib the brown noter and it's completely unavoidable.

This - and I prefer it as-is as a super hard counter.