I also first enabled both, but later disabled Pushover. I don't know if you can do what you want but:

  • You probably already did it,
  • It's logical it can be done.

:)