Yes, there is that, but there is also the real life example that war is inevitable. There will always be two factions at odds with each other. Just as you had Grindlewald before Voldemort, there would always be somebody to fight. So, there most likely would have been multiple factions growing that we weren't even aware of.