You can point to territorial disputes between non-agrarian humans to suggest otherwise

Hunting grounds and shelter were more important than mutual aid

You don’t even need to use humans, you can use other primates