To me, I think it's the event. Supporting the team is part of what makes the event. If you're just there as a neutral then it's less of an event because it is, by definition a competitive thing.
I can't seperate the idea of 'being there' and 'supporting' as they're entwined.
It's the fact it's live. The sense of something playing out in front of you. There's something magic, hypnotic about it. It's familiar and on one level predictable about it, but on another level, each game is unique. There's a kind of constrained chaos in a football match, a randomness within the way it unfolds that is really engaging.
The shared experience is equally part of it. I'm not sure sitting in an empty ground watching the game would be any better than watching on ifollow. Being part of a crowd, all focussed on the same thing. Feeling the reactions of thousands of others. It's quite a unique experience. I can't think of many other places I've felt like that. Even at a gig or something, there's not the shared tension of 'what happens next'
I agree about the tribal thing as well. It taps into something deep. The need to be among 'your people' and guard your borders from 'the others from over the hill' or something! I dunno. There's lots in our psychology which is from long ago.