Invitation Etiquette

I'm a little bored so I thought I'd post this question I have. My husband has a single male friend. They fish together and that is fine with me. But on two occasions this friend has hosted a BBQ at his brothers home, where all of his family, kids, grandma etc. have been invited. But this friend doesn't extend the invitation to me and our son. The one that bugged the most was the Super Bowl Party. My husband is a huge fan and me, not so much...but I was in to it this time. His friend invites him to this all family get together and when my husband asked if me and our son could come too, he was told "There isn't enough room" Now don't get me wrong...the question really isn't about this specific incident. My question is, as a married couple, should we accept invitations that don't include the our spouse? In general, I don't want to go to sporting events, or fishing with the guys. But it is nice to be asked and then have the opportunity to decline. The friend has been over for parties etc at our house, but has never brought the "girlfriend" (I'm not convinced he has one, but that's just me.) Where I get hung up is expecting everyone to behave like I would. And the reality is they don't. And the fact that they don't is really more my problem than theirs. My husband has a higher tolerance for idiots than I do and maybe he thinks his friend will annoy me. And to be honest, I don't need or want to be involved with my Husband and his friends, all the time. I told my husband how I felt about the Super Bowl party and he chose to stay home. But I gave in during the last quarter and told him to go. Bonding over a sporting event is something that I don't understand. The hubs said it is more fun to watch it with like minded people. Him going to the party wasn't the problem. Not being invited was. It's a specific example of a broader question.