Ok I played this mode one time at a friends house where there was a square and we had to stop a bunch of enemies from getting in the square. What's it called? Thanks.

If it as you and your friend against enemy ai then its called co-op Defend, if it was you and friends against other people then it was called Siege or Blind Siege.