The Social Contract is the entire agreement - both spoken and unspoken components - between the players of an RPG.
Fundamentally, you play an RPG by meeting up with your friends and saying things and expecting them to react in a particular way. The way in which they react will, compared to most situations, be unusual - after all, if you walk up to most people and say that you're hitting the goblin with your axe, they'd probably run away and/or call the police. Thus, there is an agreement between the players covering how they will react to statements of this kind, and to any other statement or action taken during the game. This is the social contract. (In RPG theory, social contract does not refer only to basic human social standards such as civility and hygiene.)
As such, the social contract absolutely defines the roleplaying experience: aspects such as the game system rules are just more explicit parts of the social contract.