The dichotomy between story (or narration) and rules (or simulation) is a constant topic of discussion in the world of RPGs and a source of heated debate. Today we have RPGs spanning the entire range from freeform story-inspired to highly-mechanical simulation-based games.
To this wide spectrum of games corresponds a distribution of players with varied preferences and opinions about how to best combine narration and simulation. Inevitably, players sometime identify these preferences with what an ideal or optimal RPG should be, thus possibly leading to their determined efforts in the spreading of this conviction. Much has been written on the complicated relationship between story and rules in RPGs, with extreme positions subordinating one to the other. Here I will add my two cents to this problem.
A matter of balance. In reality, RPGs should not be characterized by the dichotomy between narration and simulation, but, on the opposite, by their confluence. The origin itself of RPGs is rooted in the encounter between heavily simulative games like wargames and narrative fascination with fantasy and sci-fi fiction.
The distinctive trait of RPGs, with respect to other boardgames or to storytelling, lies in their ability to find a working point of equilibrium between the two aspects of narration and simulation. The strenght lies in the balance and in the interaction between them.
A matter of degrees. Although the existence of this equilibrium point may be recognized as the discovery of the first RPGs, this space of harmonic interaction turned out to be quite wide. Players may freely follow their tastes in privileging either storytelling or simulation.
Most of the time, this will just be a matter of degree: how much space do we want to give to mechanical simulation over storytelling, or vice versa? The answer to the question is probably the result of an implicit negotiation among all the players on their expectations and ideas of fun. Each equilibrium constitutes a particular variant of RPG.
Efforts to prioritize one of these two aspects at the complete detriment of the other lead to the exploration of those peripheral areas of RPGs that border with pure storytelling or with simulation games. That, in itself, is not problematic, although some richness derived from the interaction of narration and simulation may be lost.
Two domains. Thus narrative and simulation constitute two essential sides to RPGs: you definetely lose something if you give up one of these two aspects; in the extreme, you would lose the identity of RPG itself. Agreeing that narration and simulation should co-exist, and granted that they may come in different ratios according to the desires of the players, a different question is now how the two domains of narration and simulation should interact.
My personal opinion here is that simulation and narration constitute two distinct domains which normally should not be confused. Simulation is the domain of objectivity and verisimilitude; it is driven by clear rules, dice rolls, tables and results designed to capture and emulate the real world (whatever real means in a given campaign). Narration is the domain of subjectivity and invention; it is moved by interpretations, ideas, dialogues and exchanges that reflect how meaning is built in an imagined world.
Rules into narrative domain. Personally - and I remark personally - I find that rules should be confined to the domain of simulation. Storytelling, either on the side of the DM or the players, should not be constrained by quantitative rules in whatever form.
Take for instance the common dynamics of using forms of narrative tokens exchanged between players and DMs to affect the story according to their interests or tastes. I am aware of the argument supporting such a system as a means to promote a better sharing of storytelling between the DMs and the players, or simply to enhance engagement of the players and spur their contributions. However such rules seem to me an undue interference of a rule-based mindset into the domain of storytelling. I believe that the narrative act should be absolutely free. Neither player nor the DM should account to any form of economy for their creative narrative contributions.
Despite good intentions, I believe that the best atmosphere for the expression of creativity is a place free from any nudge or pressure, no matter how well-intentioned. I believe every player should be made comfortable to express herself, and the DM should make an effort to guarantee such a space is available and not overstepped by other players. Contributions can not be pushed, and if a player is happy providing a more limited input (in terms of the number of contributions or their impract), such a degree of participation should be respected, not artificially spurred to increase via game mechanics.
Narration in the simulation domain. Similarly, storytelling is not normally expected to bend simulation to its will, except in few rare cases. After dice are rolled, it is the common (implicit) agreement that the outcome of the roll will be followed. The DM or the players may contravene the dictated results, but this requires a strong justification. Contradicting the mechanics of the game too often simply weakens the rules and deprives the roll of dice of its tension, meaning and significance. Even if the DM would have the authority to decree against any single result, doing so would unravel the foundation of any simulation.
Although narration often informs the simulative part of the game (for instance, determining which dice to roll, what a success or a failure looks like, and so on), once this information has been provided, the simulation normally runs its course all the way to its output. From the moment the dice are dropped, narration hands over its authority and is forbidden from interfering with the random process and the interpretation of its results.
The collapse of the domains. An egregious exception to this principle of separation of concerns between narration and simulation happens when the DM decides to take a roll behind the screen. Here, suddenly, the necessity of mechanics may be easily bent to narrative purposes. The DM can take the roll at face value, but she is also free to interpret the result on any scale she judges important.
Notice, though, that this process is not an open violation of the rules. There is a difference between rolling in the open and disregarding the result, and grabbing the dice and rolling a hidden result. Rolling behind the screen is part of the ruleset; it is a moment in the simulation where some human input (alongside a random input) is accepted.
This follows from the special status that the table behind the screen holds. This is an area of singularity where the word of the DM has utmost authority. This may be derived from that sort of paradoxical rule 0: a rule (as such, part of the simulative world) which established the free will of the DM (a narrative act) above any (mechanical) rule. Such a rule should be used sparingly, but behind the screen is a place where it can be applied more freely.
Synergy. So, although an overlap of narration and simulation is explored in many forms of RPGs, my personal opinion is that the separation of the domains should be maintained, or, at least, upheld except for a few circumstances.
Delimiting and enforcing this division is a matter of promoting a concordant synergy: simulation and narration provide the best service to the expression of creativity as long as they are concerned with their own domains.
The genius of RPGs, then, lies in recognizing this dichotomy, yet understanding the potential in their encounter, and harmonizing this relationship. The exact way of doing it, is a matter of experimentation and fun.