Well, last time I talked about how I want to focus more on roleplay than on game mechanics in my starter RPG. Don't get me wrong: I'm a huge game mechanics nut. Sometimes I enjoy designing game rules more than I do actually playing them. As a kid, I once stayed up all night designing a wargame for little green army men that we never got to play because we were so excited about coming up with rules for each weapon, each vehicle, and every situation we could imagine. I mean, that is why I'm writing a game design blog right now, after all. So, for a starter RPG, whatever rules I do come up with, I want to be quality rules, but I want them to push the focus towards roleplaying rather than on to the interaction of different parts of the rules (such as which combination of options makes the best combat monster, or whatever). Later on, I'll have a whole series of posts on what I've learned from reading the works of smarter and more experienced game designers than myself, but for now I'll layout some general thoughts on the interaction of rules and roleplaying.
In my post on Setting vs Genre, I talked about how rules interact more with the 'genre' of a game than with the setting. Basically I defined genre as having more to do with how things in your game world act and what people do with them, whereas setting is the stuff that's there. With this distinction, it's pretty obvious that rules would have more to do with genre since rules cover actions. You don't have rules for whether a tree is there or not (well, you might if you have an anally retentive terrain generation system for a focus on tactics, but even there your interest is in how it will interact with the people around it). But you do have rules for the more common actions, like fast-talking and shooting and investigating and such like. And I'm willing to bet that your game has much more detailed investigation rules if it's about discovering hidden truths and following mysteries than a game about fighting monsters and taking their treasure would have.
And this gets us into an important interaction between rules and roleplaying: rules are for when the characters do stuff. And, more specifically, they're for when they do stuff that someone else might take issue with. You usually don't roll because a character walks across the street. But when he rushes across the street attempting to dodge machine gun fire, you probably would roll something! There's a conflict of interest here. This is what makes the situation interesting enough to focus in on with rules. That conflict also makes it important to determine which party gets to have its say, and things are more fair when there are rules involved. Otherwise it might be roleplaying, but it's not a game.
This brings us back to genre. Genre helps you to determine what conflicts of interest are worth paying attention to, and how much attention to pay to them. Pretty much any game is going to pay at least some attention to people trying to do violence to one another (or to owlbears or mynocks, or whatever), but it says a lot about the genre desired if the game has detailed rules on quick-draw duels or back and forth sword fights or large scale battles. A particular game system might have some kind of rules for all of those violent conflicts of interest, but it's a good bet that it focuses in on some more than others, and it's a better bet that that focus is on those ares most relevant to the genre of the game.
Since actual play time is the only currency that really matters in a game (hat tip to Ars Ludi), genre influencing rules will actually bend the roleplaying towards the genre the game is designed for. If you cover a pistol duel with one quick set of highly abstracted rolls that nobody is really optimized for, pistol dueling is a lot less likely to take a central place in the headspace of the players. If, on the other hand, you have detailed step by step rules that tie into multiple characteristics such that everybody has something relevant to pistol dueling and some folks are especially good at it, the game is going to take more time and focus on that situation, and players will devote more time and attention to it.
The other side of this coin is pretty important, though. And that is where rules substitute for roleplaying. Imagine a game where you simply said "Okay, roll to see how well your conversation went" with zero actual talking. Again with thanks to Ars Ludi, one common example is the spot check. Overuse of perception type skills takes players out of asking questions about their environment and becoming involved in it. And this brings me to another important element in 'what rules are for in an RPG': rules are for doing the things that the players can't do.
This might cover the fact that there are no actual goblins to hit with a sword, or it might cover the fact that most folks who want to play a scholar of arcane secrets probably aren't themselves scholars of arcane secrets. But when it comes to deciding what a player pays attention to, or coming up with plans, that very much is in the player's control. When it comes to running a game, my rule of thumb when deciding whether something should be handled by the player or by the character's abilities is to go with what results in more interesting play. Thus, if a fairly dumb and unimaginative character is played by a clever player who comes up with a zany plan for a heist that will make the game more fun for everyone, then the dumb character has a brilliant flash of insight. Or, if a player doesn't know much about astrophysics, but it will make the game go along better for his character to come up with something in that field, then he can break out the dice. But for writing rules, especially those meant to encourage roleplaying more than mastery of a particular game system, I want to build in the right preferences, rather than relying on individual GMs to 'make it work' (though I'm sure they will, as GMs making fun games out of whatever rules they use is a long and proud tradition).
I think the way to go here is to create broad, flexible rules that deal more with generals than with specifics. Perhaps build in the option for players and GMs to delve deeper into specifics if they want, but if I have a choice between creating a game where you more frequently here "Well, there's not really a rule for that, so just roll X + Y and we'll see how it goes" than "No, you can't do that, because you didn't take X trait, Y feat, and Z specialization", I'd much rather go for the former. Some of the game rules I've read recently have introduced me to some really exciting ways to get a lot done with few, streamlined rules, and in the next few posts I'll discuss some of these ideas. But in the meantime, what do you think about how rules encourage or discourage roleplaying?