Political Science 30: Politics and Strategy, Lec 16, UCLA

okay i want to say one thing about the last homework that's coming around i'm sure some of you guys are thinking wow I've done all five homework so I've got full credit on them um and I know that one of the grades is going to be dropped I guess I don't have to do this one right and um no that's the wrong model that's why I'm saying it that is the wrong way to think about the homework you are correct that if you've done all your homework so far this one your homework grade is SAT this one it's not going to affect your homework grade but it's definitely going to affect how you do on the final exam okay the impact of all of those homework assignments is not so much the small increment that each assignment has and your overall homework grade which again has a small impact on your course grade the homeworks matter because that's how you practice solving the kind of problems that you're going to have to solve on the final exam okay so what could I say you'd be nuts not to do this homework do it it's also the second question on these blew me back up and say something about kind of a direction I've moved in the last two homeworks those I think was the second question on the homework you turned into the day the one about choosing a candidate that story was a little harder to parse that some of the other stories and one of your assignments one of your questions on this assignment is also going to be a little bit harder to par it's just the the process of going from the scenario to figuring out okay so what kind of game does this mean how do i set up the payoffs it's looking a little bit harder because that's where we're at in the class okay one of the skills that you can't underestimate that we're trying to learn in this class is the ability to go from that kind of unorganized informal description of reality that's the way we talk about and read about the world we live to something that is organized into a game where the assumptions are clear and the logic can be can be spelled out okay so if you find yourself spending more time just figuring out how to approach that second problem that's appropriate it's a little bit harder but I think you guys are ready for it okay okay so last week we did a lot of yeah wait let me look you LAN is asking how prepared you are to do the homework and that's actually a good question okay the first question on the homework is actually a pretty easy one it's just a little warm-up you are you were prepared to do this at the end of last week and if you think you're not prepared to do and you haven't looked at the lecture notes some of the stuff that I put in the lecture notes but didn't go over explicitly in class will also help you with this problem so yeah that problem you're good to go on and I think that by the end of class today you could at least get started on a problem problem too okay if you remind me at the end of the class i'll revisit that but certainly the thing i was just talking about now that setting that up is a problem i'm trying to figure out okay what kind of game are we even talking about here you can you could do that right now as well okay so somewhere I had notes alright so the main thing we did last week was kind of a prelude to thinking about the reputed prisoner's dilemma we talked about one kind of repeated prisoner's dilemma where we couldn't get cooperation and that was the case where the prisoner's dilemma was repeated for a fixed and certain number of times we saw that the shadow of the future undermined cooperation in the known last period and the fact that the last period was known in advance undermine cooperation in all the previous periods okay so if we have a clear and definite time horizon in an interaction that is got the form of a cooperation problem prisoner's dilemma problem the shadow of the future probably isn't going to get us to that happy equilibrium um then I floated the idea that in many cases we don't actually have a clear time horizon maybe I'll see you again maybe I won't the probability can be high the probability can be low but in the real world time horizons aren't necessarily that's certain and that that fact opens up the possibility of cooperation on Thursday we had to take a little interlude to talk about comparing rewards in the future versus temptations in the present that's essentially the problem in the repeated prisoner's dilemma and we sought to use discount factors to do that so now today I'm going to go back to the main narrative okay if we're in a situation where we are playing a prisoner's dilemma repeatedly and we're not sure how long this interaction is going to go on okay can the possibility of future reward for cooperation get us to make cooperative choices in the short run okay can the shadow of the future bring about cooperation can the possibility of if I have been a good guy to you you'll be a good guy to me Connecticut me to overcome the dominant strategy I have to defect in the short term okay so I think you guys already figured out the answer to that is sometimes and the main thing we're going to do this week is elaborate on just what we mean by by sometimes when can the shadow of the future be enough to support cooperation okay so what we need to do now is we need to go back to some basics of game theory now we're in this world of a repeated game which is different than a sequential game with lots of steps remember I talked about this last week we could weave weren't fine shape for analyzing sequential games with lots of steps where we have lots of moves and countermoves till we finally get to a payoff you guys knew how to do that before the midterm we just solve it with rollback now we've got a different story now we've got a game that's repeated over and over again and what this means is we choose our strategies we get our payoffs then we interact again okay and again and again and we're not sure how long our interaction is going to be we're going to be focusing on indefinite time horizons throughout this week what this means is we play our prisoners dilemma we get our payoffs maybe it's the punishment pay off because we both played our short run dominant strategy maybe it's the reward pay off because we both cooperated maybe I got the sucker pay off because I cooperated and you defected maybe vice versa we got that payoff and then when we go to the next round we remember what happened in the game before that's the big difference between a repeated game and a sequential game that just has a lot of moves and repeated games you get the payoffs then you make more choices ok so it's starting to sound more like real life than any of the games that we've done so far okay so what it means for strategies is strategies and repeated games can be a lot to manage okay if I'm saying we're playing and a repeated prisoner's dilemma was an indefinite time horizon my strategy has to tell me what to do at any point in the game for any history okay so my strategy has to tell me what to do the first time we play each other then my strategy has to tell me what to do the second time if you cooperated the second time if you defected then my strategy has to tell me what to do in the third round given all the possible moves that have gone before in the second round okay so that just the set of strategies for a reputed game is a lot to me in it I'm never going to ask you something like right down up all the players strategies in a repeated game because you wouldn't be able to do it okay not something I asked you about regular sequential games where the payoffs happen just once okay so we're just going to have to accept that fact that we're not going to be able to think about all the strategies in a repeated game but we could think about some I'm just really trying to emphasize this at the beginning because it's different than what we've done so far so far I've been I think very thorough and you think about every possible strategy and the strategy is in equilibrium if given the other person strategy there is no alternative that could have given you a higher payoff you're very clear about comparing each strategy to every other alternative we are still going to do that but we're going to have to be creative in finding ways to group this infinite number of strategies okay okay so what we're going to do is we're going to think about one possible strategy in a in the repeated prisoner's dilemma and it's a famous strategy it's called the grim trigger strategy so i'm going to write up here what the grim trigger strategy is cooperate as long as no one has defected in the past if anyone has ever defected then defect so this is a contingent strategy okay the short little strategy can tell me what to do in any repeated prisoner's dilemma at any point in the game okay because this strategy is kind of divided up all the possible histories that a repeated prisoner's dilemma can give rise to and there's lots and lots of them and organize them into two groups okata hey either were in a situation where everybody's been good all the time I've always cooperated you've always cooperated nobody has I've ever defected either we're in that situation or we're not okay if we're in this good situation what grim trigger says is if nobody's ever defected if you've been good to me and I've been good to you I'm going to cooperate hi that's the good part a grim trigger the grim part the reason why it's called so does it's even called the grim strategy is there is no forgiveness in grim trigger okay you the fact you gets me once I will never ever ever forgive you I'm going to defect against you anytime I interact you with you there is no possibility of ever making a man's okay so the idea behind the trigger part is one defection triggers punishment okay and it is grim because there is no forgiving no forgetting so now we want to ask is does both play if both players play grim trigger in a reputed prisoner's dilemma is that a Nash equilibrium okay when we're asking ourselves can these people cooperate can these people overcome their short-term incentive to defect to do something that helps them but hurts the the group as a whole what we're asking is whether grim trigger is in equilibrium or not okay it's always a strategy it's always something that you can do the question is if your partner is playing grim trigger are you glad that you played it or do you wish you done something different okay that's the answer that's the question that's going to have sometimes a yes answer sometimes a no answer if the answer is yes I'm glad when you play grim trigger I'm glad that I'm playing at two then we're in equilibrium okay the answer is when you're playing grim trigger can I do better by doing something else then we're not in equilibrium okay all right so now to answer that question to apply our Nash equilibrium logic the logic that we've been using ever since the midterm and really and mostly before the midterm as well in order to do that now we need to have of a specific prisoner's dilemma up here with numbers that I can actually compare so that's that's where we're going to go now so what I'm going to do now is I'm going to write down the stage game and just again we're calling from last week the stage game is the thing that we're repeating each round in the repeated game means playing this game and getting the payoffs once and then we go on to the next round and we play the stage game again and again we get the payoffs so um i'm using the slack off work hard version of the prisoner's dilemma the one that we were working on last Tuesday player a controlling the Rose player be controlling the columns they have the same choices slack off work hard when they both slack off the payoffs are 0 and they both work hard their pants are higher they get pay us of three um yes I work hard and use slack off I get the low sucker payoff of negative 2 and you get the high temptation payoff of 5 so you like that and I feel bad about it and when the roles are reversed the the payoffs are reversed when I'm the slacker who's free writing on you I get a high payoff and you get the low sucker payoff so it's pretty sure those are the same numbers i was using last week if we play the stage game just once slacking off is a dominant strategy it is a prisoner's dilemma this would be the only nash equilibrium in a single shot game in the repeated game though we're not just saying would you do better in this round by slacking off than working hard we're saying would you do better now and in the future would your total stream of payoffs what you whether you're getting now what you're going to get in the future would it be better if you slacked off or would it be better if you stuck to this strategy all right so when we do the Nash equilibrium calculation when we do that kind of analysis for a repeated game we're thinking about short-term and long-term consequences so we're thinking not just about the higher payoffs this time but lower payoffs in the future because my behavior now is going to affect the way the person I'm plane was going to treat me in the future okay so that's what we have to be able to do and what I'm going to do here is I'm going to do a supplementary table here that will just show the the moves and the payoffs that one player gets from playing grim trigger versus doing other strategies and in that table i'm going to hold the other players strategy constant so let's just get that going up here player a's payoffs started to write from your trigger there let me not say that because I'm going to look at player A's payoffs from different strategies in this table or am i know i think i am going to separate this just from grim trigger here i'm going to trigger and i'll do a different table for other strategies all right so what I need to do now is I need to account for what's going to happen in various rounds I don't know how many rounds is going to be I think by the time we get out to the fourth round you'll see the pattern here okay and we're going to have player a's action we're going to also in this table have player B's action and we're going to have player a's payoff yeah that's how I want to do it okay so this is player a in this table this table is going to show the actions and the payoffs from playing grim trigger okay in the very first round of this game you guys look over there the blue that's the grim trigger strategy I'm player a what am I going to do in the first round what am I going to do you I'm going to cooperate okay grim trigger says cooperate as long as no one has defected in the past well it's the first round no one's had a chance to defect ok so in grim trigger you start off nice no one's defected I am cooperating ok now you might have been wondering why in a table about player a I needed to put in player B's action and the answer is I need to know player B's action in order to get player a is payoff right that's what makes it a game okay so I need to add something to the table we're going to put e in green here we're going to assume be plays grim trigger all sup okay and to get a sense of where we're going with this I'm going to do another table where a does something different here but B is still going to do grim trigger ok that's the Nash equilibrium comparison they are ok so these playing grim trigger in this table same as a sobe is also going to cooperate let's say is payoff 3 that's good okay round 2 i'm player a i'm playing grim trigger in round one i cooperated be cooperated I got three what do I do now cooperate what is V do now cooperate this is where we want to be what's my payoff 3 ok as long as we're both playing grim trigger I'm going to take the liberty of filling in a few more here as long as we're both playing grim trigger all we're ever going to see is cooperation on the equilibrium path alright so for all the mean name of grim trigger strategy the outcome of both players playing grim trigger is a happy story ok grim trigger good everybody cooperating hi payoffs all right so now let's make the other table over here where again the table is going to focus on player a's payoffs from i'm going to call it alternative strategy number one here okay which i haven't said what it is yeah i will say it in a minute but it's going to be something different i bet you guys can pick and guess what it is we're still going to assume that be plays grim trigger all right and we're going to do the same thing and we're going to look at hays action from this alternative strategy these action from grim trigger A's payoff here in round one and round two up to Round four we'll see if we are getting the picture okay what would be an alternative strategy that a could play here what would be an alternative what would be something that might tempt a away from grim trigger just a fact I think I heard that okay and alternative strategy will write it and keep strategies in blue here okay so here's grim trigger alternative strategy number one is always defect okay that's something I could do right go with my short-term incentive every time it's not the only other thing I could do but if this strategy gives me a higher stream of payoffs where my opponent is playing grim trigger then grim trigger is not going to be in equilibrium okay because then if my partner's playing grim trigger if this strategy gives me higher payoffs I'm going to play it all right so alternative strategy it's just always defect okay so that's an even simpler strategy than grim trigger it's telling me I'm just a bad guy I do what's good for me and every round okay um be is playing grim trigger what is B do in the first round he cooperates right in the first round nothing's happened B is following the grim trigger strategy and grim Kruger strategy says nobody's defected you try to do the right thing all right everybody see this I'm varying a strategy here but I'm still assuming the B is playing grim trigger okay what's A's payoff sucker says a I get five isn't that good okay round true now in round 2 i'm be i'm playing grim trigger what am I going to do defect I'm done with you you betrayed me once am I ever going to forgive you am I ever going to cooperate with you again no that's what grim trigger tells me what's a is payoff here okay so now what we want to do is what we want to look at is the overall value of these two strategies this strategy playing grim trigger when my opponent is playing grim trigger is going to give me three in this round three want one round in the future three two rounds in the future 33 rounds in the future but remember i'm sort of uncertain about what our time horizon is i might not even get to three rounds in the future for all i know i might not even get to the next round right this is back to the anybody can be hit by a bus thing okay so what I need to do is I need to find some way to look at the set of payoffs that are going to occur at different points in time and put them all on a scale that we can compare to something I'm going to get right now okay so what I need to do is I need to find the present value to play or a I'm going to just abbreviate a GT okay okay when I'm writing this at the bottom of the table it's the present value to player a pain playing grim trigger given the B is also playing it right you can't assign payoffs unless you know what both players are doing ok so the present value to me of playing grim trigger when my partner is playing it is going to be three today plus what do I need to do to this am I sure I'm going to get this no I'm not okay is it worth absolutely free to me it's worth a little bit less I need to discount it okay I need to use the discount factor like we were doing in class on Thursday because i'm not sure i think that we're going to interact again i think i'm going to see you again i think you'll have a chance to reward me for the fact that i was good this time but nobody can be perfectly sure so i need to multiply that three by the discount factor by this number that's between zero and one when it's close to zero it's telling me that the future does not loom very large to me okay it's telling me that those future rewards are not very big okay I don't think that my probability of getting to the next round is very high or I'm just a very impatient person as Delta becomes close to 1 i'm becoming a more patient person the future is really important to me i think it's really likely that i'm going to interact with player beat again okay but no matter what Delta is less than one and that future award is not quite worth as much as to me as the 3 i'm getting right now okay it's always some uncertainty of course it's not just one round in the future maybe I'll see player be two rounds in the future as well okay now there's even more uncertainty and we're having to wait there so I discount that twice again just like we were doing at the end of class on Thursday I discount this three talents okay I'm um kind of gotten out of ya out of the range of the columns here but not out of the range of the potential interaction this could go on and go on up here to very high values of Delta to some high-power times three ok so that's looking a little bit weird and here we are and we're not done yet okay there's no definite end to this interaction so there's no open it and to the sum here okay that's looking bad but we're going to solve this problem don't worry we're not going to really have to calculate an infinite do an infinite number of calculations here we've got a shortcut for that before I talk about how to evaluate this infinite sum here let's just do the same thing over here let's figure out the value of always to fact so this is the present value to play or a of i'm going to call this so i'm just going to write it out always the fact again this is given that B is playing grim trigger okay well this one's easy okay it's 5 i'm going to get 5 right now i'm going to get 0 every round of the future because you're going to be punishing me forever no matter what so there's no no concern about adding up an infinite number of terms over here okay so what we need to know we need to know is the present value to a of playing grim trigger greater than or equal to the present value a always defect okay if yes then a does not regret playing grim trigger as long as be plays grim trigger I'm not repeating as long as the plays grim trigger every time but it's always in there yes the question is if the payoffs for slacking off here we'll one instead of 0 which they could be it would still be a prisoner's dilemma and then we would have ones here then you're exactly right then we would have an infinite sum here as well ok and that's you're going to see that that's going to be falling ok for this example I'm making it as simple as possible but you'll be able to handle that case in about 20 minutes ok so if we had nonzero numbers here it would be 5 plus delta x 1 plus delta squared times 1 etc if the payoff here was to it would still be a prisoner's dilemma it would be 5 plus 2 times delta delta 2 times delta square plus 2 times delta cube whole thing the overall thing we're doing is we're comparing an infant stream of payoffs from the potential equilibrium strategy okay the potential equilibrium strategy is grim trigger that's the one we're focusing on we're comparing it to other things that the players could do in the game in both cases we're looking at the present value the stream of payoffs over all the periods but appropriately discounted so that we can compare benefits that are occurring at different points of time all right so there's just this little technical issue right now that's preventing us from plugging this on one side of an inequality plugging this on the other and solving it the way we've always done before and the little technical issue is what do we do with this sum that never ends okay so here we've got math to the rescue okay this there's a formula that i'm going to show you in one second that will allow us to evaluate this this geometric sum all right so let's do a little interlude on geometric psalms what they are and what they add up to this is one this is a geometric sum um the simplest geometric sum looks like this one plus first Delta to the second cubed going on forever in this pattern okay um the way you may well have seen it written before is using summation notation so let me just write the same thing here it is the sum as i goes from 0 to infinity of delta to the power I ok now I know these summation symbols people don't like them right in many many classes people will do fine you're doing some mass on the board you put the summation symbol up and people don't like them I think I know why people don't like these symbols they are very dense there's a lot of information in this kind of simple okay so in this class or in any others when you see something like a summation symbol instead of thinking oh there's that confusing thing again that I get wrong all the time just take a deep breath and remember that it's confusing because there's a lot of parts and think about the parts as individuals okay so let's go through what the individual parts of the summation symbol mean okay so the big symbol here is a Greek letter Sigma same thing that some of the fraternities have on the outside of their buildings it is in fact the letter that in Greek makes the S sound & S is for some okay sigma s thumb and it's telling you that this is the sum with a pattern in it the pattern is what's in front of the song okay part of the pattern is changing and that part the part that is changing appears above and below ok so just writing this out in words this is I said it before I'm going to write it though the sum as I goes from 0 to infinity of Delta really be write everything out to the ice power so one thing you can do if you see these symbols and you're unhappy about them you're feeling lost is just try to read them out loud in English and expect it to take some time all right just literally reading the symbol out loud will often help you see what it means another thing you can do yeah this probably wouldn't happen in Pali so I 30 but it may well in another classes these symbols are used to long statistics classes if you see a summation symbol and again you're not sure what it means and yes you've read it out loud and it's still looking a little fuzzy write it out this way sometime this takes more time to do it's kind of tedious it's kind of ugly with Dada and so on going off to the end but looking at this you can see a pattern that's just left and plus it here alright so that's some thoughts on summation symbols what getting back to my main argument now I can have all sorts of things here I can add up this would be a different summation x from 0 to infinity I this would just be some the number is beginning with um the integer is beginning with 0 and going up to infinity that's a some but not a geometric so a geometric sum has exactly this pattern it says you're taking some base number raising it to higher and higher powers and adding them all up okay one other thing to notice about this is that the stum starts at zero you may have forgotten something that I know you guys all learned at one point you may have forgotten the fact that any number raised to the power 0 is not 0 it's one okay so this one right here is your Delta to the zero power so this is I equals 0 i equals 1 equals to i equals 3 we go on and on on and on and we never stop these are alternative ways of writing and talking about geometric sums so why am I making a big deal of it right now we've got one right here okay the very thing that we're trying to evaluate that we're trying to compare to something else look I can factor out the three okay I'm getting the three in every round and let me keep using blue there am I getting I'm getting three times one plus Delta plus delta squared plus delta cubed and so on okay so this is the number three times this geometric sum so that's one reason why I'm be laboring it the other reason why I am laboring it is that we actually have a formula for this infinite sum under some circumstances and the circumstances are good ones for us so this geometric sum equals 1 over 1 minus Delta whenever Delta is between 0 and 1 well how lucky can we get those are exactly the cases we care about here exactly the set of cases that make sense for us when Delta is a number between 0 and 1 that is the case where this is this fact about the geometric sum can help us so how can this be how can this be that we're adding up an infinite number of things but the sum is still going to be a finite number the way it works out is that when Delta is between 0 and 1 and we're raising Delta to higher and higher powers these powers of Delta are getting smaller and smaller okay so yes we're adding an infinite number of things but each successive term in the sum is smaller and smaller and sometimes when that happens the sum what you hear in a math course converges okay when the geometric series converges this is what it's equal to and the conditions under which it converges are precisely the ones that we care about Delta being between 0 or 1 okay when I'm writing this on the board and saying that it's a fact what I'm kind of telling you guys is I'm not like proving this to you okay I'm trying to give you some sense of why it might not be true um depending on your kind of the level of the calculus course that you got most of you guys have taken you might have proved it in that class if you take a real analysis class you'll prove all sorts of things like this so it is logically valid poli-sci 30 we're not doing the proof we're just accepting that it is true and this is going to be very helpful to us last thing I will say about this is I think by the time you're done with this week's homework most of you guys are going to have it committed to memory that's had some people remember things better than others every year goes by I remember things worse than I used to so I'm sympathetic to not being good at remembering things so if you need to use this on the exam i will put the formula on the exam okay if that's a hard thing for you to stick in your memory that's okay it'll be on the what will not be on the exam is detailed instructions about how to use it that's what you guys are responsible for for internalizing okay alright so now we've got this fact and now we can go back to think what I'm going to do is same thing I just SAT i'm just going to copy the fact over here the sum is i goes from 0 to infinity adults the I power is 1 over 1 minus Delta whenever Delta is between 0 & 1 okay we can use this because now we can apply it right here I'm going to I it to the present value of playing grim trigger when your opponent is playing grim trigger hi so this is what we need to know I'm going to erase the reason why we need to know like we'll come back to that and we're just going over here to say that the present value to player a of playing grim trigger is 3 i'm looking right over here that pre pay off i'm going to get in each round x this geometric sum this geometric sum that tells me how to discount rewards that are happening further and further in the future ok so this geometric sum is doing a lot the first period no discounting in the next period i discount by one after that does count by 2 etc all the way off toward this infinite horizon that the geometric sum formula tells me that is 3 x 1 over 1 minus delta it's just a fact very nice tact very convenient packed for us in trying to figure out whether grim trigger is an equilibrium right in this case the present value of always defect didn't require a geometric sum it's just five but as the person's back pointed out there could be cases and we will see cases this week where we have to apply the geometric sum formula on both sides but now here we have two things that we can very nicely very easily compare to each other so I'm going to leave those tables up anybody want to ask anything about this before I raised it's kind of a lot of wind up I hope you're seeing that the application of that formula is really easy okay you see the geometric some pattern you just plug in 1 over 1 minus Delta so great so getting back to what we want to know the present value of grim trigger is at least as good as the present value of always the fact what does that mean it means that 3 over 1 minus delta is greater than or equal to 5 okay so what we're going to do now is something we've done all along we're going to solve this inequality for a threshold value of Delta okay we did this when we looked at variables in sequential games when we look for the minimum or maximum probability that would make a certain choice the optimal one we're going to do the same thing here but we're going to be looking for some threshold value of the discount factor how patient do we have to be to make grim trigger an equilibrium that's what the question is going to come down to okay so we're going to multiply both sides by 1 over 1 minus Delta when we're working with inequalities we always have to remember the business about when we multiply by a negative number switching it but that's not applying here because we know that the deltas between 0 and 1 emphasizing that because if you forget the Delta is between 0 and Y if you just kind of focus in on the symbols things will get very hard if you remember that dell's is a discount factor it's like a probability you only have to think about values greater than 0 less than 1 things are easier ok so we're multiplying by a positive number we get 3 greater than or equal to 5 11 is Delta okay multiplying through by the five here let's just do it at 5-5 Delta I'll bring this over to this side this over to that five delta greater than or equal to two what is the same delta greater than or equal to 2 fils if arm player a and a the promise of a dollar tomorrow a dollar in the next round is worth at least forty cents to me then I would rather play grim trigger than always defect a dollar tomorrow being worth forty cents today that's not a real stringent requirement I don't have to be extremely super duper patient to UM to say yeah a dollar tomorrow is worth at least forty cents today okay so this is the two fifth is the minimum value of the discount factor for which player a would rather play grim trigger then always the fact okay so that's good news that that is very good news I think before I am talking about asymmetric payoffs I want to say a little bit more about what the the Nash equilibrium calculation is here okay I haven't quite gone as far to say that when delta is greater than or equal to two fifths that mutual grim trigger is an equilibrium I haven't quite said that but it is in fact true and I've almost shown you everything you need okay what what I've shown strictly speaking is that as long as Delta is at least forty percent okay as long as the penalty you pay for waiting is at least um or is not the penalty I'm saying it's a long way the penalty is not worse than giving you forty percent of the future value today as long as that's true if your partner's playing grim trigger you will want to deviate to this alternative strategy in order for it to be a Nash equilibrium though we have to say you don't want to deviate to any alternative strategy and we haven't quite gone that far yet so let's go that far right now can any of you guys think of another different strategy another thing that player a could do different than all is the fact that would be better for her it would be a more tempting way to deviate from grim trigger okay that would give her a higher stream of payoffs than what she gets from always defect yes the wing yeah okay so one thing Elaine is thinking of is cooperate for a while and then to fact okay so one thing that wouldn't make sense so I'm just going to elaborate a little bit here we'd be cooperate for a while and defect in the last period you can't do that in this game because you never know when the last period is ok but one thing you could do is we're going to do I ok we're going to do it on this board we're going to do the same table with that idea so another way that you could deviate from grim trigger would be to cooperate for a while to lull your partner into thinking that you're really a good guy and then defect then get that sucker payoff ok so this is going to be player a's payoffs from and what I'm just I think to give the flavor of it i'm going to say cooperate for two rounds then to fact ok as before as throughout this we're holding these behavior constant like we've always done when we think about Nash equilibrium ok so be plays grim trigger and we have round one two three four we have a action these action is pay off ok so bees playing grim trigger I'm playing and thinking of other things I could do wondering if any of these alternative strategies would give me a higher stream of payoffs okay so i can cooperate for two rounds then defect so this strategy pretty easy to map what the strategy is telling me to do into these actions it says cooperate here cooperate here then defect and um it doesn't actually say what to do after that I'm going to say defect forever we could think about the alternative of then going back to something else what's these action in the first round cooperate second round cooperate third round cooperate fourth round the fact that did it play ray we're history ok I'm going to defect against you again and defect against you forever into the future so haze pay up here three good three again five oh yeah we like that okay here 0000 okay so what this is giving us this is actually this is a good example because it shows yet another form that the present value calculations can take and they're all applying the same basic idea here we had nonzero payoffs in every period so we had to use the geometric sum to calculate our present value here we had a payoff that wasn't zero in the very first round and then zeroes after that we didn't have to worry about discounting there's no future payoffs here here we have its blues my present value color today so let me get blue the present value to a of we're going to call this what ccd cooperate cooperate to fact that's just a name i'm giving to this alternate strategy is going to be I get three in this round I get that free in round two okay but I got a discount because we don't really know if I'll get to round two we don't know if I get to round 3 either but if I do that's when I pounce ok take her by surprise get the five and then zero thereafter okay okay so um that's not obviously a worse strategy than this one indeed it might actually be higher might not so let's look what we're going to do now is we're going to compare this way of deviating from grim trigger too grim trigger okay theoretically what we would have to do is compare every single thing we could think of but we're going to see is that the different things to do are going to fall into categories okay so let's finish with that is the present value to a ccd actually let me just write it exactly the same way I did before before I put grim trigger first so let me do it that way present value to a grim trigger greater than or equal to the present value to a of this cooperate cooperate to fact strategy if that's not true then we know grim trigger is not in equilibrium we know the present value of grim trigger this is 3 over 1 minus Delta and the question is is that greater than or equal to 2 3 plus Delta 3 plus delta squared over 5 ok watch how I do this watch how I first correct my inequality what I'm looking at here is something that looks like the start of a geometric sum right is the del T equals 0 del T equals 1 term and then if not a geometric sum anymore ok so here I'm going to use that geometric sum formula in Reverse okay it seems like that this formula would only be convenient going this way but sometimes it's convenient to expand it out the other way okay another way to think about it is suppose I hadn't done this step suppose I just have the present value to a written out like this 3 plus 3 Delta plus 3 delta squared y delta cube etc the question is is that greater than 3 plus delta 3 delta squared 5 ok no dot dot dot here ok this is this is the finite sum because we start getting zeroes yeah why did I do it this way I did it this way because the payoffs in the first two rounds from this different strategy this cooperate cooperate defects strategy in the first two rounds it gives me the same payoffs as grim trigger okay right in these first two rounds another way to think about it is you couldn't tell on the first two rounds whether player player a was playing grim trigger or playing this alternate strategy the payoffs are exactly the same so let's just get rid of them they're on both sides of the inequality ok so then our question is whether 3 delta square plus delta cube going on and on ever whether that is greater than Delta squared five okay so that's looking a little bit trickier that doesn't exactly fit our geometric some pattern so let's do a little work on it to get to something that may or may not fit this okay one thing I'm going to do is I'm going to notice that once I've canceled out the terms that appear on both sides every term here every term in this infinite sum and this delta squared five here has at least a delta squared so i'm going to factor that out okay that's going to write it here factoring out delta squared leaves me with three plus three Delta let me just to keep the pattern clear here we could have written three Delta to the fourth here it doesn't matter how many terms I right before I start doing the ellipses this da always means that there's an infinite number of terms and once i factored out delta squared here's my 3 3 delta plus 3 delta squared now I'm going to put in my dot dot dot that has to be greater than or equal to 5 okay guys are thinking where's she going with this maybe you see at this point where I'm gone with it now I do have my geometric sum okay and this is something that anytime you're working with geometric sum formula sometimes it's really easy sometimes as easy as it was in our first comparison where the formula for the geometric sum just pops right out at you okay it's exactly there other times you have to do a little bit of work okay and the two things that I did here will almost always get you what you need subtracting off common terms and factoring out common terms yes this one here this one is a delta squared so what i'm doing here is i'm factoring out the delta squared so factoring out this delta squared leaves me just the three yeah it did line up very well but this is my delta to the fourth power one with two of them factored out and similarly over here okay so now i have over here 31 plus deltal that when is that greater than or equal to 5 well that's exactly what we did before right this is going to be true exactly when the present value of grim trigger is better than the present value of always defect and if you look at the pattern of payoffs and actions you'll see why okay once we get to this point in the game the game is an infinite horizon so we get to this point it's just like the the stream of payoffs are just like on what would be starting at the beginning the only thing that's different is the history once we get to this point in the game it's just like grim trigger okay so waiting for two rounds to de fact will be a good idea under exactly the same conditions that defecting at the outset would be okay way to think about it is that if it's worth your while to defect in round 3 after cooperating two times it would have been worth your while to defect right here okay because from this point forward the games are exactly the same okay so a generalization and actually the generalization that Elaine used when she brought up this alternative is that the condition for defecting after some number of rounds is exactly the same as the condition for defecting around one okay so that's big we've ruled out a whole lot of alternative strategies here okay and I hope this makes intuitive sense right if after having played the game for a while you think it's worthwhile to get this one-time gain okay to take advantage of your partner in the short run knowing that you're going to be punished in the future if that's worth your while in round 3 it would have been worth your while in round 12 okay so yes sometimes the strategy will be a better one than grim trigger but it's under exactly the same circumstances that defecting right from the beginning would have done as well so there's one other thing that you could do differently and grim trigger that one other form of a strategy that um you might think would be tempting for players will talk about that on Thursday okay but we're almost done with that

Maurice Vega

Leave a Reply

Your email address will not be published. Required fields are marked *

Post comment