[CSGibson]

For those that track systemic successes and failures, how do you track it? What things do you compare, and what do you take into consideration?

I just started a project where I was tracking the success/failure of all of our partnership's 3 and 4 level preempts. I noted the actual imp result, the par imp result, the difference between the two, whether the preempt was standard or not standard, and in what way, who the opponents were, the date, and whether there was a reasonable opportunity for further gain which we did not exploit from our preempt (ie, we didn't double/sac when it was clearly indicated). Edit: I also included the vulnerability

Against good opponents, it seems our 3/4 level preempts gained an average of a little more than 2 imps vs par, and that the biggest gains were non-standard preempts (preempts with an outside control, or with an unexpected number of cards in the suits) - normal preempts made up 3 of the 10 results in the time tracked, and had a net gain/loss of 0 against par. The biggest loss came from a preempt where we had an ace and a void outside of our suit - that was too much offensive potential, we did not get to game on a hand where slam makes. Hands with 1 flaw turned out to be huge winners, but again, the sample size was relatively small.

[FrancesHinden]

CSGibson, on 2012-June-28, 11:04, said:

For those that track systemic successes and failures, how do you track it? What things do you compare, and what do you take into consideration?

I just started a project where I was tracking the success/failure of all of our partnership's 3 and 4 level preempts. I noted the actual imp result, the par imp result, the difference between the two, whether the preempt was standard or not standard, and in what way, who the opponents were, the date, and whether there was a reasonable opportunity for further gain which we did not exploit from our preempt (ie, we didn't double/sac when it was clearly indicated).

You also need to include whether you missed out on an opportunity to gain because of your style, or you gained because of your style.

For example, I play a very random and aggressive style of pre-empt in certain positions and vulnerabilities. This means I pre-empt more often, but it also means that partner does not always save (or raise) when it is right. So I might gain on hands where a disciplined pre-emptor has to pass (and that is worth tracking) but I lose on hands where a disciplined pre-emptor can save accurately.

Also, of course, it's really hard to say if you should have saved or not. It's not uncommon for a technically wrong sacrifice (800 against game, or against a non-making game) actually wins because they bid on; or at least breaks even because they bid on and make the same 11 tricks they had al along.

And what about hands where you put them under pressure and they got it right? If you pre-empt and oppo guess well to bid a making slam you haven't directly gained from the system, but you've made their life harder which is worth something in later boards that set.

[JLOGIC]

And of course the biggest downside to preempting is helping them play the hand. It will never show up in comparison vs par if they played double dummy to make a game after you preempted, that would be them achieving par. Of course had you not preempted, they wouldn't have made it, so you really lost a game swing by preempting but that will not show up the way you are analyzing if I understand it.

[CSGibson]

JLOGIC, on 2012-June-28, 13:55, said:

And of course the biggest downside to preempting is helping them play the hand. It will never show up in comparison vs par if they played double dummy to make a game after you preempted, that would be them achieving par. Of course had you not preempted, they wouldn't have made it, so you really lost a game swing by preempting but that will not show up the way you are analyzing if I understand it.

By par I don't mean double dummy par, but by likely result - for example, there are two lucky slam hands that I didn't include as par because it was unrealistic to get there or expect to make (though if our preempt pushed them there and they made it, I would obviously include that as a loss). So I try to include play decisions as part of par, which makes calculating par on some hands impractical. There are some hands where there are too many permutations in both play decisions and bidding decisions to calculate one par, on those I tend to give a range of likely results as well as the actual result. This is obviously more time intensive than the simplistic model that I think you were assuming, but also more valuable in evaluating the effectiveness of our preempting decisions.

[Mbodell]

I think it is easier to track pairs than teams, at least in the ACBL style where teams are nearly always shuffle and deal, but you can track the contract, lead, field strength (power rankings or subjective), and MP result as well as the hand for each and every hand you play. You can use that to do a lot of analysis like we average 56% when I'm dummy with partner A and 60% when I'm dummy with partner B or we score 50% when I'm on lead defending but 55% when partner is on lead defending, maybe I should work on my leads. You can also notice other trends like we score better on hands that are slams and games than part scores or vice-versa. You can also then chop up the hands by what happened in the auction. This was an auction we opened 1nt. This was an auction where we had a strong club sequence. This was an auction where we preempted. If you track the hands you can also go back and do things like this was a hand where some folks would bid something (weak 2♦ say) but we had to pass because we didn't have it in system and this is what happened.

It is a fair bit of work, even without tracking the hands and analyzing the auctions that didn't happen. I tracked a few thousand hands at one point in time (just for partner, play, field strength, contract, MP result) but it became cumbersome, especially coming back from a national when I hadn't tracked anything while away from my computer.

[TimG]

CSGibson, on 2012-June-28, 11:04, said:

For those that track systemic successes and failures, how do you track it? What things do you compare, and what do you take into consideration?

You'd also need to track the instances where you chose not to use your systemic device. For instance, if you want to track your preempts, you also need to track those borderline hands where you choose not to preempt. Or, if you don't preempt with a [void, outside controls, etc.] track the results for whatever you did open with these hands.

[Cthulhu D]

TimG, on 2012-June-29, 09:21, said:

You'd also need to track the instances where you chose not to use your systemic device. For instance, if you want to track your preempts, you also need to track those borderline hands where you choose not to preempt. Or, if you don't preempt with a [void, outside controls, etc.] track the results for whatever you did open with these hands.

I suspect the solution heavily depends on your skill level and the homogeneity of the field you are playing in. While I agree that the comprehensive approach is better, separating out systemic factors from the general noise of factors like opponent competence is probably practically impossible.

So for example, I track abnormal results and also where we make a highly unusual pre-empt (I define 'highly unusal' as 'only people in the room to open the hand'). Conversely, I've never bothered really trying to evaluate the effect of the multi 2D if our opponents got to the normal spot. It's possible that the 2D might have made it easier than a natural weak 2S for example, but determining that for sure is impossible.

[TimG]

Cthulhu D, on 2012-June-29, 09:25, said:

Conversely, I've never bothered really trying to evaluate the effect of the multi 2D if our opponents got to the normal spot. It's possible that the 2D might have made it easier than a natural weak 2S for example, but determining that for sure is impossible.

You will need to track the cost of not being able to open 2D with whatever multi replaces to get a true value of multi, not just the results when multi 2D is opened.

[Cthulhu D]

Sure, but the only way to determine that is if you got an abnormal result. In my case though I can still preempt at the two level in diamonds, but I generally agree you must consider other uses for the bids. This is why multiway bids (e.g. 2C: strong or weak 2 in diamonds) are the hardest to examine.

[wclass___]

Is there any software for this?

OP asked "How", but I don't see any answer here.

[32519]

CSGibson, on 2012-June-28, 11:04, said:

For those that track systemic successes and failures, how do you track it? What things do you compare, and what do you take into consideration?

My opinion on this topic is that every single bridge player, regardless of skill level, subconsciously tracks systemic successes and failures. Why do I say that? Easy: partnerships are forever making system changes, tweaks, experimenting with new gadgets/systems etc. The stuff that works is retained. The stuff that doesn’t work gets discarded. These systemic changes tend to escalate in number as the partnership steadily gets better. The reasons are straight forward –
1. Your bidding system needs to be optimal to get your side to the best contract.
2. Your defensive agreements also need to be optimal to either defeat the contract or to prevent overtricks.

My guess is that it is only once you reach international level that the systemic changes drop off substantially. By then you and partner knows what works best for you.

[MrAce]

32519, on 2012-July-09, 00:18, said:

My opinion on this topic is that every single bridge player, regardless of skill level, subconsciously tracks systemic successes and failures. Why do I say that? Easy: partnerships are forever making system changes, tweaks, experimenting with new gadgets/systems etc. The stuff that works is retained. The stuff that doesn’t work gets discarded. These systemic changes tend to escalate in number as the partnership steadily gets better.

Are you sure they make system changes, tweaks, experiment new gadgets because they subconciously track their systemic successes/failures? Or is it because it is easier to make changes to system and think it is the system/style/convention that sucks instead of improving their judgement/logic/card play ?

Imo the majority falls into 2nd category.

[Antrax]

I agree we track things subconsciously, but I don't think we necessarily do it effectively. A devastating loss or an overwhelming success leave a much stronger impressions than more average results. So if something consistently pays off small but occasionally you go for a number, you are more likely to think it bad, regardless of the actual EV.
That's why poker players use statistical tools (or just Excel) instead of relying on their subconscious.

[Zelandakh]

Have you ever played in a face-to-face club 32519? There are many pairs who have played exactly the same system completely rubbish for 20 years or more. It is how they were taught and they see no reason to change it. The same applies to defensive carding too - how many players do you know who lead "top of nothing" in partner's suit? There are plenty more examples to be seen here. The players who mess around with their systems most tend to be better players, advanced and up, imho.

[gnasher]

wclass___, on 2012-June-30, 17:37, said:

Is there any software for this?

I use Excel. You don't need anything more than a few columns for categorisation, the date, event, hand and result. Any analysis can be done with filters and pivot tables.

Quote

OP asked "How", but I don't see any answer here.

He doesn't seem to be complaining.

[32519]

Zelandakh, on 2012-July-09, 03:47, said:

Have you ever played in a face-to-face club 32519? There are many pairs who have played exactly the same system completely rubbish for 20 years or more. It is how they were taught and they see no reason to change it. The same applies to defensive carding too - how many players do you know who lead "top of nothing" in partner's suit? There are plenty more examples to be seen here. The players who mess around with their systems most tend to be better players, advanced and up, imho.

You certainly have a valid point here although not 100% in line with my own personal experiences.

I have been a member of the F2F club in my home town for 20 years now. Over those 20 years, members have come and gone. Apart from myself, there are only 5 other members who are still members of the club who were there when I joined. One of them is my current F2F partner. The remaining 4 are by no means going to set the world on fire, either with their standard of play, or with the system they play.

Here I am forced to agree with you. Over the 20 years that I have known them they have made limited changes to the system they were taught. The key word here is limited. On the rare occasions they ended up in new partnerships, the new partner “forced” them into trying something different. When the “something different” proved to be better than what was previously used, it became the new standard.

Amongst the players who have come and gone, there have been a fair amount good players and some really bad ones (hopeless would probably be a better description). Yet even amongst the hopeless players I would often find them (usually in the bar) discussing some of the hands played trying to figure out between themselves a better way of doing things, usually confined to the bidding. They never manually tracked the changes made, whether on paper or on computer, but changes certainly were made.

Based on my own experiences I still believe that subconsciously we all track system successes and failures.

[Cthulhu D]

32519, on 2012-July-09, 04:59, said:

Based on my own experiences I still believe that subconsciously we all track system successes and failures.

It's obvious from every field of human enterprise that the evaluations based on 'well, I reckon' without a solid statistical or theoretical underpinning are wrong. Attempting to assess your systematic failures on this basis is ridiculous. Pick any random sport to see this in action, though the statisticians are taking over now.

[32519]

Cthulhu D, on 2012-July-09, 07:53, said:

It's obvious from every field of human enterprise that the evaluations based on 'well, I reckon' without a solid statistical or theoretical underpinning are wrong. Attempting to assess your systematic failures on this basis is ridiculous. Pick any random sport to see this in action, though the statisticians are taking over now.

Let me give you another example from personal experience and what I keep witnessing amongst the players in my home town.

1. The better or stronger the players are, any changes made to system agreements are –
.a. First verbally discussed.
.b. Then agreed upon.
.c. Then written down on a piece of paper (initially on the back of a personal travelling score sheet).
.d. Before finally updating the system change on a computer where the full system agreements are kept.
.e. A printout of the full new agreement is made for both partners.
2. Conversely the worse or the weaker the players are, any changes made to system agreements are –
.a. Verbally discussed.
.b. Agreed upon.
.c. No written record is kept of those new agreements resulting in:
....i. Either or both partners forgetting what they had verbally agreed upon.
...ii. Fights and arguments at the table as to who it is that screwed up.
..iii. Inconsistency from week to week regarding those agreements (what applied last week no longer applies this week).
...iv. The fights and arguments upsetting the partnership to the extent that their results for the evening are worse than they normally are (as the score keeper, on numerous occasions I have seen their results drop from the low 40s to the high 20s because they are now angry with each other).

The better a player is, whether it be golf, tennis, athletics etc they inevitably have a top quality coach to identify any weakness or whatever in their game. The really bad bridge players I have known were unable to fathom why they were last week in and week out.

[Antrax]

At least this observation explains a lot about your posting style.