Ben on BBO - Feedback thread Discussions about Ben models trained on BBO data
#41
Posted 2024-January-12, 02:49
Raising 5d to 6d after 5d was doubled, presumably for penalty? The lack of double on the final contract is also not good but ok i suppose i could have doubled myself.
#42
Posted 2024-January-12, 09:46
helene_t, on 2024-January-12, 02:49, said:
Raising 5d to 6d after 5d was doubled, presumably for penalty? The lack of double on the final contract is also not good but ok i suppose i could have doubled myself.
same hand; different bidding: http://tinyurl.com/ytbtf4pw
#43
Posted 2024-January-12, 10:08
kgr, on 2024-January-12, 09:46, said:
thanks, i am aware of this silly behavior and plan to fix it.
the reason why it keeps bidding more of the same suit after being doubled is because it naively hopes that this time it will get away without a double )
afterall, when it corrects the 5Dx to 6D, the 6D isn't doubled yet and it has some equity on it being passed out.
#44
Posted 2024-January-12, 10:31
- How did the robot in N decide to play ♦9 on trick 1 in http://tinyurl.com/yv64cb7a
- You said somewhere that Robot play is based on Human play. Does it mean that Ben has no rule to play from 2 equal card (eg play K when following with AK32) but looks at what Humans tend to do? Or do you have these kind of rules.
- Is the Claim button part of the Robot functionality (that could be improved). And can you also let the Robot Claim when all tricks are for him (that is nicer to use it)?
- Are any signals used now? (parity, Lead 4th best, ..)?
- signaling would also improve the play, and preferably allow human to define the signaling. And also allow human to set the bidding methos and conventions....Possibly I give you to much work now
- Is the coding open source?
- How strict is Ben on the system? e.g. open 1NT 15-17 balanced: is 2336 ok if it looks like NT; and how strict does it believe humans; will it know that it is not impossible that one can op 1NT with e.g. 12HCP?
- why lead C8 http://tinyurl.com/ysp5hwrb . Is there a rule for it?
This post has been edited by kgr: 2024-January-12, 10:43
#45
Posted 2024-January-13, 01:05
kgr, on 2024-January-12, 10:31, said:
- How did the robot in N decide to play ♦9 on trick 1 in http://tinyurl.com/yv64cb7a
- You said somewhere that Robot play is based on Human play. Does it mean that Ben has no rule to play from 2 equal card (eg play K when following with AK32) but looks at what Humans tend to do? Or do you have these kind of rules.
- Is the Claim button part of the Robot functionality (that could be improved). And can you also let the Robot Claim when all tricks are for him (that is nicer to use it)?
- Are any signals used now? (parity, Lead 4th best, ..)?
- signaling would also improve the play, and preferably allow human to define the signaling. And also allow human to set the bidding methos and conventions....Possibly I give you to much work now
- Is the coding open source?
- How strict is Ben on the system? e.g. open 1NT 15-17 balanced: is 2336 ok if it looks like NT; and how strict does it believe humans; will it know that it is not impossible that one can op 1NT with e.g. 12HCP?
- why lead C8 http://tinyurl.com/ysp5hwrb . Is there a rule for it?
kgr, thank you for your interest. i will try to answer your questions.
rules: there are no rules at all for any situation. neither bidding or card play.
we only program things like "you have to follow suit", "you can't double your partner", but even those rules are not strictly needed, we added them for just-in-case.
although there are no rules, it learned to lead top of sequence, high from doubleton, play lower of touching honors, etc. because that kind of play often occurs in the training data.
(in principle i am not against adding rules, in fact i will probably add some in the future)
why it played D9?
hard to say exactly. i think the bot didn't feel the urgency to play the K as it doesn't see losing a trick that way. probably the bot thinks that playing high or low is about equally good and then chooses the 9 because finessing dummy's T is salient in the data.
why it leads C8?
we didn't program any lead convention. it picked up from the data that people often lead top of nothing.
claiming: gib and ben use the same claiming engine. indeed i am working on extending it and hope that the things you wrote about claiming will be available soon.
signalling: as said above, only what comes implicitly from the data, which means not much signalling is going on. i plan to add some more structured signalling in the future.
code: yes, the code is open source here https://github.com/lorserker/ben
at BBO we added a few things like human play, but it's essentially the same bot.
system: ben doesn't have the concept of system or convention defined explicitly. it just tries to immitate the bidding that we show during training. this means that it's not programmed that 1NT is 15-17 for example. nevertheless it bids 1NT reliably, and I have also seen it upgrade some 14 counts or have 6 card minor. subjectively, i would say that ben takes bids a bit loosely (which is not always good)
i hope this helps. let me know if you have any more questions.
#46
Posted 2024-January-13, 02:47
lorserker, on 2024-January-13, 01:05, said:
rules: there are no rules at all for any situation. neither bidding or card play.
.., it learned to lead top of sequence, high from doubleton, play lower of touching honors, etc. because that kind of play often occurs in the training data.
(in principle i am not against adding rules, in fact i will probably add some in the future)
..
Thanks, very interesting. And the good thing compared to Gib is that there is worked on and it will evolute.
(I play sometimes with Argine on another site. And that is in my opinion much better than Gib.)
#48
Posted 2024-January-14, 20:05
#49
Posted 2024-January-15, 01:35
Going to 5♥ with this hand without consulting the partner appears to be rather bad bridge.
http://tinyurl.com/yvdmgd3f
Since 1N opener may have a hand where an ace is missing, committing to the grand without invoking ace asking appears to be an avoidable error.
#50
Posted 2024-January-15, 11:35
fuzzyquack, on 2024-January-15, 01:35, said:
Going to 5♥ with this hand without consulting the partner appears to be rather bad bridge.
I think it's 4♥ that was bad bridge.
#52
Posted 2024-January-15, 13:25
lorserker, on 2024-January-09, 05:42, said:
Learning the bidding from gib is not, in principle, a cap on the potential. True that starting from something better will probably result in something better.
According to some experiments I did, when judging the bidding alone*, ben is better than gib basic by almost 1 imp/board and is also slighty better than gib advanced. This results I have confirmed also by training ben to bid like argine and then comparing it with argine.
*the way i am comparing only the bidding is by playing a match between a team of bens and a team of gibs.
after the bidding, the opening lead is made single-dummy. the rest of the card play is skipped and the result is determined by the double dummy score of the hand. (we do this because different bots have different ability in card play and we want to isolate the bidding).
if you have an idea of a better setup to test only the bidding, i am happy to listen.
Would it be worth considering generating a set of 10-20 hands based on the NS hands (and the EW bidding)? That's what I did when I was trying to figure out optimum contracts. Maybe it doesn't matter if you're playing thousands of hands in the simulation however. Might be better for an indepth case study analysis.
#54
Posted 2024-January-15, 14:00
http://tinyurl.com/ypdz59wb
#55
Posted 2024-January-16, 03:12
Lots of stuff in one hand. Putting aside further peccadillos, Ben appears to follow the GIB lead and overcall a preempt with a preempt. True, it was a potentially winning decision for this particular board, but in general it is bad bridge.
#57
Posted 2024-January-16, 03:56
Ben shows 0 or 3 keycards when in fact it has two.
Maybe responses to keycards ought just to be coded, rather than left to the neural network? I realize I am going down a rabbit hole now....
The failure to lead ♦A is also not great, when declarer and dummy have both shown some length in diamonds it should be clear to lead ♦A against a grand slam. As it happened, the contract makes on any other lead.
#58
Posted 2024-January-16, 04:01
0 carbon, on 2024-January-14, 20:05, said:
Using Google is likely to lead to better results even if the search length is changed.
For example, you can enter "ben site:www.bridgebase.com/forums/" and it returns forum results.
#59
Posted 2024-January-16, 08:35
helene_t, on 2024-January-16, 03:56, said:
Ben shows 0 or 3 keycards when in fact it has two.
Maybe responses to keycards ought just to be coded, rather than left to the neural network? I realize I am going down a rabbit hole now....
The failure to lead ♦A is also not great, when declarer and dummy have both shown some length in diamonds it should be clear to lead ♦A against a grand slam. As it happened, the contract makes on any other lead.
yes, coding blackwood is definitely a better idea
#60
Posted 2024-January-16, 08:35
fuzzyquack, on 2024-January-16, 03:12, said:
Lots of stuff in one hand. Putting aside further peccadillos, Ben appears to follow the GIB lead and overcall a preempt with a preempt. True, it was a potentially winning decision for this particular board, but in general it is bad bridge.
wow! even i am shocked by this bidding