Colluding makers with very large bond values using an out of sync bitcoin node?

[PyGryhapoRbryDer]

A bit over 48 hours ago I started to get warnings that makers were providing already spent utxo. Out of 10 (randomly chosen) makers there are now often 6 or 7 makers with such warnings. I looked at the log and to me it seems that the utxos that are being provided have been spent a few blocks after they were generated, and they were generated tens of hours ago. The most logical explanation I can come up with is that they are all from the same entity using an out of sync core node. What am I missing?

The following output is from 2024-07-04 03:13 UTC (previous block was 850,618):
Makers responded with: {'J55fUQXuJoyF88xZ': [['61e5070fc1e5611c6f3d0071a2d8c4784318cc6839094dad67486a8900734f77:12'], '02134c7bcd223532d104732c0ce72cb1db3b95d70c4faf652cc1d00613e50bb957', 'bc1qrluxpftpq58lr2rgyh2fznp4s8yg5cdldcj3x7', 'bc1q2utmcllq7ulwa0u238l5ujq74xcgs9v33nzs9k', 'MEUCIQDGbG8nFtVatg1eolqrtdv+Hwz5u4k10zk5bqceLhx0ZQIgBx//KugROvjAC79H14lisVwRvjQ/qqKnVkksXKk779E=', 'cd47cf9e21403cfaf531ebf2e8919c669e4fa2977d6111427430eb250dc60062'], 'J53219brWJ75xnsy': [['228a62916d881c6422b800e3ba48e1f72485d422aa70ff4654d20d4b007ad0d8:1'], '029ff570add043cbb2cb68d63abe5d2b8405e46462a3fb16467d36324ce53aa314', 'bc1qa5tpxeypjyp6dghydc3xc02yuf0c5cs38fkas8', 'bc1q8yf67ys3py5dy2eslqxrl8rrk0yxjzj2gnnzrd', 'MEUCIQD33PGXiF2pKuPbNBVoQ/xd5xvXxzWtq9f0G865cb4hPAIgeuCDnkCu6baw9YxlW69AbOW7C85K9GpzfNqgCnfiw6I=', '8328a7e26af10ac0345a095ce8b2ee9c12def9295fde3004ae4baca68c169e79'], 'J5dDKCGEn4VRVKsO': [['61e5070fc1e5611c6f3d0071a2d8c4784318cc6839094dad67486a8900734f77:0'], '032e9704a4e13e9a9d45650e4d08c2bea3375fed110df4ffd3608afc420bc7145e', 'bc1q2ku7m5e7qga9yty5ejyp5zws2ru872e7rez3l3', 'bc1q7887de4fct7uq356eqpy0mmr5cpvc537tfzlm3', 'MEQCIEpQAIkQgi3Nh7N8soX5WNmQ8MT1KYeNKjfkvRKMa6IPAiALhxXwP0295up1m76RlLkET+Itk6hskhKUjh8dt8mJAQ==', 'c2df6a5f0f1620dec0a49ead7dc69d06ef22524a2a057186fd4d4b3f5b900929'], 'J5B2DBBt5vMzxDz8': [['61e5070fc1e5611c6f3d0071a2d8c4784318cc6839094dad67486a8900734f77:9'], '030e7861b8bfd7671c17a2414fdf709e325e95a3597bfd173696125b594fd7e0bf', 'bc1qhk9c4qk4zp5mhd4udw29lygdcwwpwr3dwh3497', 'bc1qmv6dewg8sjvfp63z4pesuw9ypmnwtsstun783e', 'MEUCIQDyrA3+yeFRYbm5Q2a+y44hD0M1gVgIE+ZGRRwsT29RUwIgR3Iudm8n80/pbQ70hbX9+pgpMuNIAIx1/GW3wEGQGHM=', 'a30b518feff9ac48998e8f65834068fe2d331d8a4f312e0a6693c63eba602877'], 'J56Eb7gDSiUCUJQG': [['08fd00ce8d4c018968391b642c8cdb4a056e46bb930f6c57509887c994b939d1:3'], '023a46a19fed299d37a1c947ff99614319c0a1e179be5b74bec2b36a9ca4fe901b', 'bc1qkx6y2k7sgy22jfp2m42canx7rru2z6xmq5sju2', 'bc1q2e6gejz8ythennxqdxvnuwugplypvh2nyvw09v', 'MEUCIQCR7cPSv/zFxixDgiXDAaWtb8k2WSzHL6XFOaeT9ZepQgIgIX/Q1ogqhpZRX9OaEfEtXJgGGjeRFp9R7Ou+VTOmtUs=', 'fcf96fb6fe928579ea4e60af2781fc63adf5885b2407d747c1dd59ca9df4c961'], 'J53ciG7bUk69kcTj': [['35e33b7030a56614f93ac08d87fdfa2ef3bd756b88e47300abcd643e6eb6b768:4'], '0266bfaab5a627777580eeda1772ce1061bbd6dfa31f28119ea841844aa7a807e5', 'bc1qgh5wpfl8um0cjq9z9586xszzmeq2hq0l7unkv9', 'bc1qxg5vjpkul7uz7lmct5vk6hxk6krlzlxfcjyeqy', 'MEUCIQCVgNOgGo1tjkPPSeobCufuYm+XtHI+cnsdm8iC4MsfkQIgNyA5TaKACk0jX5KpDbkNjxbgdzeJSu+OrdZElyihfGA=', '5153ee0fe5e7faf07bcca61f510e5ea80159818c3ffca2a33f107067dfa83f32'], 'J5BuNxqvAduoz56w': [['fe1cc5ebc84d4ead4053dc9acdf4518fb046ed2239097a721268dfe5cc5b4c00:2'], '029010c7a625d40d3b7f4742cd7b6a3de3658fb1865b0f2dc450d8d70da7bcf1f7', 'bc1qku97eq3d5qvnc6wc56mu9uj2vxz70y9nag60hf', 'bc1q6kn2m672ghpyepkurll3cj03lsmnl9zg9h99kw', 'MEUCIQDvWIYt+8yBFD57ZNQIPiCBKDosSE3TogtlCuHXEw8dLwIgFiTQ0UCerkKOA1BmvWfn3yNh0fK3uQm/qUHr2M6KI6o=', '02867ce2f331799b0d5ad1ed96b5586fbf396e236f570ff7727950c960c42862'], 'J57K9EAN3tJZYpFL': [['99b4df6a3bb19aaf5f81f0c3c6121cd511870220bc47b934d774eff4f9198273:14'], '02e33dc24294f87f333f7d16fd29a4a519c7d1665e34db61cb1ffc6ce2f0554fa3', 'bc1q9wwqrsdtv2nves00lr7mt78x4nddqhdk87wcpp', 'bc1q85knnfk40y39euqutvn6wa0p8q884ps6huuqlq', 'MEQCIFQbh253vihEH7qSRUX89C2ZhoUDOy+dOd0nkKGJQ4dsAiAYE+2IVohMcp8EKEi0njADP5ge1S3AJ3f5JmPvAyOimw==', '41c66e241c6f23fbdfe0be74aa7ca694f7e56f6b1c92324adc7ef22ef09f2544'], 'J5us5S6NdVM1v6XO': [['9d3deca10afd0614e17c5f91b5d44f1cfd416ffa1a356cb660ff5facfbfa1532:1'], '02936762e3afbc1ff2da426b9e10ad270358c6086cc51b7e570ec783df3ba6b8da', 'bc1q3dqq66xky3nnj20ryt6n7da39d09mpwtjyzq4n', 'bc1qr0eetct2qm2t03p5qleptlfmd492nxkg2a9r9t', 'MEUCIQDOUdxSk80PTMkfI8/e6e0uNSwUvAbuJY5eTagWBFzNRQIgCMoRDICihYIJFKoqtprEyiimlSlHpakKonjPzG3qpqk=', '134d79270ab8cac3c6268f48d049f6ff4cfaa98af686ad0ee91f0b02d7270d76'], 'J58ZjuF4TR7uifRn': [['99b4df6a3bb19aaf5f81f0c3c6121cd511870220bc47b934d774eff4f9198273:12'], '028a8f74b0aac38947f570a3c9a669c04a177e7b408276a805b8abca009ee20da5', 'bc1qn27ecuh4ssd5pslt58h5xuaumu5sgqr8z747zl', 'bc1qlatcz2hddf9zm28hdxkeyzkeg2kksramdczvjl', 'MEUCIQCiU6nwPbLKLcDT/I0fhSH0c9VEWBIGkHS5gIsDSmi9owIgeGba4reMZmckcRwBHzbqQQ5BQC/ej8sDgkxuyS2g0cY=', 'a7540cee957c9bb252fdc1692251a71023f21fa9eb47259a10de61407276cb13']}
[WARNING] ERROR: outputs unconfirmed or already spent. utxo_data=[None]
[WARNING] Disregarding this counterparty.
[WARNING] ERROR: outputs unconfirmed or already spent. utxo_data=[None]
[WARNING] Disregarding this counterparty.
[WARNING] ERROR: outputs unconfirmed or already spent. utxo_data=[None]
[WARNING] Disregarding this counterparty.
[WARNING] ERROR: outputs unconfirmed or already spent. utxo_data=[None]
[WARNING] Disregarding this counterparty.
[WARNING] ERROR: outputs unconfirmed or already spent. utxo_data=[None]
[WARNING] Disregarding this counterparty.
[WARNING] ERROR: outputs unconfirmed or already spent. utxo_data=[None]
[WARNING] Disregarding this counterparty.
[WARNING] ERROR: outputs unconfirmed or already spent. utxo_data=[None]
[WARNING] Disregarding this counterparty.

Notes on when the provided utxos from makers were spent:
J55fUQXuJoyF88xZ: Block 850,362 (already spent)
J53219brWJ75xnsy: Block 850,620 (ok)
J5dDKCGEn4VRVKsO: Block 850,360 (already spent)
J5B2DBBt5vMzxDz8: Block 850,370 (already spent)
J56Eb7gDSiUCUJQG: Block 848,946 (already spent, much earlier than for other offers)
J53ciG7bUk69kcTj: Block 850,625 (ok)
J5BuNxqvAduoz56w: Block 850,535 (already spent)
J57K9EAN3tJZYpFL: Block 850,365 (already spent)
J5us5S6NdVM1v6XO: Block 850,687 (ok)
J58ZjuF4TR7uifRn: Block 850,362 (already spent)

[PyGryhapoRbryDer]

I looked at my logs and the problematic makers I listed above had abnormally high fidelity bond values it seems when the issue was going on:
({'counterparty': 'J55fUQXuJoyF88xZ', 'oid': 0, 'ordertype': 'sw0reloffer', 'minsize': 27300, 'maxsize': 43180865736, 'txfee': 0, 'cjfee': '0.000031', 'fidelity_bond_value': 304002224755.63916}, 124)
({'counterparty': 'J5dDKCGEn4VRVKsO', 'oid': 0, 'ordertype': 'sw0reloffer', 'minsize': 27300, 'maxsize': 34331147454, 'txfee': 0, 'cjfee': '0.000027', 'fidelity_bond_value': 74442570797.6315}, 108)
({'counterparty': 'J5B2DBBt5vMzxDz8', 'oid': 0, 'ordertype': 'sw0reloffer', 'minsize': 104818, 'maxsize': 10163338997, 'txfee': 0, 'cjfee': '0.000108', 'fidelity_bond_value': 153961079604.07205}, 431)
{'counterparty': 'J56Eb7gDSiUCUJQG', 'oid': 0, 'ordertype': 'sw0reloffer', 'minsize': 93647, 'maxsize': 9318700198, 'txfee': 0, 'cjfee': '0.000067', 'fidelity_bond_value': 97986384414.36588}
({'counterparty': 'J53ciG7bUk69kcTj', 'oid': 0, 'ordertype': 'sw0reloffer', 'minsize': 1053222, 'maxsize': 2808901417, 'txfee': 0, 'cjfee': '0.000051', 'fidelity_bond_value': 30326145734.23268}, 203)
({'counterparty': 'J5BuNxqvAduoz56w', 'oid': 0, 'ordertype': 'sw0reloffer', 'minsize': 108573, 'maxsize': 3497110298, 'txfee': 0, 'cjfee': '0.000107', 'fidelity_bond_value': 92604461961.9015}, 427)
({'counterparty': 'J57K9EAN3tJZYpFL', 'oid': 0, 'ordertype': 'sw0reloffer', 'minsize': 105070, 'maxsize': 19663019863, 'txfee': 0, 'cjfee': '0.000154', 'fidelity_bond_value': 138499120870.07065}, 614)
({'counterparty': 'J5us5S6NdVM1v6XO', 'oid': 0, 'ordertype': 'sw0reloffer', 'minsize': 99585, 'maxsize': 11073881548, 'txfee': 0, 'cjfee': '0.000021', 'fidelity_bond_value': 234706061844.2802}, 84)
({'counterparty': 'J58ZjuF4TR7uifRn', 'oid': 0, 'ordertype': 'sw0reloffer', 'minsize': 103137, 'maxsize': 8637958937, 'txfee': 0, 'cjfee': '0.000101', 'fidelity_bond_value': 114557102085.28133}, 403)

Because of these very high values compared to other makers, the problematic makers that were using a (common?) out of sync node kept being selected for every transaction. Currently it seems that all these makers no longer have fidelity bonds. I think what happened with these bond values should be looked at if possible? Also is it possible to blacklist makers when they appear to be colluding and attacking the integrity of the service?

I think something that could help reducing the impact of such an issue would be to cap the weight of a given maker beyond the fidelity bond value calculation. For instance, if x makers are to be selected for a given transaction, I think the probability of any single maker of the orderbook to be selected should be significantly less than 1/x for each draw. This would prevent multiple transactions in a row to essentially contain the same makers when the bond values are completely uneven. Also when makers provide utxos that were already spent more than x minutes before the orderbook was queried, they should get excluded temporarily or permanently by the tumbler for future transactions, or at least they should get down weighted in the selection process (e.g. divide their weight by two for each failure).

[overcoin]

I can confirm this issue. In addition to already said.

The number of failed txs is increasing. As a rule, tx is correctly finalized with [MainThread ] [INFO ] goodtx and after 3 minutes it ends up with [MainThread ] [INFO ] Timed out: transaction with outputs: (...) not broadcast. Mempool log reads: Failed to broadcast transaction, reason: bad-txns-inputs-missingorspent.

[PulpCattel]

Thanks for reporting guys.

Adding here a little more context from Telegram, please correct me and/or add more info if you can.

OP has attempted at least 6 consecutive CoinJoins as a taker spread over 36-48 hrs (ran out of commitments 2 times). They all were aborted by the taker itself because too many makers responded with non-eligible UTXOs (so it either failed the max_sweep_fee_change check or simply there were not enough makers remaining). At the time, those makers (6-7 makers overall?) appeared to have big bonds and were (kinda as expected) picked repeatedly in all the attempts.

Overall, it seems there could be potentially a few different issues:

• Abnormal number of non-eligible UTXOs returned by multiple makers.
• Fidelity bonds appearing and disappearing
• The possible interpretation that the above means a Sybil attack or the like

I think something that could help reducing the impact of such an issue would be to cap the weight of a given maker beyond the fidelity bond value calculation

I'm not sure that would help, it seems it could make the attack cheaper. Of course it depends on the details. I'll also repeat myself from other issues that changing how fidelity bond works is horrible, we are rug pulling makers that already locked their coins. If we were to do it I'd hope it would be as close as possible to a order of magnitude kind of improvement, and not just something that it's maybe somewhat better. I.e., someone would need to look deeply into this.

I think ultimately JM would benefit from a better Sybil resistance mechanism.

Also is it possible to blacklist makers when they appear to be colluding and attacking the integrity of the service?

The maker identities are by design ephemeral, they can restart and get a new identity. In practice, they often run for long period of time, so banning them could still serve a purpose and IIRC there's already somewhat of a design to do that in JM but it's not fully implemented or something. Also, if they have a bond, banning the bond seems effective. And in some cases maybe we could ban UTXOs, not sure.

I can confirm this issue. In addition to already said.

@overcoin
Can you clarify this? The log you posted are maker logs, not taker. Do you also have taker logs? If so, can you post them to see if they match OP?

The number of failed txs is increasing

Can you quantify this? How bad are we talking? And are you always talking maker side?

As a rule, tx is correctly finalized with [MainThread ] [INFO ] goodtx and after 3 minutes it ends up with [MainThread ] [INFO ] Timed out: transaction with outputs: (...) not broadcast

This per se should not be necessarily related, the taker can crash, lose signal, etc. It's "expected" that sometime the CoinJoin fails mid-negotiation. OP's case seems different from this, it would mostly fail before you even receive the goodtx message AFAICT.

Mempool log reads: Failed to broadcast transaction, reason: bad-txns-inputs-missingorspent.

What's "mempool" here? Your node log? Or you mean the "mempool related" message in your JM log?

This also doesn't seem necessarily related IIUC. In theory you shouldn't see this all the times on your side, because as a maker you won't always broadcast the tx yourself (by default it's done by one counterparty at random). I'm also not sure it can happen in OP's case, because the UTXOs offered by the makers in his case seems like should never be accepted by the taker, while in this case the taker went with it.

[PyGryhapoRbryDer]

Thanks for reporting guys.

Adding here a little more context from Telegram, please correct me and/or add more info if you can.

You don't have a forum that is a bit more private than telegram, such as SimpleX or Nostr, correct?

OP has attempted at least 6 consecutive CoinJoins as a taker spread over 36-48 hrs (ran out of commitments 2 times). They all were aborted by the taker itself because too many makers responded with non-eligible UTXOs (so it either failed the max_sweep_fee_change check or simply there were not enough makers remaining). At the time, those makers (6-7 makers overall?) appeared to have big bonds and were (kinda as expected) picked repeatedly in all the attempts.

Overall, it seems there could be potentially a few different issues:

* Abnormal number of non-eligible UTXOs returned by multiple makers.

* Fidelity bonds appearing and disappearing

* The possible interpretation that the above means a Sybil attack or the like

Everything you said above describe exactly what I experienced

I think something that could help reducing the impact of such an issue would be to cap the weight of a given maker beyond the fidelity bond value calculation

I'm not sure that would help, it seems it could make the attack cheaper. Of course it depends on the details. I'll also repeat myself from other issues that changing how fidelity bond works is horrible, we are rug pulling makers that already locked their coins. If we were to do it I'd hope it would be as close as possible to a order of magnitude kind of improvement, and not just something that it's maybe somewhat better. I.e., someone would need to look deeply into this.

I think the equation that calculates the fidelity bond value is fine, but the issue is when a maker has a fidelity bond value so high that it becomes almost certain for this maker to be included in all attempts. What I am thinking about is to cap the value of bonds when they become so dominant in the order book that they would become included in all attempts. This would allow some room for other makers with bonds, but still proportionally to their bond value.

int nomakers; //Number of makers in the orderbook
double ombvalues[nomakers]; //Fidelity bond values for the makers in the orderbook (assuming this gets set for each attempt)
bool largemakers[nomakers]; //Makers that are too dominant;
int nlargemakers=0; //Number of large makers
bool sumnormalbvalue=0; //Sum of values for bonds that are not abnormally large;
int namakers; //Number of makers for an attempt
double f; //Maximum probability to draw a given maker in an attempt (should be <=100%);
double prob = f / nmakers; //Corresponding probability for each drawn maker in an attempt
double bvmax; //Maximum bond value;
bool newlargemaker=false;

for(int i=nomakers-1; i>=0; --i) {
largemakers[i]=false;
sumnormalbvalue+=ombvalues[i];
}

do { //Do loop only required if order book makers are not sorted by fidelity bond values
newlargemaker=false;

for(int i=nomakers-1; i>=0; --i) {
bvmax= prob * (sumnormalbvalue - ombvalues[i]) / (1 - prob * (nlargemakers + 1));

if(!largemakers[i] && ombvalues[i] > bvmax) {
  largemakers[i] = true;
  sumnormalbvalue -= ombvalues[i];
  ++nlargemakers;
  newlargemaker=true;
}

}

} while(newlargemaker);

bvmax= prob * sumnormalbvalue / (1 - prob * nlargemakers);

for(int i=nomakers-1; i>=0; --i) {
if(largemakers[i]) ombvalues[i]=bvmax;
}

I think ultimately JM would benefit from a better Sybil resistance mechanism.

Also is it possible to blacklist makers when they appear to be colluding and attacking the integrity of the service?

The maker identities are by design ephemeral, they can restart and get a new identity. In practice, they often run for long period of time, so banning them could still serve a purpose and IIRC there's already somewhat of a design to do that in JM but it's not fully implemented or something. Also, if they have a bond, banning the bond seems effective. And in some cases maybe we could ban UTXOs, not sure.

Yes I guess banning the bond would be the best.

[PulpCattel]

You don't have a forum that is a bit more private than telegram, such as SimpleX or Nostr, correct?

There's IRC, see https://github.com/JoinMarket-Org/joinmarket-clientserver?tab=readme-ov-file#community but as you can imagine it is nowhere near as popular. Not sure how active it is nowadays but it remains an option (the logs are available here).

[PyGryhapoRbryDer]

You don't have a forum that is a bit more private than telegram, such as SimpleX or Nostr, correct?

There's IRC, see https://github.com/JoinMarket-Org/joinmarket-clientserver?tab=readme-ov-file#community but as you can imagine it is nowhere near as popular. Not sure how active it is nowadays but it remains an option (the logs are available here).

Thanks. I updated the code in my previous message that caps bond values to prevent abuse, there was an issue with bvmax. It is just a relative cap that prevents a small number of makers from becoming too dominant. Please let me know what you think.

[overcoin]

> @PyGryhapoRbryDer

Because of these very high values compared to other makers, the problematic makers that were using a (common?) out of sync node kept being selected for every transaction. Currently it seems that all these makers no longer have fidelity bonds. I think what happened with these bond values should be looked at if possible?

Fidelity bonds can not disappear. These are well funded rogue makers that switch between fidelity bond enabled/disabled wallets using same utxos. Changing the wallet from which to load utxos and use as commitments takes time to synchronize with bitcoincore instance and against blockchain. So, very often they end up with timeouts and blacklisted commitments. I'm not sure what are they trying to achieve in technicalities, but if they keep on trying they either succeed in harming the integrity of the system or successfully suppressing competition from honest makers.

> @ PulpCattel

The maker identities are by design ephemeral, they can restart and get a new identity.

Connecting the new identity to already advertised fidelity bond from the old identity is trivial.

Can you clarify this? The log you posted are maker logs, not taker. Do you also have taker logs? If so, can you post them to see if they match OP?

My taker logs are similar. My maker logs are more interesting for they leave less space for hesitation

Can you quantify this? How bad are we talking? And are you always talking maker side?

My maker logs from yesterday show 3 "timed out...not broadcast" and 8 "error.. commitment is blacklisted".

This per se should not be necessarily related, the taker can crash, lose signal, etc. It's "expected" that sometime the CoinJoin fails mid-negotiation. OP's case seems different from this, it would mostly fail before you even receive the goodtx message AFAICT.

I understand what you say. This is why maker logs are more interesting to me. All the negotiation process was finalized and raw tx was duly signed. The goodtx message on JM logs is the final step just before the broadcast into the mempool.

What's "mempool" here? Your node log? Or you mean the "mempool related" message in your JM log?

Apologies for not being clear enough. The "not broadcast" tx hex from JM log was manually broadcasted by me using the mempool.space tx broadcast tool. The delay was only 4 minutes. I have no doubt that within the 3 minute "time out" one or more of the original utxos were spent.

On my cnf file I have 7 directory services listed. The yg-enhancedprivacy.py script connects only to "3kxw...". All the rest are blacklisting/blocking my JM identity [nick]. This is the only explanation I can think of because at the same time ob-watcher.py connects with 4 of them.

[PulpCattel]

Fidelity bonds can not disappear. These are well funded rogue makers that switch between fidelity bond enabled/disabled wallets using same utxos. Changing the wallet from which to load utxos and use as commitments takes time to synchronize with bitcoincore instance and against blockchain. So, very often they end up with timeouts and blacklisted commitments.

I don't get what you are trying to say.

As you mentioned, it doesn't seem obvious why they would want to disable their bonds in the first place, but even if they do, why would it take time? They could patch JM to begin with and make it instant, but even if they use vanilla JM can't they just load another wallet that uses the same seed words but with no fidelity bond enabled? I mean, are we talking a few minutes at most every few days or whatever? And why would it lead to timeouts? Are you saying they switch during CJ negotiation or something? Or are you saying something else?

Why do you mention commitments here? Makers don't use/need commitments. And why would they end up with blacklist commitments?

Connecting the new identity to already advertised fidelity bond from the old identity is trivial.

Yes, as I said just a few lines below the part you quoted, if they have a bond, banning the bond seems effective.

My taker logs are similar.

It would still be useful if you could post some parts, maybe the same parts OP posted? How many attempts in your case? In the same time period? Are the makers the same? What fidelity bond values do you see? Did they give you the same UTXOs? Etc.

"Similar" can mean a lot of things unfortunately.

My maker logs from yesterday show 3 "timed out...not broadcast" and 8 "error.. commitment is blacklisted".

The 8 commitment blacklist are definitely a separate issue, that is the taker doing something wrong. It could be an attack as well but an unrelated one.

The 3 other cases don't seem too bad per se. It could definitely happen organically. Do you have data from the time period mentioned by OP? If the overall numbers are much worse that would also be helpful to know.

The "not broadcast" tx hex from JM log was manually broadcasted by me using the mempool.space tx broadcast tool. The delay was only 4 minutes. I have no doubt that within the 3 minute "time out" one or more of the original utxos were spent.

What TX did you broadcast? As a maker, you don't necessarily have the complete and valid TX, so you might not be in a position where you can broadcast anything. Or are you talking as a taker here?

On my cnf file I have 7 directory services listed. The yg-enhancedprivacy.py script connects only to "3kxw...". All the rest are blacklisting/blocking my JM identity [nick]. This is the only explanation I can think of because at the same time ob-watcher.py connects with 4 of them.

This is of course unrelated, probably you are seeing #1435 in your logs? Not sure why there's this discrepancy with ob-watcher, I would be surprised if they were banning your nick. If it's reproducible for you, maybe comment in #1445 which directories are you talking about and maybe the owners will answer? Or at least it would be good to know which one are you referring to.

[PyGryhapoRbryDer]

> @PyGryhapoRbryDer

Because of these very high values compared to other makers, the problematic makers that were using a (common?) out of sync node kept being selected for every transaction. Currently it seems that all these makers no longer have fidelity bonds. I think what happened with these bond values should be looked at if possible?

Fidelity bonds can not disappear. These are well funded rogue makers that switch between fidelity bond enabled/disabled wallets using same utxos. Changing the wallet from which to load utxos and use as commitments takes time to synchronize with bitcoincore instance and against blockchain. So, very often they end up with timeouts and blacklisted commitments. I'm not sure what are they trying to achieve in technicalities, but if they keep on trying they either succeed in harming the integrity of the system or successfully suppressing competition from honest makers.

Thanks! So do you think blacklisting a maker based on both his identity and his fidelity bond when he provides problematic utxos is a solution?

What do you think of my solution that prevents any maker from being included in every single attempt?

[PyGryhapoRbryDer]

Was it determined when this set of makers started using these bonds the first time?

[PyGryhapoRbryDer]

I think something that could help would be to increase the number of outputs whenever possible. One challenge in doing that with JM is that there is a single taker per transaction and he is normally paying for all costs. Fidelity bonds help against Sybil attacks, but it appears to be insufficient. At the end my understanding is that we don't want to lower the overall cost for a maker to reduce the number of outputs that are not under his control. One possible solution I see would be to incentivize a larger number of outputs by including additional makers in a transaction (beyond what is defined in the schedule file) when their miner fee contributions is such that the additional outputs do not increase costs for the taker. This would:
-Not affect fidelity bonds incentives
-Increase the number of outputs, making it harder for an attacker to identify all outputs
-Increase the number of participating makers, allowing in more makers that do not have large bonds if they are willing to contribute to miner fees.

[PyGryhapoRbryDer]

I submitted a PR #1718 that implements a model that limits the ability of an attacker to prevent transactions and/or to constitute the only counter-party in a transaction. It is a bit more complex than the code I posted here earlier due to the handling of bondless makers and to support cases where the number of bond makers is close to the number of selected counter-parties, but it seems to work well and as expected. It minimally caps the bond value for makers that have abnormally large bonds.

[overcoin]

@PulpCattel

Sorry for the delay, I don't have the time to meaningfully answer all your questions at once. Let me answer the first lot.

I don't get what you are trying to say.

I'm sorry to confuse you. Just trying to guess how do attackers manage to render the messaging system inappropriate.

As you mentioned, it doesn't seem obvious why they would want to disable their bonds in the first place, but even if they do, why would it take time?

As already said, they would want to disable bonds because the bond is the obvious way to identify the trouble maker. Creating new onion endpoint, setting new onion hostname and finally sending handshakes with the new identity (aka nick), all this takes time. If you run tor-only bitcoincore instance it takes even more time to get new onion service id, advertise it and connect to some bitcoin nodes.

They could patch JM to begin with and make it instant, but even if they use vanilla JM can't they just load another wallet that uses the same seed words but with no fidelity bond enabled?

I'm not sure what do you mean by 'patching' JM? Yes, loading and unloading wallets using a single bitcoincore instance takes time. My experience show that loading a wallet with, say 1000 txs, usually takes couple of minutes.

I mean, are we talking a few minutes at most every few days or whatever? And why would it lead to timeouts? Are you saying they switch during CJ negotiation or something? Or are you saying something else?

Couple of minutes is more than enough to register valid utxo as a maker and then spend it before PSBT is signed by all parties. As a side note, one question. Why was a tool like add-utxo.py developed if it is not recommended to be used?

Why do you mention commitments here? Makers don't use/need commitments. And why would they end up with blacklist commitments?

I'm fairly new to the terminology introduced in JM as coinjoin implementation. But in every coinjoin implementation makers have to register valid utxos to be included n the PSBT. This is what I had in mind under commitments - the process of registering valid utxos by the maker.

@PyGryhapoRbryDer

Thanks! So do you think blacklisting a maker based on both his identity and his fidelity bond when he provides problematic utxos is a solution?

Identity based on the nick handle can be easily disposed. Fidelity bond is the way to identify dedicated honest makers and is the way to also identify rogue makers providing problematic utxos.

What do you think of my solution that prevents any maker from being included in every single attempt?

Didn't have the time to look at it in details, but my understanding is fidelity bond value should be excluded from the blacklist algo. Focus should be on identifying the provider of irregular utxos, disconnects, delays, timeouts, and not broadcasted txs.

[PulpCattel]

Just trying to guess how do attackers manage to render the messaging system inappropriate.

If you have experienced something or have data or ideas about a possible attack, you should probably open a new issue or a discussion, since what you are trying to explain does not seem related to OP case.

As already said, they would want to disable bonds because the bond is the obvious way to identify the trouble maker.

Note that in OP case they did not change nick, this is how OP was able to determine the missing bonds in the first place.

In any case, I can't figure out what the benefit would be. The moment they disable the bond they won't be picked nearly as often anymore, and the moment they enable the bond again (to presumably resume the attack?) they will be just as recognizable because the bond is the same.

If you run tor-only bitcoincore instance it takes even more time to get new onion service id, advertise it and connect to some bitcoin nodes.

Why does this matter? You don't need to restart Bitcoin Core.

I'm not sure what do you mean by 'patching' JM? Yes, loading and unloading wallets using a single bitcoincore instance takes time. My experience show that loading a wallet with, say 1000 txs, usually takes couple of minutes.

If all they want is to disable the bond, they could patch JM to automatically post an offer without bond, and do that for them without needing to load any wallet. I'm also not sure the "single bitcoin core" instance is relevant, the time should be kinda the same even if they use another Core instance.

Couple of minutes is more than enough to register valid utxo as a maker and then spend it before PSBT is signed by all parties.

Note that this is not the OP case, in his case all UTXOs were never valid, the makers were sending him already spent UTXOs repeatedly.

As a side note, one question. Why was a tool like add-utxo.py developed if it is not recommended to be used?

Who said it is not recommended?
See https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/master/docs/SOURCING-COMMITMENTS.md. Note that this tool is only useful for a taker, it's a possible way to use UTXOs external to your spending mixdepth as commitments. OP could have used it when he ran out of commitments for example.

But in every coinjoin implementation makers have to register valid utxos to be included n the PSBT. This is what I had in mind under commitments - the process of registering valid utxos by the maker.

Ok, see the doc above for explanation of what commitments are in JM. Note again that the problem for OP was that makers weren't registering valid UTXOs to begin with.