script does not recognize duplicate addresses (187 missing BTC)

[carryforward]

I am modifying this script to create the factom genesis block. I found it does not recognize duplicate funding to the same address.

If someone sends in BTC twice to the same ether address, only the last one* will be credited by the script.

https://github.com/ethereum/pyethsaletool/blob/master/genesis_block_generator.py#L120

*not necessarily last one in time, because bc.i and json mix things up.

[carryforward]

I hope we caught this early enough to prevent large losses of value.

Here are some examples where this bug is evident:

• 1 https://blockchain.info/address/14fZq9Swf4afbwVrJVRxDxtBSmwd11qR4x
  This buyer sent BTC in in the first 2 weeks and should be credited with 0.2 * 2000 = 400 ether. The address 283396ce3cac398bcbe7227f323e78ff96d08767 is only credited with 360 ether according to this converter
• 2 https://blockchain.info/address/1FrWqyHKYV5NfKWg89dRvoMchyLV46TEn3
  This buyer sent in 46 bitcoins valued >$20,000 at the time.
  The address 5d3f3b1f7130b0bb21a0fd32396239179a25657f is only credited with 2674 ether according to this converter currently cost ~$300 in BTC. The numbers are off by a bit with variable rates everywhere, but shouldn't be off by two orders of magnitude.
• 3
  Here is an incomplete list of the transactions which have duplicate funding addresses.
  252f42f5fc5b352b924c7f91770caf8e8e1e6dcc40e951a18735eb5076d24e40 560bc991da50657ec82df09eb32a982ec6bfcecb50912eb84663cccf62578a42 1acb4cc4a7bdeddad2faf3042b6fe9e33ea6f420ef10e42f450d95e4074c1f21 2d8bb472e80e705820db6a48f70dc307a7bfdde0f346d52e7c78539d010c8180 a7af519d82488703db7cfa46f0eb81116f7c18e0f5d66fd3ce8fa7618c96f9d1 3c5361325a7d114bebcaa5a44907bdebdbc3708e16d131d05f7ac3e09b83afe1 df8b526ac71d29a085502101f88625af070e8df17c133a45a560d0822fece594 3a14e0eceab9e68a297e9fff6b480a8395a95f4161d480a165016f6782c2f354 c3651f15a030126f6aba7389aca34ff63676619daccb89c599ee8ae94e4a1cad 0450ddf6cd955f169cf7a90184133ced433c42fd76f906face14ddb06c30404b c50b7ac3064318286f0d5d7255d2d92efb8637cd31a04b5f2c8d5f2c2a2848d1 b1b68dbdfb71e559bba16e3737084ebc8d9a5b8749cb326d4a498ca45e28c048 579a35711b7dd9570be080fc8fd324c0e4293136c892af79861cc22e43344777 02c201b64fde5aad18c566eb42f502e1cdba9d54888bad23be26c5c4f9eb2864 30e7fc5d367efae865635d97c90b2a01317cbffddb7a564db003b7b33cf06cb1 7d5a59a2c42ba9fe5d1865778125d08fd57695aeef5ef3493393f9a6a62cd1b3 595a0a6fdfc5767708c29d763c83315e06bca7d4c077daa0503d85642cabda11 a6dd7e5d865ab371e9f9722ca03c890906777db5e93d4ca4e1884a5ee8834925 bf4913d99033f54c0fe0ac106b6576c606263a9fab8e16f99ba5732c9596bd73 20d55cd150bc9789b34e17f3b91b0c975cb5588ff257b4c03c51ac13e2f7cadf 7a9bc31cb1ae34c54371bff5a9135063bf87df19f8b3833e9bf3e13897e7c08a b67cde2f5e97bfeb14b0e0ca96e458aff1dd8ee1ade6ec26a9999f53910bbbfc 600f893bbe5c985048cbbb08c5c785cfa9673b4d4a94cc14434a54ce1ac2f81d db06caf8d218bcc55fcbcdb7d512a9f1c025b5a3efb6fcd29133e8a6d96f5a11 fb0b9e0d010180f6f34fe0bcc3dbc9f19b5a0302db36e68c78be09d1653c138c 63d5110d94c388ae2a81040d81b1a9962956ed6ee10b8bacffa5eb30cc2f9e7f e32227ca11af8d61a615ad9ad3249c7acaf4cc12b33cd98c6a7d7ded9010137b 1cba314b6545314c3cbf98d8c974ec243854dff57371c5daca3017b68122dd3e 80da305f56a1077fc10776dfec5bbcd6d1612980fba9d76100d19194235121bd 85b325abd2f6454ee980cccf3f6439ab7cfdffb1b857c4aede127001f65cd3bd 06695cc1008e8630fcb78332502416b8e3a4da3f47616df6034d5eb21aa77040 2c2d3006795695ccd85691d48ed11088d6465ee673342e907618201fc4707875 61b59d4facb816f2ef50cb4a753184b410e844fd6ceff0827e62c40c0e90cc6c 14c7a93a24707efd80e9569c409b17428e8f364c16bb95b8b5934eede21006db 0145b1700f39fcb2670d4b03a0380af841541b7a14326e73d2e92f3732557482 af5f94b3fbc1d41242496c9c30de9c623a61ea7d985ad9903b62fb26e90bbf4a 3d2471371f3650d61afc016e9cde94da7509ae90fcc89500a6281b0a1659f198 abd8dab3ae7342267db9b454c6dc3ee279ac4dfd6e92a8165b059057025c7093 87b34f2488223010bb2c4be556c3ba7e1768d7331b9947013d2defe9c4ea5ef1 ecfa3be55c52584552a278ecc7d1ee15d9b1188f0e9d63b9fbb46cb6978c9363 af7d5af64eded460ce694e6a41dcdfb5bedcd6248168965d4056e78a5219807a ece43e4cc48a30947939bbf84f22104e339a311eff1b5ed3fb391acfb644492b c535b882362114b99879411d7c6c71847a88ce6229e7f1e0b86f619fc4c4e82c e5d83fcb112d61773b238c38046096932cddc4b47b805bec2c36e117738419ad 4f8dfdf56b582dc77d27ff413776f39f2a928e501f33a8822201547b3430c223 7b00a94da57f786839058aeb325a4ea8d0c5cc47b284c9d944de29c00ad3a6fe 48584b28af306d1d4434ba9e6c3dc101344c843f58ca2329f592b4c8a587eaba 59ac610a4961f9a96248c2cbc317bccc7196ae20c205d08e2fbc7cc440cc4935 59f401c040f4fb261245189ab2c2008f27823c3f0375450c6a04d3491a57399d ec209a5517066c7285509e1320d82a6f9711dc6e3722652c0623bf87ad02c338 977adf5006e30e488428c36b615b834a919feba3932da662bd21145f9adc9296 355f2710c1e720d97629cfa03ce9be70f1136179fd613b314c7e4d18b9abe2a8 3fc5230ad00eeb3d5c1009c98609a6bbb4ba0f4c0dcf21231c27e3fcd81868a5 826a2cc05903e18ff0059c74ef579f5e9352c5d446746efe4552ec64fe828eb9 3de04b5695300af031ab1c1daba98be33b556616775019f6e3c037974662ed4e 99e501e7167ba0e66f8fce339ce07727431cbc527f69d15c80dbeeb81056a31c d581e2ac85c5112454fa314647b5b981eea97c0647a8b41e02dc6a1796a3dab8

[carryforward]

I finished downloading the transactions and cached the b.i transactions, so I can write more detailed scripts.

With the order of transactions that b.i returned data for me (which may not be deterministic) there would have been 187.67369862 bitcoins not credited to the genesis block. (which had an exchange rate of ~$116K at the time the ether sale started.)

Here is a list of transactions which would be not included in the current genesis block:

{'wei': '415292000000000000000', 'tx': '87e57a0ea1ac7097d8fb3a56e8f6bdccb40328a99b7a9d3b045c607319ded411', 'btc': '22818259'}
{'wei': '397909000000000000000', 'tx': '252f42f5fc5b352b924c7f91770caf8e8e1e6dcc40e951a18735eb5076d24e40', 'btc': '21508607'}
{'wei': '5582670000000000000000', 'tx': 'c1313f4e4789f91c81907ca4da1d0808342ffad07daef5bd594f0618409f1371', 'btc': '279133520'}
{'wei': '20000000000000000000', 'tx': '76d736393549ac990fc903f48927824e59723756be6fec5da2b659ec71bf5727', 'btc': '1000000'}
{'wei': '25403000000000000000000', 'tx': 'f0cf9d6499fc55218e215a4b668caa1577d75decd124f7774bc2574aa88c0c02', 'btc': '1900000000'}
{'wei': '2944200000000000000000', 'tx': 'c0fd894f4718588cb465f7c37a025c925b1f15ab39c103efaa2e20e5611ef366', 'btc': '147210000'}
{'wei': '20000000000000000000', 'tx': 'd4285cbc8bd2a71f379a3091e5a219f3fb35c722f5a425d1bbd44b3d792b0718', 'btc': '1000000'}
{'wei': '20000000000000000000', 'tx': '2d8bb472e80e705820db6a48f70dc307a7bfdde0f346d52e7c78539d010c8180', 'btc': '1000000'}
{'wei': '13400000000000000000', 'tx': '7ca33abc63b6ed19272f4b0a54658bf1fd9981695750893803c75f19f4ce6e8e', 'btc': '1000000'}
{'wei': '8000000000000000000000', 'tx': '74a48df33094fefee31b25cac6ab81d7a749e62653ba762353ddd2b6192178e5', 'btc': '400000000'}
{'wei': '780000000000000000000', 'tx': '4762ace6bb8967957153bcc70b6841690ecca516e81bcf0f26f7c5d7ef331e98', 'btc': '39000000'}
{'wei': '160000000000000000000', 'tx': '7b370b8528384b03ec0871d3de15e483defad68781aafa50f9bc16354a383767', 'btc': '8000000'}
{'wei': '26740000000000000000000', 'tx': 'a7af519d82488703db7cfa46f0eb81116f7c18e0f5d66fd3ce8fa7618c96f9d1', 'btc': '2000000000'}
{'wei': '152000000000000000000', 'tx': 'b31ada0855322d215a95b935103eb42dde30b2cec0ecb7d2ce442e2638077c1c', 'btc': '10000000'}
{'wei': '18685000000000000000', 'tx': '560bc991da50657ec82df09eb32a982ec6bfcecb50912eb84663cccf62578a42', 'btc': '1010000'}
{'wei': '20000000000000000000', 'tx': '2517e27690563573784e850685063cd96ecf3633098d6b0b0c8eff08ea4c5a5f', 'btc': '1000000'}
{'wei': '100016000000000000000', 'tx': '02c201b64fde5aad18c566eb42f502e1cdba9d54888bad23be26c5c4f9eb2864', 'btc': '5320000'}
{'wei': '4800000000000000000000', 'tx': 'e9648e2fb6bd861bf8310b5c4fc45c2d9ff7d55e1c38fbd414c633631bdfe754', 'btc': '240000000'}
{'wei': '500000000000000000000', 'tx': 'e8870de68d4f4865f4e8a87f2e9f21433f1234fe8fce4686ec7004df8d92aecf', 'btc': '25000000'}
{'wei': '137013000000000000000', 'tx': 'a6dd7e5d865ab371e9f9722ca03c890906777db5e93d4ca4e1884a5ee8834925', 'btc': '6955000'}
{'wei': '23900000000000000000', 'tx': '70a00d46100fd442066a25d7d13b2f53da92a7221ceb04586232f06f674106e7', 'btc': '1195000'}
{'wei': '10000000000000000000000', 'tx': '4ef0ff81d4c1f52b1e92d1c6eaf9bfb61d3bb6746fc4d2a38ab5bde861e4a3f5', 'btc': '500000000'}
{'wei': '1337000000000000000000', 'tx': '732a0f2d894505a671b7a7d4e3a7aaab959fab7bb21541154a20e2c624549e60', 'btc': '100000000'}
{'wei': '500000000000000000000', 'tx': 'a73d152f429a88f9b0a366d8b1c825d8e2d77fd8a9dd2678c8f79bd4e653f73d', 'btc': '25000000'}
{'wei': '13370000000000000000', 'tx': '600f893bbe5c985048cbbb08c5c785cfa9673b4d4a94cc14434a54ce1ac2f81d', 'btc': '1000000'}
{'wei': '1000000000000000000000', 'tx': '7a9bc31cb1ae34c54371bff5a9135063bf87df19f8b3833e9bf3e13897e7c08a', 'btc': '50000000'}
{'wei': '65100000000000000000', 'tx': 'cdd4cb2fe63a380527a01cfeb825bb635799fddd1f04f034f3cb2ed725674f7b', 'btc': '4200000'}
{'wei': '26437000000000000000', 'tx': '63d5110d94c388ae2a81040d81b1a9962956ed6ee10b8bacffa5eb30cc2f9e7f', 'btc': '1342000'}
{'wei': '100000000000000000000', 'tx': '9bec9f22416c99988f197a6101a70affe795bcfb6502367614ae5a925eb1d652', 'btc': '5000000'}
{'wei': '2000000000000000000000', 'tx': '0450ddf6cd955f169cf7a90184133ced433c42fd76f906face14ddb06c30404b', 'btc': '100000000'}
{'wei': '199120000000000000000', 'tx': 'e32227ca11af8d61a615ad9ad3249c7acaf4cc12b33cd98c6a7d7ded9010137b', 'btc': '13100000'}
{'wei': '20000000000000000000', 'tx': 'ea1663cfe66bdf90a57d25c5d0d357ff2e4b0dd6be9ee4de9250f75ab0eb942e', 'btc': '1000000'}
{'wei': '1490745000000000000000', 'tx': 'c3651f15a030126f6aba7389aca34ff63676619daccb89c599ee8ae94e4a1cad', 'btc': '100050000'}
{'wei': '20000000000000000000', 'tx': 'c50b7ac3064318286f0d5d7255d2d92efb8637cd31a04b5f2c8d5f2c2a2848d1', 'btc': '1000000'}
{'wei': '3417090000000000000000', 'tx': 'ffd06d07a1da1b8d51e8d644223816347432309dc2afafdb1a872f6d182a866e', 'btc': '170854500'}
{'wei': '6320000000000000000000', 'tx': '579a35711b7dd9570be080fc8fd324c0e4293136c892af79861cc22e43344777', 'btc': '400000000'}
{'wei': '1337000000000000000000', 'tx': 'af5f94b3fbc1d41242496c9c30de9c623a61ea7d985ad9903b62fb26e90bbf4a', 'btc': '100000000'}
{'wei': '200000000000000000000', 'tx': '75d38c702590fc11f1fe4679c8cf00bdc9d776d1e0ac05a01a5424dab0dc6ed7', 'btc': '10000000'}
{'wei': '13370000000000000000', 'tx': '556a79f2b7ce74e13043ca4ea5dcbaec74425b97d8e72edf74f7cc0b84cd2975', 'btc': '1000000'}
{'wei': '1799800000000000000000', 'tx': '334ad252304622db736ccd853a995b492ab3fe3d0e19066bf6c84b34044a62ce', 'btc': '89990000'}
{'wei': '98500000000000000000', 'tx': 'b67cde2f5e97bfeb14b0e0ca96e458aff1dd8ee1ade6ec26a9999f53910bbbfc', 'btc': '5000000'}
{'wei': '45912000000000000000', 'tx': 'c1287d02d214b62a548aa932abf571e65bc43bfd27aeb5eacd65c4bc8b5bc05c', 'btc': '2799534'}
{'wei': '13370000000000000000', 'tx': '55ae93b35899583e67e7435fa0fa1e7df0b44efe42a5a88ba96e72c5db61ba46', 'btc': '1000000'}
{'wei': '259378000000000000000', 'tx': 'cac80fe396cf08cb81462471bd96563a02c49c013f62f6c8d37196f8d5a2fdfa', 'btc': '19400000'}
{'wei': '4850000000000000000000', 'tx': '07dd2adf29a43f9caab86c38f10533fb7290dc7640d04655426af38c511f13f1', 'btc': '250000000'}
{'wei': '20000000000000000000', 'tx': 'b1b68dbdfb71e559bba16e3737084ebc8d9a5b8749cb326d4a498ca45e28c048', 'btc': '1000000'}
{'wei': '329840000000000000000', 'tx': '595a0a6fdfc5767708c29d763c83315e06bca7d4c077daa0503d85642cabda11', 'btc': '21280000'}
{'wei': '267400000000000000000', 'tx': 'e5d83fcb112d61773b238c38046096932cddc4b47b805bec2c36e117738419ad', 'btc': '20000000'}
{'wei': '10000600000000000000000', 'tx': 'b3991c29153980f547f03eb53190d7cda63e0d2a39f278eea6da7263c9e27e4c', 'btc': '500030000'}
{'wei': '21340000000000000000000', 'tx': '15085427eb8da8fa0d65f8a3224b3c7198e4f891e502de59bfce1bc089c25bc8', 'btc': '1100000000'}
{'wei': '419240000000000000000', 'tx': '30e7fc5d367efae865635d97c90b2a01317cbffddb7a564db003b7b33cf06cb1', 'btc': '22300000'}
{'wei': '413526000000000000000', 'tx': '977adf5006e30e488428c36b615b834a919feba3932da662bd21145f9adc9296', 'btc': '23102030'}
{'wei': '13370000000000000000', 'tx': '8e876f10b0bd9b7ff10371a3f2c63329b5dfa3d670112c5369498a5a714ea766', 'btc': '1000000'}
{'wei': '20000000000000000000000', 'tx': '2c2d3006795695ccd85691d48ed11088d6465ee673342e907618201fc4707875', 'btc': '1000000000'}
{'wei': '197000000000000000000', 'tx': 'af7d5af64eded460ce694e6a41dcdfb5bedcd6248168965d4056e78a5219807a', 'btc': '10000000'}
{'wei': '3838527000000000000000', 'tx': '20d55cd150bc9789b34e17f3b91b0c975cb5588ff257b4c03c51ac13e2f7cadf', 'btc': '200970000'}
{'wei': '267400000000000000000', 'tx': '59ac610a4961f9a96248c2cbc317bccc7196ae20c205d08e2fbc7cc440cc4935', 'btc': '20000000'}

I added this code:
         else:
             rate = post_rate
         # Round to the nearest finney (nearest ether would be // 10**8)
+        if p["addr"] in o:
+           print str(o[p["addr"]])
+           totalbitcoins += p["value"]
+        
         o[p["addr"]] = {
-            "wei": str((p["value"] * rate // 10**5) * 10**15)
+            "wei": str((p["value"] * rate // 10**5) * 10**15), "btc": str(p["value"]), "tx": str(p["tx"])
         }
+    print  "total_bitcoins" + str(totalbitcoins)
     return o

[carryforward]

I have created a pull request fixing this issue.
#4

[jmiehau]

paging @vbuterin and @tgerring

[alexvandesande]

Transcript from the IRC:

[7/23/15, 10:49:37 AM] Metalic Boterin: This issue open on PyethSale affect the Ethereum crowdfunding? #2
[7/23/15, 11:54:17 AM] Metalic Boterin: [9:48:47 AM] jmiehau: This issue open on PyethSale affect the Ethereum crowdfunding? #2
[7/23/15, 11:54:19 AM] Metalic Boterin: yep I caught it
[7/23/15, 11:54:35 AM] Metalic Boterin: well, someone else also poitned it out and I fixed it

[carryforward]

Vitalic fixed the script here:
c598351#diff-f3f5a210ac5d70eacb2d8ef719d757daR120

but it doesn't look like the example output was changed.
carryforward@cab1c9d

Just make sure to run the script instead of using the example output. Note, running the script takes hours, since BC.i rate limits you.

This other bug would require changing the source data that the blocks are built against. You could probably do it by modifying this code and running a full bitcoin node, which also has full indexing turned on. it took several days to get up to speed on my machine.

It may not be worth fixing, though. it is only a small amount of money that will be lost.

[vbuterin]

The genesis block generator there is NOT the one that will be used for the frontier launch. The one that will is currently sitting in a private repo and will be released very shortly before launch. I just checked, and the current release candidate frontier genesis block assigns the correct 400 ether to 283396ce3cac398bcbe7227f323e78ff96d08767.

[jmiehau]

👍