NIP-05: Moves from kind 0 to its own event kind.

[vitorpamplona]

This PR moves the nip05 property from kind 0 to its own kind 10008 so that we can:

• have multiple nip05s.
• stop polluting kind 0
• filter 10008 events by nip05

It also rewrites the extremely verbose NIP-05 text without changing any behavior other than the new kind 10008

Read HERE

[tkhumush]

I like this change.
This will allow me to have users in multiple communities in nostrhood.social.
Although I would wish if there was a way to make one as the "Primary" nip-05 which we typically show on profiles when rendered in apps. Otherwise, how would apps know which one to display?

[vitorpamplona]

Usually the first tag is the primary. It's like the title on NIP51 lists: only the first tag counts.

[dannym-arx]

I don't like this to be honest. Why not just put two nip05 tags in the kind 0? Clients that parse profiles incorrectly will see the first one or the last one, but others will see all of them.

[fiatjaf]

Noble but it's impossibly hard to effectively get rid of the JSON. Same for the other fields. nip05 was my mistake, but the lazy people who gave us lud06 and lud16 and website and display_name, username, displayName and other variants aren't going away.

Instead we should think about the future, create one or multiple new kinds for all the extra stuff and hope that people will add things to it instead of to kind 0. Extra nip05 addresses could go there mixed with a load of other stuff like Monero addresses, background music and gender pronouns.

#2165 is a start.

[fiatjaf]

The only question in my mind is

• if we should just do one kind for everything, assuming it won't be fetched for everybody but only for profiles actively browserd by user; or
• if we create two new kinds, one for small things that could be fetched for everyone, like preferred color and alternative addresses, and one for obscure stuff like background music; or
• if we create one different kind of each thing.

[vitorpamplona]

Different kinds for each thing.

I am going to follow this PR with a change to the zaps nip to move lud fields there in a new kind.

Same for external identities tags. We should move them to that nip on a new kind.

Let's stop overloading kind 0 for everything. Otherwise we will get to a point that nothing will fit there.

Every nip that needs new params in kind 0, should just add its own event kind instead.

[vitorpamplona]

I don't like this to be honest. Why not just put two nip05 tags in the kind 0? Clients that parse profiles incorrectly will see the first one or the last one, but others will see all of them.

Because very few clients support nip05. Why are we even bothering them with this field?

[pablof7z]

cACK. This is a good move. We need to start setting the right precedent with these things.

This is better than the status quo, but I think we should go for a standard, non-replaceable, kind:

{
   "kind": 1234,
   "tags": [
      "n", "<nip05>",
   ],
  "content": ""
}

[sondreb]

There is still a reference to kind 0 that should be changed:

"then it tries to fetch the kind 0 event for that user"

[dannym-arx]

Because very few clients support nip05. Why are we even bothering them with this field?

I can't think of one that doesn't. amethyst, damus, nostrudel, jumble, primal, nosotros, yakihonne.

Maybe clients that are not kind 1 feeds don't. But for kind 1 clients I can't think of a single one.

cACK. This is a good move. We need to start setting the right precedent with these things.

how is it a good precedent to have yet another extra event kind that needs to be fetched for each user?

kind 0 is already large for stupid reasons (json content). A minimal kind 0 with two fields, name: "A" and nip05: "b@c.d" is 378 bytes. I have 263 people I follow, loading the kind 0 for them is at a minimum 99414 bytes and now we need to change it to 358 + 359?

We're doubling the amount of data required just to fetch a profile. That's almost 188571 bytes in my case, for no benefit.

My proposal of getting rid of the json stringified content and moving to tags would make it less data (372 bytes, which is still a ton for a minimal profile, but whatever).

and all my math ignores the messaging prefixes ["EVENT", "subname", that also needs to be sent over, as well as the REQ which now also needs to be longer. 200kB+ for what should be a fourth of that, if even, is terrible.

it's not a good precedent to keep making nostr slower. someone who follows 1000 people will need 1 MEGABYTE just for loading profiles.

Entire operating systems fit in less data than that. And that's not a stretch, there are issues and PRs in this very repo talking about how follow lists being limited to 64k is a problem. Those people will need to load multiple megabytes just to see a name and dns name attached to an npub they follow. This is completely unacceptable.

[vitorpamplona]

There is still a reference to kind 0 that should be changed:

"then it tries to fetch the kind 0 event for that user"

Fixed.

Maybe clients that are not kind 1 feeds don't. But for kind 1 clients I can't think of a single one.

Most clients are not kind 1 these days. Nostr has passed the point where kind 1 is the central point.

how is it a good precedent to have yet another extra event kind that needs to be fetched for each user?

Amethyst already loads about a dozen extra "settings" event kinds for each user. It's great because each client can choose what and when to load instead of forcing everyone to load the settings for every other application out there. Furthermore, keeping these "setting kinds" in the same NIP as their usage puts everything in one place and isolated from interfering with the rest of the NIPs. It's just a far better NIP design pattern for anything I can think of in Nostrland.

[vitorpamplona]

I think we should go for a standard, non-replaceable, kind:

I would agree, but deletions are optional on Nostr and generally coded incorrectly (fuck you strfry), creating far too many problems for "user settings" events. Replaceables just work almost everywhere.

[dannym-arx]

Furthermore, keeping these "setting kinds" in the same NIP as their usage puts everything in one place and isolated from interfering with the rest of the NIPs. It's just a far better NIP design pattern for anything I can think of in Nostrland.

I can see the advantages, but really don't agree with making nostr even slower. Data requirements are already huge due to json serialization and websockets, and the protocol being noisy, and there are already so many event kinds one has to fetch just to get a basic experience working that I honestly don't think it's going to help us in the long run.

I don't have problem with this change in itself, I think for the most part you're right, but I think that if this becomes the norm it will only put even more things into their own event kinds. You mentioned lud16 for instance... and I can only imagine there will be more.

[sondreb]

I'm stating the obvious, but events are cryptographic objects. Each new one compounds the amount of compute that is required (signature verification), in addition to the fact that we're communicating over the Internet.

Right now to discover a user, either kind 3 or 10002 is needed, and then a profile can be retrieved. This is enough to be able to verify (NIP-05) and send payments (zaps) to a user.

Here is also the obvious: Events shouldn't be treated as database columns. They are much more resource demanding, so going the route of having even more events for basic stuff is in my view not optimal. I don't think this should change, I don't see much valid reasons to remove existing definition. We've already started doing tags instead of string encoded JSON for kind 0, this enables multiple NIP-05 values.

You also suggest moving external identities and zap address away. This introduces a whole lot more effort for relays and clients.

This will definitely move Nostr more away from being decentralized to require centralization of data as the UX will undoubtedly increase loading times for clients that doesn't rely on server-caching and pre-computation.

"lud16" is now a tiny little field on an existing stored and processed event. Moving it into it's own event, increases the storage taken by the value by multiple times. It also increases the bandwidth to load it multiple times. It increases the processing time multiple times.

I think there has to be some really good reasons to go down this route.

[dannym-arx]

Exactly, there's absolutely no reason to go down this route, if anything we need to think of ways to make it MORE efficient to gather the data you need, not less.

[vitorpamplona]

Verification is super fast. I don't get why 4 new verifications will change anything since we already certify 1000s of events in a single second.

And you don't need to load all of them. Why would a client even download them if the client doesn't support nip05, which is most if them?

It feels like you two think all clients MUST support zaps and nip05, which is ridiculous. Most clients don't need those things at all. Yes, your clients will need to create faster verification procedures if that is slow, but other clients are relieved from having to even download this data. Don't force your client's need into everyone else.

[dannym-arx]

Verification is super fast. I don't get why 4 new verifications will change anything since we already certify 1000s of events in a single second.

Well, event verification is actually slow on nostr in general, because of the serialization requirement (memory allocations are slow, and JSON stringification in general is slow) + secp isn't the fastest. We can easily get the serialization and deserialization speed an order of magnitude faster (look into binostr), and some later tests that I did later made it even faster, closer to 2 orders of magnitude.

But this is not the point of this discussion imo.

And you don't need to load all of them. Why would a client even download them if the client doesn't support nip05, which is most if them?

But clients that do need to now need to do twice the work, and download twice the data. Thrice if we also move lud16.

It feels like you two think all clients MUST support zaps and nip05, which is ridiculous.

I don't, but I think that we're already in a TERRIBLE situation with data requirements for nostr, for lots of reasons that don't make sense and come from poor design choices, and we don't need to make it worse, that's all. If clients HAVE to put more data into kind 0 for whatever reason I'd rather have that, which is gonna add a few bytes over a whole new kind which is gonna add hundreds of bytes + more computation.

[vitorpamplona]

If clients HAVE to put more data into kind 0 for whatever reason I'd rather have that

If Amethyst puts everything it needs in kind 0, we will have 1MB kind 0 events easily. I don't think that should be a burden that everyone needs to suffer just because my client needs a lot of info to do its thing. That's my point. We can't just add everything in one kind just because it makes it faster for one client and slower for everybody else.

[staab]

Maybe a good middle ground would be something like I proposed for lists. In other words, a new metadata kind 3xxxx with a tag f for which fields it defines. Reading clients can fetch only fields they care about based on the f tag, while writing clients can consolidate updates and avoid overwriting values due to missing events by publishing an event with a fresh d tag if no 3xxxx is found.

So for example:

// Event 1
{
  kind: 3xxxx,
  created_at: 1,
  tags: [
    ["d", "298374932634"],
    ["lud16", "whatever@provider.com"],
    ["pronoun", "cow", "cow"],
    ["f", "lud16"],
    ["f", "prounoun"]
  ],
}

// Event 2
{
  kind: 3xxxx,
  created_at: 2,
  tags: [
    ["d", "298374932634"],
    ["lud16", "something-else@provider.com"],
    ["website", "example.com"],
    ["f", "lud16"],
    ["f", "website"]
  ],
}

If you want to know the website, you ask for f=website. If you want to know lud16, you ask for f=lud16 with limit=1 and take the most recent one.

That said, I don't think we should move existing widely used kind 0 fields, either out of the json or to another event kind.

[sondreb]

If Amethyst puts everything it needs in kind 0, we will have 1MB kind 0 events easily.

What is this special need to store 1MB of data on kind 0? Thousands of LUD16 address and thousands of NIP-05 addresses? Because you need to store that amount of extra data for someone, does it warrant changing away from an already established industry standard and protocol? Use any other custom kind for your needs, why does it warrant moving existing fields out of an kind?

This will fragment Nostr even more. There will be many clients that will not implement this. This will not reduce bandwidth, it will increase it. It increases computation needs, storage needs and it also increased faults due to network issues - the more requests needed, the more failures will happen. It also leads to data fragmentation on Nostr network.

There is a difference between polluting kind 0 and removing established industry fields.

The NIP should instead specify only allowed kinds (NIP-01 + NIP-24 +lud16) and not more. It should explicitly state that clients should clear out any custom fields when saving. This will ensure nobody will start polluting kind 0 ever again, because it will be wiped whenever user saves their profiles in most of the apps.

I believe segregation of data into individual events is detrimental to the health of Nostr as a protocol. Each field should not be self-contained with it's own cryptographic object encoded as a JSON file. Splitting up data into individual units should come with great care. Having more fields in less kinds can be more optimal, depending on the data.

Nostr protocol has a lot of fundamental issues and those are not getting fixed, it would be a much better idea to start working on a version 2 of the protocol that can take lessons learnt from version 1, than making changes to the core kinds that will involve massive efforts across the ecosystem.

Don't force your client's need into everyone else.

These changes are forcing every client to modify their codebase. It is also forcing a multi-year long transition period that might never actually end at all. Many clients will have to implement support for both methods. Most will have to duplicate the events, every single profile save will now involve multiple signing operations, multiple publishes. UAC-amole (Windows Vista) was a real security issue... we are pushing users to use signers and extensions, but the end result is that they will simply always do "auto-approve", even those who previously wanted to be a bit more careful. I myself never approve permanent, I want to verify.

There will likely be bugs introduced, and there will definitely be even more data fragmentation on the network. One client might have a bug that it fails to publish or create the correct new NIP-05 kind - leading to fragmentation. Duplication of data is never a good idea.

1. NIP-05 is important for verification (in my opinion), as it's the only mechanism to verify a profile outside of npub. Anyone can easily clone anyone's profile, except they cannot get a valid replication of the NIP-05 field, it must be changed to something else.

2. LUD16 is payment addresses for Zaps. What will happen for many users, is fragmentation of payment addresses, leading to user's receiving (or failing to receive) payments that goes to either kind 0 address or new kind address. This will inevitably happen. Damus (their web viewer) and Primal already caches data in such a manner that it shows outdated and invalid payment addresses - This change and transition period will result in even more of those issues.

I think the best option is simply to update the NIP to ensure clients remove all "non-industry-adopted-fields" and move those to new kinds. This ensures we clean up the kind 0 events that is out there.

[vitorpamplona]

The NIP should instead specify only allowed kinds (NIP-01 + NIP-24 +lud16) and not more.

Cool, so is NIP-05 out?

NIP-05 is important for verification (in my opinion), as it's the only mechanism to verify a profile outside of npub.

NIP-05 doesn't verify anything. It's written in the NIP.

Relying on it to detect impostors is not a good idea since all the impostor needs to do is generate a valid nip05 in a similar domain. If users are not reading the entire e-mail all the time, enough attention to detect minor changes (vitor@vitorpamplona.com vs vitor@vitopamplona.com), they will be fooled by verified checks. And yes, this has happened in the past.

But more importantly, no one codes NIP-05 anymore, exactly because it doesn't provide much security at all. And those who do are trying to sell nip 05 services by overselling a "verification" that has never been there.

[sondreb]

The NIP should instead specify only allowed kinds (NIP-01 + NIP-24 +lud16) and not more.

Cool, so is NIP-05 out?

I wrote industry adopted in my comment as a whole. To be complete:

NIP-01, NIP-05, NIP-24, NIP-57.

These are established NIPs, they have been adopted.

NIP-05 is important for verification (in my opinion), as it's the only mechanism to verify a profile outside of npub.

NIP-05 doesn't verify anything. It's written in the NIP.

It doesn't matter what the NIP says, what matters is what it actually does. It verifies trust. The NIP says it's to look up users, but it has a verification capability in the reverse.

Relying on it to detect impostors is not a good idea since all the impostor needs to do is generate a valid nip05 in a similar domain. If users are not reading the entire e-mail all the time, enough attention to detect minor changes (vitor@vitorpamplona.com vs vitor@vitopamplona.com), they will be fooled by verified checks. And yes, this has happened in the past.

This is exactly the same as domain spoofing. It has the same issues as domain spoofing, but that does not invalidate the capability and value it brings. People can buy domains that look like "Microsoft", "Micros0ft" and get fooled, but that doesn't invalidate that you can actually verify real employees, real web servers, through the Microsoft domain.

It is currently the only way to differentiate and VALIDATE profiles on Nostr, other than the npub.

But more importantly, no one codes NIP-05 anymore, exactly because it doesn't provide much security at all. And those who do are trying to sell nip 05 services by overselling a "verification" that has never been there.

Doesn't anyone use NIP-05 anymore? That's wrong. Yakihonne, Nostria and I'm sure others, actually support friendly URLs based upon NIP-05 values. We also display the value and perform validation that it's correct.

There are absolutely NO other mechanism on Nostr for validating ownership of profiles, so this is very important to not start messing with.

[vitorpamplona]

NIP-01, NIP-05, NIP-24, NIP-57.

I think it is already too much. We def should get rid of NIP-24 and NIP-05 to simplify things. NIP-57 barely mentions kind 0, so we could just add the extra event to offer NIP-57 implementers more options to send zaps to for now.

[dannym-arx]

Maybe a good middle ground would be something like I proposed #2123. In other words, a new metadata kind 3xxxx with a tag f for which fields it defines. Reading clients can fetch only fields they care about based on the f tag, while writing clients can consolidate updates and avoid overwriting values due to missing events by publishing an event with a fresh d tag if no 3xxxx is found.

that's not gonna fix the speed problem, data problem and computation problem... in fact it kinda makes them worse since there's more data haha

[staab]

that's not gonna fix the speed problem, data problem and computation problem... in fact it kinda makes them worse since there's more data haha

Sure it does, and it helps with the contention problem too (concurrent updates on a single monolithic event). Instead of minimum 1 event per field, you get minimum 1 event, while the maximum remains theoretically the same.

[sondreb]

I think this is very bad development of the Nostr protocol. I think it will destroy the capability of Nostr to stay decentralized and it makes global scaling much harder and might require more centralized indexers.

We should avoid changing existing NIPs, especially those who have been adopted by developers, apps and users.

I think outbox model is needed to ensure Nostr will continue to be both decentralized and scalable.

If someone in Australia want to grab some data from someone in Europe, that's a physical minimum of ~250-320ms roundtrip.

Moving NIP-05 from kind 0 to something else, means a minimum of additional 250ms.

And now LUD16 is moving out as well. And external identities.

Where will this end? That's the problem here, we're moving towards a model where each piece of data is encapsulated in individual events. This does not scale globally, it's detrimental to scaling and global use of Nostr.

If it's true as Vitor says that most Nostr apps no longer are kind 1, what kinds are they? Which apps are these? Have it been decided that the Nostr Protocol should move away from being a protocol for these social media apps and become an "app protocol", where the relays and the clients are on the same regional network?

This makes Nostr protocol a Client-Server protocol, not a decentralized protocol.

There are so many potential issues with network communication, this compounds for every new event. There might be problems retrieving the event, there might be issues publishing the event, and then we have the issue of sync between relays. Many users don't have all their data on all their relays that they have specified. This data rot and data fragmentation that exists on Nostr, will massively increase with these changes.

If a client doesn't retrieve the kind 3, we all know what happens. Following lists get's wiped.

And everyone will need to implement backwards compatibility for many years to come.

Retrieving the LUD16 address upon user performing the action of zapping, without having the address before hand, makes one-tap zapping more risky. And with this chatter about removing NIP-05 entirely, we're building a "network of events" where there are no mechanism for verification of a public key. No mechanism at all for a user to establish some level of trust, it will require people publishing their public keys in text on website or using centralized messaging and social media to share their keys. We're going back to "PGP Key Sharing Parties".

Nostr was successful because it was simply. This increases the complexity, making it less approachable to developers.

I won't be implementing any of these changes in Nostria and I believe that this is not good development for the protocol.

I suggest building Nostr 2.0, or do what has been done for years - just come up with new NIPs and leave the old ones alone.

[dangershony]

Nostr is successful because it is simple.

This imo is a crucial detail of Nostr.

[vitorpamplona]

Nostr is successful because it is simple.

This imo is a crucial detail of Nostr.

Yes. This is why kind 0 must change.

Kind 0 is by far the most complex kind in Nostr right now. It's virtually impossible to design a kind 0 edit without fucking up the JSON inside of it for other clients. We have random NIPs inserting fields and tags in it at will, and clients inserting random complex properties in it. This is unique to kind 0. We don't allow NIPs to interfere with one another in any other part of Nostr. AI can't get it right. New devs don't get it right. Kind 0 was just name, about, and picture. Then grew up to be a mess because the big client mafia couldn't be bothered designing new kinds for each of their features.

[sondreb]

To elaborate, I don't think this has been well enough discussed and validated. The consequences are probably much larger than you think.

Here is additional issues:

• If there is not two fields for names, it means tagging must be done with full names, not aliases/"usernames". This fundamentally changes every single current client who does tagging of other profiles. It more than doubles the requirements for bandwidth, storage and computation compared to current solution.

• Removing "i" tags, which can be used for external proof verification, has the same effect. It increased the requirements to parse, increasing errors, reducing user experience.

Removing what is already used, does nothing as I have said before. Majority of clients need to be backwards compatible anyway, so "i" tags will have to be written to kind 0, lud16 needs to be written to kind 0, NIP-05 needs to be written to kind 0, display_name must be written to kind 0.

This does nothing to improve Nostr. It's detrimental to Nostr's development. I've said my words now, if nobody else (Damus? Primal? Yakihonne?) cares about this revisioning of the protocol for the sake of changing (I have not been presented with a single piece of good reasoning for the changes), then there is nothing I can do about it.

I've forked the NIPs repo, reverted the latest change which removes "i" tag from kind 0. It changes nothing for most apps that this is changed in the NIP, but the is becomes that the protocol no longer correspond to how majority of clients has been implemented. Having a protocol that does not adhere to the implementations, is definitely not a standard.

Changing the protocol after everyone has implemented it, deserves a version bump of the protocol. Since everything is ending up extremely fragmented anyway from these changes, why not simply launch Nostr 2.0 and clean up everything?

NACK.

[vitorpamplona]

This is definitely not going to be painless, but the consequences of keeping kind 0 a loosey-goosey, heavy, and difficult to parse and understand event in the long run far outweigh the short-term pain of compatibility changes.

Nostr is super small right now. Let's fix things up while we can.

Otherwise, we are going to have bigger problems in the future when AI starts dumping everything in kind 0 because "that's the way it was always done".

[Semisol]

What is the point of this? This is very much solveable by adding a tag to kind 0, and it could be considered core profile information.

filter 10008 events by nip05

Why would you even do this? You would go and ask the NIP-05 server to give you a clear-cut answer instead of asking relays who don't do any filtering on correctness.

[vitorpamplona]

What is the point of this?

Clean up of kind 0, since there's too much trash in that kind. Old NIPs are setting this terrible example of putting stuff in kind 0, and AI is just copying it as a design practice to add everything there too. Which is not what we want.

The goal here is to make it a good example before Nostr gets too big and we can't change this anymore.

it could be considered core profile information.

I don't think it is. NIP-05 is optional.

Why would you even do this?

Because Web of Trust providers would like to know all keys that are claiming the user/domain, but are failing the verification to mark them as dangerous keys and/or potential impostors/scammers. The query is just a way to pre-compute that information and make it ready for when clients ask to verify an account with them.

[Semisol]

I don't think it is. NIP-05 is optional.

But it is? It is about as important, or even more so than their display name. NIP-05 is the username system on Nostr.

Because Web of Trust providers would like to know all keys that are claiming the user/domain, but are failing the verification to mark them as dangerous keys and/or potential impostors/scammers. The query is just a way to pre-compute that information and make it ready for when clients ask to verify an account with them.

Why is this actually needed? A NIP-05 can fail for many reasons and it failing doesn't mean the user is automatically trying to impersonate someone.

Reputation algorithms shouldn't use a misconfigured profile as a scammer indicator, and even then this approach would require the reputation algorithm to query all NIP05s anyway.

What would normally happen is clients never show the invalid NIP-05, problem solved.

[vitorpamplona]

But it is? It is about as important, or even more so than their display name. NIP-05 is the username system on Nostr.

No, nobody is really using NIP-05 for anything other than a little symbol in the profile page of the user. Definitely, no one is using it as a username.

Why is this actually needed? A NIP-05 can fail for many reasons and it failing doesn't mean the user is automatically trying to impersonate someone.

Agree, but if I am a WoT provider and my customer Alice has a valid nip05, I can get everyone else with the same nip05 and mark them as an impersonator.

It's just pre-caching/pre-computing. Impostors are usually not in the follows of follows.. network of a user. So, WoT can find additional sources of information by checking the same name, same ln address, same nip-05, etc in other users. They can decide to downgrade other users as they see fit.

The single letter tag here just makes it possible.

What would normally happen is clients never show the invalid NIP-05, problem solved.

That also happens. But that is not everything NIP-05 allows for.