feat: add new fields for the ruler file

[SPFabGerman]

At the moment there is a weird discrepancy which information can be displayed where. Some information can only be shown in the info column (with the set info command) and others only in the status / ruler line below (with set statfmt or the rulerfile),
This PR aims to make all information consistently available everywhere, so that users can display information where they want to. (Coincedentally this also fixes #2325.) And as a little bonus I also added some sortby options for the new information.

In detail I added the following:

• for the info option: lcount (link count), target (link target) and rsize (real size)
• for the sortby option: user, group, lcount, target, target-natural, rsize
• for the Rulerfile: Extension, AccessTime, BirthTime, ChangeTime, DirCount, TotalSize, CustomInfo
• for the statfmt: %f (name), %F (path), %e (extension), %z / %Z (total size), %A /%B / %C ([abc]time), %i (custom info)

Additionally I also added Directory, User and Host to the Rulerfile field to match what promptfmt provides. These may come in handy for some users (for example as if conditions), especially as there are no ways to customize, reload or change a Rulerfile during runtime. (Similarly, the new Stat.Extension can for example be used to provide colors for file names.)
The CustomInfo field is especially noteworthy as it allows users to customize the entirety of the rule line via shell scripts, far exceeding even what the current Rulerfile offers.

However, there are some things that should be noted and improved before merging, but I wanted to first gather feedback and opinions:

• The new rsize option and Size field are pretty useless if compared to size / TotalSize, as they only differ for directories and give each directory the same size. So they offer no real advantages over the existing size option / new TotalSize field.
• It might be worth considering to split the existing size info-option / sortby-option into two, one for dircounts and one for total dir size (this is for example already done in statfmt and the Rulerfile) and get rid of the dircounts configuration option entirely. Then we would only calculate the dircounts on demand, so when the option is actually used in info, sortby, statfmt or the Rulerfile.
• For the target information I also added the -> arrows, as was suggested by [Feature Request] Symlink reference #2325. I think they look nice.
• The target information is prone to being entirely hidden when it's too long. I'm not sure if that is the intended effect or if it would be better to truncate it.
• For the user / group sortby-options it might be better to sort by user / group id instead of name.
• I used the existing linkCount() function to get link counts. But this function returns strings instead of integers (for some reason unbeknownst to me). This causes issues with sorting (though these could probably be resolved by using natural sort instead) and also means we can't make any integer comparisons with it in the Rulerfile.
• For the target sortby-options I opted to order all symlinks first, instead of last. It seems natural to me that when a user wants to sort by symlinks they are probably mainly interested in symlinks, so they should appear first.
• For the TotalSize Rulerfile field and statfmt pattern it might be better to provide explicit failure codes (similar to the DirCount field) instead of having a 0-fallback.
• I haven't implemented a sortby option for Permissions, as there is no single good way to sort them.

Documentation is still missing, but I can add that when all these things are resolved and finalized.

[CatsDeservePets]

Hello @SPFabGerman 👋
Thank you for trying to improve lf for everyone, it is always appreciated.
Both ruler/stat and info are topics I am invested in and contributed to in the past.
Here are some of my thoughts:

In detail I added the following:

• for the info option: lcount (link count), target (link target) and rsize (real size)
• for the sortby option: user, group, lcount, target, target-natural, rsize
• for the Rulerfile: Extension, AccessTime, BirthTime, ChangeTime, DirCount, TotalSize, CustomInfo

Even though being related to each other, I'd prefer having separate PRs for info, sortby and rulerfile.

• for the statfmt: %f (name), %F (path), %e (extension), %z / %Z (total size), %A /%B / %C ([abc]time), %i (custom info)

The rulerfile is intended to eventually replace the existing rulerfmt/statfmt options, following a deprecation process. That means, there are no plans for introducing any new features to either rulerfmt or statfmt (I even did create a PR to add %f myself to statfmt but ditched it in favour of rulerfile).

Additionally I also added Directory, User and Host to the Rulerfile field to match what promptfmt provides.

I case you didn't know, you can simply use environment variables inside the rulerfile like this:

{{env "PWD"}}
{{env "USER"}}
{{env "hostname"}}

These may come in handy for some users (for example as if conditions), especially as there are no ways to customize, reload or change a Rulerfile during runtime. (Similarly, the new Stat.Extension can for example be used to provide colors for file names.)

The rulerfile currently not being reloadable during runtime is something I dislike myself. Two possible solutions for this would be:

• Introducing a command similar to source that works for the rulerfile instead of the lfrc
• Converting the rulerfile option from bool to string, matching the previewer and cleaner options (this is the solution I'd prefer personally, but I am open for discussion)

The CustomInfo field is especially noteworthy as it allows users to customize the entirety of the rule line via shell scripts, far exceeding even what the current Rulerfile offers.

The custom info is actually the first big thing I contributed to lf. Without that, I would probably not have gotten into lf development. I mainly use it to display the git status of files, but it has such a huge potential and I love seeing people actually using it.

That being said, you don't need a CustomInfo property to display custom information generated by shell scripts. It already supports user defined strings (so-called 'user options'). I use them myself to show the current git branch.
This is the part of my ruler using an user option called 'ref':

{{with $.UserOptions.ref}} {{printf "(%s)" .}}{{end -}}

I set the variable inside my lfrc like this:

cmd on-cd &{{
    ref=""
    if git rev-parse --is-inside-work-tree >/dev/null 2>&1; then
        ref=$(git rev-parse --abbrev-ref HEAD 2>/dev/null)
        if [ "$ref" = "HEAD" ]; then
            ref=$(git rev-parse --short HEAD 2>/dev/null)
        fi
    fi
    lf -remote "send $id :set user_ref '$ref'"
}}

And here you can see it in action :D

• The new rsize option and Size field are pretty useless if compared to size / TotalSize, as they only differ for directories and give each directory the same size. So they offer no real advantages over the existing size option / new TotalSize field.

To be honest I really don't see a point in introducing rsize. One can either enable dircounts to basically get the link count or not. There is even a calcdirsize command to display the actual dir size in info.

• It might be worth considering to split the existing size info-option / sortby-option into two, one for dircounts and one for total dir size (this is for example already done in statfmt and the Rulerfile) and get rid of the dircounts configuration option entirely. Then we would only calculate the dircounts on demand, so when the option is actually used in info, sortby, statfmt or the Rulerfile.

We try to avoid breaking user configs by removing settings like this. Also, we already only calculate/populate the property when dircounts is enabled. I am uncertain about the rulerfile situation. Should setting dircounts have an affect on .Stat.Size so it displays the dircount instead? And should calcdirsize work in the same way? Or should we add new fields?

• For the target information I also added the -> arrows, as was suggested by [Feature Request] Symlink reference #2325. I think they look nice.
• The target information is prone to being entirely hidden when it's too long. I'm not sure if that is the intended effect or if it would be better to truncate it.

lf always expects info columns to have the same length. For columns with variable len like custom or group, some special logic has to be implemented to keep the columns aligned. Take a look at printDir().
I also did some work on #2325 (nothing pushed yet) and I really don't think that it should be an info property. Due to the length constraints given to info fields which result in a blank info and due to the fact that target should ideally always be next to the filename, we should modify the displayed name directly instead of it being a property.

Let's compare these two approaches with both also trying to show size and time:

target in info (taking up too much space)	target in filename (can be truncated)

I'd prefer working on this myself, I hope that is ok. The only real question I have to figure out is how truncation should work here, but that is another story...

Also, I really don't think there is any value whatsoever in being able to sort by target. I can construct a theoretical use case but don't see anyone actually doing that. Just because something can be implemented, it does not mean we should. Every single, little option makes the code less maintainable.

• For the user / group sortby-options it might be better to sort by user / group id instead of name.

Again, I don't see many people actually using these for sorting. Neither ranger, joshuto, or yazi support any of these. On a side note, I have seen different users wanting to sort by tags (#2256). In my personal opinion, we should rather support this.

• I used the existing linkCount() function to get link counts. But this function returns strings instead of integers (for some reason unbeknownst to me).

The reason being that it is only ever used in the context of displaying a string. It is never used as a number.

---

To wrap it all up, I honestly don't see a real use case for some of the changes while others are either already possible or need to be discussed.
I will happily accept a PR adding Extension, AccessTime, BirthTime and ChangeTime to the rulerfile.

Regarding your other changes, I am open for discussion and perhaps @joelim-work has completely different opinions here.

[SPFabGerman]

Hi, thanks for your feedback @CatsDeservePets.
I generally agree with many of your points. My main interest and motivation with this PR was extending the capabilities of the Rulerfile, as I noticed some limitations while creating my own Rulerfile. Everything else was really just an extra for the sake of parity. So I have no problem limiting the scope only to extending the Rulerfile. If there are no objections, I would close this PR and open a new one solely for the Rulerfile. (Basically just the first commit made here.)

Other than that, I agree that allowing to set the rulerfile option to the actual file that should be used is probably a good idea. Even just allowing to set a different file during initialization can provide great benefits.

[CatsDeservePets]

Hi, thanks for your feedback @CatsDeservePets. I generally agree with many of your points. My main interest and motivation with this PR was extending the capabilities of the Rulerfile, as I noticed some limitations while creating my own Rulerfile. Everything else was really just an extra for the sake of parity. So I have no problem limiting the scope only to extending the Rulerfile. If there are no objections, I would close this PR and open a new one solely for the Rulerfile. (Basically just the first commit made here.)

You can either close this PR and create a new one or just revert the other commits (they won't appear in lf's commit history due to git squash) and change the title of the PR. Both is fine.

Other than that, I agree that allowing to set the rulerfile option to the actual file that should be used is probably a good idea. Even just allowing to set a different file during initialization can provide great benefits.

Yeah, I think so too but that should be done in yet another PR.

[joelim-work]

I largely agree with what @CatsDeservePets has said above. I will just add a few notes below:

• The list of fields supported in the info column and ruler is generally just based on user demand. Over time users made requests to add a field to either the info column or ruler, and support was added, but we never made a proper effort to maintain parity between the two.
• I am fine with adding the fields mentioned above to the ruler file. When implementing it, I only added the fields from rulerfmt, which again was based on user demand.
• I do not mind making the ruler file dynamic so that it can be changed during runtime. But I do not like using the existing source command for this as it implies that the file will contain further lf configuration syntax to be executed. I consider it to be a low priority feature though, and perhaps it can wait until the ruler file has fully replaced rulerfmt. Alternatively you can make it so that rulerfile loads the given ruler template, and not providing a path (or a blank path) will fall back to rulerfmt (or the default ruler after rulerfmt has been removed).
• Tags was added in Add tags #791. For the most part it is simply a display marker that is assigned to a particular file. I don't think anyone at the time considered the possibility that it could be used for sorting/filtering, but I will post in tagged folders/files at top #2256 since it is not relevant to this PR.

[SPFabGerman]

@CatsDeservePets As discussed I limited this PR to the first commit, which only adds the new fields to the Rulerfile.
However, I kept the CustomInfo field, as I think it is quite useful to have and only adds one line, so it doesn't add any real maintenance burden. While the suggested approach using user options can accomplish the same thing, I believe that CustomInfo is easier to use (especially when using a different custom info per file) and has the advantage that information can be easily generated asynchronously in the background, which is not as easily possible with user options.

I also added a second commit thats attempting to combine the existing Size field with the new DirCounts and TotalSize fields. It now behaves just like the size field in the info column, displaying the dircounts if the option is set and total dir size if it is available.
I implemented it to also return the negative numbers to communicate error codes, which the user might want to match against. This however has one small problem because the humanize function expects unsigned integers and so cannot currently be applied to the Size field. I see a few possible solutions to this:

1. We change the Size field back to a uint64 and simply set the Size field to 0 on errors. I don't like this, as it makes errors indistinguishable from empty files or directories.
2. Same as 1, but we also add an extra field SizeError. But having two fields for just one information seems cumbersome.
3. We change humanize to expect int64 and simply return negative values unchanged.
4. We change humanize to expect int64 and handle negative values with appropriate strings, similar to how it is done in the info column.

Personally I think option 3 or 4 is best, especially to avoid breaking user configurations.

One final thing to note is that with this change the Size information for directories changes, which could technically be considered a breaking change. Previously directories were always reported with a constant size (4096 on my system). This is identical to what ls does. Now the Size field is negative by default. This is more in line to what many modern ls replacements, like for example eza, do.
Also, it doesn't make sense to apply the humanize function if dircounts option is set. However, users can simply check if that option is enabled in the rulerfile itself and then apply the function conditionally.

[CatsDeservePets]

@CatsDeservePets As discussed I limited this PR to the first commit, which only adds the new fields to the Rulerfile. However, I kept the CustomInfo field, as I think it is quite useful to have and only adds one line, so it doesn't add any real maintenance burden. While the suggested approach using user options can accomplish the same thing, I believe that CustomInfo is easier to use (especially when using a different custom info per file) and has the advantage that information can be easily generated asynchronously in the background, which is not as easily possible with user options.

I guess I am fine with that. I agree that custom information per file are much simpler using addcustominfo.

I also added a second commit thats attempting to combine the existing Size field with the new DirCounts and TotalSize fields. It now behaves just like the size field in the info column, displaying the dircounts if the option is set and total dir size if it is available. I implemented it to also return the negative numbers to communicate error codes, which the user might want to match against. This however has one small problem because the humanize function expects unsigned integers and so cannot currently be applied to the Size field. I see a few possible solutions to this:

1. We change the Size field back to a uint64 and simply set the Size field to 0 on errors. I don't like this, as it makes errors indistinguishable from empty files or directories.
2. Same as 1, but we also add an extra field SizeError. But having two fields for just one information seems cumbersome.
3. We change humanize to expect int64 and simply return negative values unchanged.
4. We change humanize to expect int64 and handle negative values with appropriate strings, similar to how it is done in the info column.

Personally I think option 3 or 4 is best, especially to avoid breaking user configurations.

One final thing to note is that with this change the Size information for directories changes, which could technically be considered a breaking change. Previously directories were always reported with a constant size (4096 on my system). This is identical to what ls does. Now the Size field is negative by default. This is more in line to what many modern ls replacements, like for example eza, do. Also, it doesn't make sense to apply the humanize function if dircounts option is set. However, users can simply check if that option is enabled in the rulerfile itself and then apply the function conditionally.

I am really unsure about this one. Let's quickly compare it to the way the info column handles file sizes:

if f.IsDir() && getDirCounts(d.path) {
    switch {
    case f.dirCount < -1:
        info.WriteString("     !")
    case f.dirCount < 0:
        info.WriteString("     ?")
    case f.dirCount < 10000:
        fmt.Fprintf(&info, " %5d", f.dirCount)
    default:
        info.WriteString(" 9999+")
    }
    continue
}

var sz string
if f.IsDir() && f.dirSize < 0 {
    sz = "-"
} else {
    sz = humanize(uint64(f.TotalSize()))
}
fmt.Fprintf(&info, " %5s", sz)

The biggest difference being that the info column calls humanize on demand while rulerfile leaves it up to the user whether they want the 'raw' numbers or the human readable version by explicitly piping the size into the humanize function (which is what the default ruler does).

I think there is no way around changing .Stat.Size to uint64. Perhaps we could change the template.FuncMap so humanize calls a wrapper function which takes uint64. This wrapper could mostly follow the logic of info column. I am not sure whether this will work as I imagine it to (as it needs access to nav) and it will lead to code duplication for sure.

Other options are just leaving everything size related as it is or introduce another field. I don't have a good solution here.

[SPFabGerman]

I don't think creating a wrapper for humanize that follows the logic of the info column is a good idea, as the behaviour of the wrapper would depend on the dircounts option. Exposing a function for the template that is dependent on some external option would likely lead to a lot of confusion, as humanize is not a pure function anymore.
This is partly the reason why I originally had an additional field DirCounts. This allows the humanize function to remain pure, but means the user now has to implement additional logic if he wants the dir counts information instead of the total size. And it also doesn't allow us to communicate errors in a reasonable way, which I deem pretty essential.

[CatsDeservePets]

Honestly I'd say we just don't care about the size right now and perhaps add a ˋDircountˋ string property on user demand only.

[SPFabGerman]

As suggested I removed the DirCount information from the Size field. But I kept the dirSize information, if we have a directory.

I also added a small wrapper for the humanize function that handles negative values. This means that now the existing rulerfile with {{.Size | humanize | printf " %5s" -}} just works as expected for all cases. File sizes are displayed as usual, Directory sizes are displayed with "-" by default and with the actual size after an calcdirsize has been performed. This behaviour is now identical with the size attribute for info column, if the dircounts option is disabled. I think this is the best solution and should cause no problems for users.
It might be worth to include this change directly into the humanize function instead of creating a wrapper. But as the humanize function is used in some other places in the code too, I didn't want to do that without consultation. As far as I can see, implementing this change into the real humanize function should not cause any problems, as the humanize function is never called with negative values anyway.

[CatsDeservePets]

Hello again, @SPFabGerman!

Sorry for all the back and forth here. I'd prefer not to include the changes to the size property in this PR, as this would be considered a breaking change (directories now showing - instead of the link size, unless calcdirsize is used). Feel free to open a separate PR regarding the size property for further, in-depth discussion.

Thanks for your patience and work so far.

---

Btw, if you prefer displaying - instead of the link size of a directory, you could just replace this

{{.Size | humanize | printf " %5s" -}}

with this

{{if eq (substr .Permissions 0 1) "d" -}}
    {{"-" | printf " %5s" -}}
{{else -}}
    {{.Size | humanize | printf " %5s" -}}
{{end -}}

You could also use .LinkCount which is not the same as dircount but close.

[SPFabGerman]

Sorry for all the back and forth here. I'd prefer not to include the changes to the size property in this PR, as this would be considered a breaking change (directories now showing - instead of the link size, unless calcdirsize is used). Feel free to open a separate PR regarding the size property for further, in-depth discussion.

Sounds good. I will open a new PR for the Size field once this is merged.

Btw, if you prefer displaying - instead of the link size of a directory, you could just replace this

{{.Size | humanize | printf " %5s" -}}

with this

{{if eq (substr .Permissions 0 1) "d" -}}
    {{"-" | printf " %5s" -}}
{{else -}}
    {{.Size | humanize | printf " %5s" -}}
{{end -}}

You could also use .LinkCount which is not the same as dircount but close.

I know and I do this already in my own config. But I would also really like to see the directory size in the ruler, which is partly why I created this PR in the first place.

[CatsDeservePets]

Looks good. Do you want to update the docs as described in https://github.com/gokcehan/lf/blob/master/CONTRIBUTING.md as well as the CHANGELOG.md?
I can also do this myself after merging.

[SPFabGerman]

I added the documentation and changelog.

[CatsDeservePets]

Thanks once again. Looking forward to a separate issue regarding the size.

[joelim-work]

This doesn't sound like a bug fix, should this be in the Added section of the changelog?

[CatsDeservePets]

Obviously! I did not even notice!

[CatsDeservePets]

Fixed in #2346