[Pitch] Custom `base_node` data storage location

[shanimal08]

Base node data (only) | Settings and Storage Location Configuration

Problem

The base_node data takes up a lot of storage and we'd like to give users the option to set a custom storage location for it.

Scope

1. the ability to pick a custom location
   • simple flow:
     • this will be a file explorer interface of your available directories nested withing your $HOME directory
   • less simple (should you want to go beyond or outside of $HOME):
     • typing a path relative to $HOME
     • validating said path on the rust side
2. new config_core items for the path - defaults to the current data_dir
3. updating all relevant processes to suit
   • tari core changes [tbd]
   • all node adapter items
4. new processes to handle migration for existing users
   • if possible migration should be "copy" (to the new location) then "remove" (from current location, once "copy" completes succesfully)
   • all current processes should be "frozen" to prevent issues
   • once complete, TU should be restarted

First time users

• IF applicable: the installer should allow for selecting where to store this data. Similar to what is outlined in the pitch [Pitch] Install Directory Settings #2304
• If not applicable (likely on Mac .dmg): before starting up any of the processes the user should be prompted to either pick a location (Scope.1), or use the default

Existing users

in Settings:

1. the ability to change the location (Scope.1)
2. post-change validation
   • check new location space available compared to current data size (should we outright exclude dirs with inadequate space from the list?)
3. if validation passes, start migration (Scope.4)

Appetite - Preliminary ESTIMATE

⚠️ NOT FINAL - all of this will need to be investigated further for more accurate estimates

FE

item	days
new (in-app) UI	3
new (installer) UI	2
integration	5

BE

item	days
config updates	1
location validation logic	1
process updates to use new path var	1.5
migration logic	4-6

Other

item	days
CI & installer updates	2-3
tari core changes	?
testing - cross platform	2

Risks

• Data migration requires sufficient disk space; failure scenarios need graceful handling.
• Installer modifications on Windows may introduce complexity or require additional testing.
• possible data corruption - i.e in the case of background proccesses not exiting properly before migration
• compromising security if allowing migration beyond $HOME dir

Questions

• what's the cut-off migration size?
• have we missed any other risks?
• do we still need to do this if we decide to rather go ahead with Pitch [Pitch] Compressed chain download #1883
• have we missed alternative simpler solitions

Who’s Involved

To be assigned after review.

[brianp]

Base node already has a configurable option to change the chain data location:

-p "base_node.data_dir=/absolute/path/to/chain/data"

As for step 4:

• Regardless of move or copy
• Technically, we only need to ensure the local node is off to change this directory. But I'm not against us forcing a shutdown of all services. It makes it simple. Perform the action, and then start back up.

Questions:

what's the cut-off migration size?

It would be based on the available hard drive size, if we went the non destructive route of copying. We would want current db size + (total HD size * 0.05) to ensure we leave 5% of the hard drive available in a worst case scenario.

In a move scenario I don't think it matters.

do we still need to do this if we decide to rather go ahead with Pitch #1883

Yes this would actually become even more relevant as users would begin to care more where the directory is.