Ability to spin up a full rippled server from EBS snapshot

[ciaranmcveigh]

Key points 

• I have an aws EBS snapshot of the full ripple ledger history and can spin up an EBS volume using this snapshot
• I would like to use this snapshot to quickly spin up a full ripple ledger once a week for the purpose of letting it sync up to the latest block (ie if i ran rippled server_info then complete_ledgers will be 32570-latest_block) to take a snapshot of the db associated data located at mnt/data and then tear down the instance
• Essentially I want an up to date EBS snapshot of the full ripple ledger while reducing hosting cost to the smallest amount 

I don't want to use the --import functionality with [import_db] in the config as this requires me to load the data to a new location. For example in my case the snapshot data is at /mnt/data/rippled/db and the new location may be /var/lib/rippled/db. The full ledger data is 10TB+ so if I want to use import i would need double the storage, 50% for the import folder /mnt/data/rippled/db and 50% on the standard location  /var/lib/rippled/db (or some other location on the mounted volume /mnt/data).

ideally i want to just mount the volume to mnt/data on the instance so that the db is available at /mnt/data/rippled/db, point [node_db] path=/mnt/data/rippled/db in the config and have the rippled process recognise that data is already available at that location and just begin syncing forwards (from latest retrieved block) or backwards (from tip to latest retrieved block) on the volume so that i only require one set of data and so that the ledger data can be updated on the volume.

I have tried the set up just mentioned and the logs just return and hang on the "Auto-tuning to 4 validation....", the rippled process is run in a container using https://hub.docker.com/r/xrptipbot/rippled, then container command is  "exec /opt/ripple/bin/rippled --net --conf /etc/opt/ripple/rippled.cfg"

Existing rippled config at host /config/, using them.
Loading: "/etc/opt/ripple/rippled.cfg"
Watchdog: Launching child 1
2020-Apr-11 16:03:19.583645063 UTC Application:NFO process starting: rippled-1.5.0
2020-Apr-11 16:03:19.583712380 UTC JobQueue:NFO Auto-tuning to 4 validation/transaction/proposal threads.

the config i have used is below

 

    [server]
    port_rpc_admin_local
    port_peer
    port_ws_admin_local


    [port_rpc_admin_local]
    port = 5005
    ip = 0.0.0.0
    admin = 127.0.0.1
    protocol = http

    [port_peer]
    port = 51235
    ip = 0.0.0.0
    protocol = peer

    [port_ws_admin_local]
    port = 6006
    ip = 127.0.0.1
    admin = 127.0.0.1
    protocol = ws

    [node_size]
    huge


    [node_db]
    type=NuDB
    path=/mnt/data/rippled/db/nudb

    [database_path]
    /mnt/data/rippled/db

    [debug_logfile]
    /var/log/rippled/debug.log

    [sntp_servers]
    time.windows.com
    time.apple.com
    time.nist.gov
    pool.ntp.org


    [validators_file]
    validators.txt

    [rpc_startup]
    { "command": "log_level", "severity": "info" }

    [ssl_verify]
    1

    [ips]
    XXXXX
    XXXXX

    [ips_fixed]
    XXXXX
    XXXXX
    XXXXX

    [ledger_history]
    full

Any help in this is much appreciated, let me know if more details needed  

Thanks