Private network configuration

[ETeOr]

Hello!

I have created a small, private test Ripple network (5 servers). However, they are out of sync and have been for many hours now. The error message I get:

2018-Oct-04 13:54:42.298393674 NetworkOPs:WRN We are not running on the consensus ledger
2018-Oct-04 13:54:42.298722554 LedgerConsensus:WRN Need consensus ledger E03490EAA4F33437A09E84C699ACF7E76DC1D808B5E1FEADE797A00124A0A0C2
2018-Oct-04 13:54:43.298152533 LedgerConsensus:WRN View of consensus changed during open status=open,  mode=wrongLedger
2018-Oct-04 13:54:43.298293566 LedgerConsensus:WRN E03490EAA4F33437A09E84C699ACF7E76DC1D808B5E1FEADE797A00124A0A0C2 to A49A59249074C2CFE62C95D6406CE9C0A86BA51294F86BA3341481D85A758A28
2018-Oct-04 13:54:43.298403323 LedgerConsensus:WRN {"accepted":true,"account_hash":"66B59C61A62025880A06884F84A44E79EE4540F447C8DAD8855371DFB42BC7D6","close_flags":0,"close_time":591976480,"close_time_human":"2018-Oct-04 13:54:40.000000000","close_time_resolution":10,"closed":true,"hash":"A49A59249074C2CFE62C95D6406CE9C0A86BA51294F86BA3341481D85A758A28","ledger_hash":"A49A59249074C2CFE62C95D6406CE9C0A86BA51294F86BA3341481D85A758A28","ledger_index":"726","parent_close_time":591976460,"parent_hash":"783B209EA04B5C6D103E8B338E9D66F64161A299975E48F1ADD7EB10BA4A0509","seqNum":"726","totalCoins":"100000000000000000","total_coins":"100000000000000000","transaction_hash":"0000000000000000000000000000000000000000000000000000000000000000"}
2018-Oct-04 13:55:02.302938772 NetworkOPs:WRN We are not running on the consensus ledger
2018-Oct-04 13:55:02.303189234 NetworkOPs:ERR JUMP last closed ledger to E03490EAA4F33437A09E84C699ACF7E76DC1D808B5E1FEADE797A00124A0A0C2

Here is server_info output:

{
   "result" : {
      "info" : {
         "build_version" : "1.1.0",
         "closed_ledger" : {
            "age" : 6,
            "base_fee_xrp" : 1e-05,
            "hash" : "FA7083560E8E69CA3525BF05C8A26157B38244B90DDC28B428EFF660BA9DF4DA",
            "reserve_base_xrp" : 200,
            "reserve_inc_xrp" : 50,
            "seq" : 72
         },
         "complete_ledgers" : "empty",
         "hostid" : "ip-172-31-37-107",
         "io_latency_ms" : 1,
         "jq_trans_overflow" : "0",
         "last_close" : {
            "converge_time_s" : 2,
            "proposers" : 0
         },
         "load" : {
            "job_types" : [
               {
                  "in_progress" : 1,
                  "job_type" : "clientCommand"
               },
               {
                  "job_type" : "peerCommand",
                  "per_second" : 2
               }
            ],
            "threads" : 3
         },
         "load_factor" : 1,
         "peer_disconnects" : "2",
         "peer_disconnects_resources" : "0",
         "peers" : 4,
         "pubkey_node" : "n94ELmtk3yy8f6Xhsa4A2xaDinx12idhHRzFX5ypSccAX56A9LDm",
         "pubkey_validator" : "nHDEy6XoYhNXAku6LPXcYx9HgX346M4HvtwH4hxAMKhZLpszrRpe",
         "published_ledger" : "none",
         "server_state" : "full",
         "state_accounting" : {
            "connected" : {
               "duration_us" : "713145358",
               "transitions" : 2
            },
            "disconnected" : {
               "duration_us" : "1505936",
               "transitions" : 1
            },
            "full" : {
               "duration_us" : "16451157252",
               "transitions" : 2
            },
            "syncing" : {
               "duration_us" : "0",
               "transitions" : 0
            },
            "tracking" : {
               "duration_us" : "2",
               "transitions" : 2
            }
         },
         "time" : "2018-Oct-04 14:24:06.237292",
         "uptime" : 17165,
         "validation_quorum" : 4,
         "validator_list_expires" : "never"
      },
      "status" : "success"
   }
}

and rippled.cfg from one of them:

[server]
port_rpc_admin_local
port_peer
port_ws_admin_local
port_ws_public
#ssl_key = /etc/ssl/private/server.key
#ssl_cert = /etc/ssl/certs/server.crt

[port_rpc_admin_local]
port = 5005
ip = 127.0.0.1
admin = 127.0.0.1
protocol = http

[port_peer]
port = 51235
ip = 0.0.0.0
protocol = peer

[port_ws_admin_local]
port = 6006
ip = 127.0.0.1
admin = 127.0.0.1
protocol = ws

[port_ws_public]
port = 5005
ip = 172.31.37.107
protocol = ws

#-------------------------------------------------------------------------------

[node_size]
tiny

[node_db]
type=RocksDB
path=/var/lib/rippled/db/rocksdb
open_files=2000
filter_bits=12
cache_mb=256
file_size_mb=8
file_size_mult=2
online_delete=2000
advisory_delete=0

[database_path]
/var/lib/rippled/db

[debug_logfile]
/var/log/rippled/debug.log

[sntp_servers]
time.windows.com
time.apple.com
time.nist.gov
pool.ntp.org

[ips_fixed]
34.219.220.245 51235
54.188.106.56 51235
34.217.59.170 51235
34.219.111.253 51235

[peer_private]
1

[cluster_nodes]
nHBuwJ7HMxYu9PvMk9d8YhrxGBw3SsvBvBvmbRbvqQhhK1XT5cVi
nHUJ7a7KPisdHyPVAC7qVB1FJBDjMdU5rLrdnimp8seeVpPpDgFJ
nHUcdhz8J214e93CrF4uVfpH5KUa1iLPWbh2k2FywuYr1BEmupn6
nHUXa2qMhA77L2c9jpqEXcUMujBEsKy1yAeT2LuJAo9ff54Tvpgr

[validators_file]
validators.txt

[rpc_startup]
{ "command": "log_level", "severity": "warning" }

[ssl_verify]
1

[validator_token]
eyJtYW5pZmVzdCI6IkpBQUFBQUZ4SWUzNi94OFp6NWpwdmRaVXlCV1JGZHpnSzdHcms1UTF1
RGhmQVBqSGN0MFpTbk1oQXZyZy9rTml1QVI0U3hLS1I3WGM5UjMya3RXU2pjcWJ3RjBrMW9O
L3JpTVlka1l3UkFJZ01hR0xlaWtQbmx4OWoveXZqSzd5djk0a3ZnblRkRTNYUVZtNmlRK3ll
djhDSURDcHJqTk02T1k0VllVcTc1MGxFdW1LczQ2bjFadlB6alRqSzdxUkpyeGljQkpBaWZa
Zk1JZWJqY0RMNHJTQm9JeUlYOWR5VXJRRzhQcDMyUXRuWWdaRU81elVrdUg5YXFEWjg5MDM5
cHNLTGVPYTZaNEFWTVhsWEdSYS9iSWtKMmYvQVE9PSIsInZhbGlkYXRpb25fc2VjcmV0X2tl
eSI6IkE3RDA5MzJEMTVBMjZDNjIyQTJFOUYzQzkzNkU3OUNCQjVBNTcyMjUyM0Y4MEM2MTFG
NUQxMDlEQzZCQTE0MDUifQ==

 What could be the cause of this problem (in particular, the "JUMP last closed ledger" error)? How can I make sure that the servers are properly configured? 

[PunishmentOfLuxury]

You might do better to post this in Technical Discussion.