Jump to content
Sign in to follow this  
fh13

Simulations using ripple-lib and historical data

Recommended Posts

Hi,

i want to do some simulations on some scripts I wrote using the ripple-lib API with historical data.

So my question is: Is it possible to simulate a rippled server that fast forwards through historical ledger version that were downloaded previously?

I couldnt find anything like that on github so...

Any other ideas are appreciated! ;)

Thank you

Share this post


Link to post
Share on other sites
1 hour ago, fh13 said:

So my question is: Is it possible to simulate a rippled server that fast forwards through historical ledger version that were downloaded previously?

Is this what you are looking for perhaps?

https://ripple.com/build/data-api-v2/#get-ledger

https://ripple.com/build/rippleapi/#getledger

For data integrity on your end however, depending on if you are developing an enterprise app here or not, you should consider downloading the full ledger history.  By doing so, you can run the command ./rippled --conf /etc/rippled/rippled.cfg --load <ledger_value> to begin the process and carry out your simulations in a much more controlled environment instead of fetching data from one of the ripple servers.  I would not strongly suggest this unless your organization is capable of footing the the server costs associated with ~3-4TB of SSD disk space an at least 8GB of RAM for a medium size node.

Share this post


Link to post
Share on other sites

Thank you for your response.

Well the data API isnt quite what I am looking for. Your second approach looks way more suitable for me.

Particulary Im interested in the orderbook and some calculations I do on them.

I have to questions to your second approach:

  1. With rippled is it possible to fetch a specific range of ledgers from the network ? If so, how do I do that ?
  2. When the range of ledgers that are of interest to me are fetched, how do I connect to the rippled with my ripple-lib client while loading the local ledger versions, and does ripple-lib still emit the "ledger closed" event ? probably not ?!

If possible I want to "replay" the historical ledgers, while being connected to the rippled like usual.

Share this post


Link to post
Share on other sites

@Sukrim or @JoelKatz will probably be able to provide you with more assistance here, however, I can answer some of your inquiries here.  First, no, rippled must bootstrap a fetch pack as defined within your configuration, I believe that the default is 2000 ledgers.  The process will continue to progress if consensus is agreed upon, so new ledgers will be generated as the historical data back-fills.  You can load rippled with --net, --start, and --load.  I believe that load will attempt to load the process from a specified ledger value, --start simply begins the process to read the configuration file an either fails or succeeds to begin validating/operating in stock mode, while --net attempts to back-fill the server once it connects to the network and determines the current ledger.

This RippleAPI resource may be able to assist you with what you are attempting to build.  If you would like a template to refactor for what you are attempting to build, you can use the source code for using ripple-lib on your web site here.  One of the above or another member of XRPchat can provide a much more satisfactory response in regard to the remainder of your post besides my speculations; I'm not sure how promises entirely operate as of yet.

Edited by Twarden

Share this post


Link to post
Share on other sites

Unfortunately the --load options almost isnt documented at all. At least from what I have found.

From the ripple wiki:

Quote

Load the locally stored ledger.

What do you mean with "ledger_value"? Is that the version number I can pass to rippled that I want it to load?

My idea is to do the following:

  1. Run ./rippled --net and
  2. send ledger_request commands until the data that is important to me is fetched and saved.
  3. Run ./rippled --load <x> --standalone
  4. connect the ripple-lib client to the rippled server
  5. send ledger_accept commands in order to go to the next saved ledger

I seriously doubt this works, but maybe now it is clear what I want to do at least.

Share this post


Link to post
Share on other sites
13 minutes ago, fh13 said:

I seriously doubt this works, but maybe now it is clear what I want to do at least.

Why this shouldn't work?

Another option is to (programmatically) mock objects that you use to access rippled API functions. Example of JS library that supports mocking: sinon.js.

You can write much more comprehensive tests with mocks than with live rippled.

 

 

 

 

 

 

Share this post


Link to post
Share on other sites
3 minutes ago, T8493 said:

Why this shouldn't work?

Because I was thinking calling ledger_accept  wouldnt load the next ledger, from the ones I saved previously. Or does it ?

 

Share this post


Link to post
Share on other sites
38 minutes ago, fh13 said:

Because I was thinking calling ledger_accept  wouldnt load the next ledger, from the ones I saved previously. Or does it ?

Do you want to query exact state of historical ledgers? Or do you want to submit your own transactions and see how they influenced historical ledgers?

Share this post


Link to post
Share on other sites

I have some trading scripts, and want to see how they would have performed if I had used them in the past in order to tweak them and get a conclusion how they will perform on the live network.

So yeah I want to replay the real ledgers, since I could influence my results by submitting my own transactions.

And because I am using the ripple-lib-orderbook extension I guess it would be way to complex to implement a mock server that replays historical ledgers.

Share this post


Link to post
Share on other sites
22 minutes ago, fh13 said:

I have some trading scripts, and want to see how they would have performed if I had used them in the past in order to tweak them and get a conclusion how they will perform on the live network.

So yeah I want to replay the real ledgers, since I could influence my results by submitting my own transactions.

And because I am using the ripple-lib-orderbook extension I guess it would be way to complex to implement a mock server that replays historical ledgers.

So basically you're asking whether accept_ledger (or some other command) replays real transactions (transactions that are present in the "real" ledger and weren't submitted by you) in addition to your own transactions?

Edited by T8493

Share this post


Link to post
Share on other sites
2 minutes ago, T8493 said:

So basically you're asking whether accept_ledger (or some other command) replays real transactions (transactions that are present in the "real" ledger and weren't submitted by you)?

Excactly !

Share this post


Link to post
Share on other sites

Well kind of...

First I want to implement some scripts to do some statistical measurements on specific orderbooks, which later I want to use to code a few trading scripts eventually. 

Ultimately my plan is to implement a market making bot based on the model of Avellaneda and Stoikov.

Edited by fh13

Share this post


Link to post
Share on other sites

The "replay" option exists to do exactly this. If you do "--a -load --ledger <hash> --replay" rippled will load the specified ledger and its predecessor, then extract the state of the predecessor and the transactions from that ledger, it will wait for a "ledger_accept" command, and then replay the close of the specified ledger;. You can use "ledger_request" in a prior run to ensure both ledgers are fully resident to ensure the replay will succeed.

This is a great way to understand precisely why and how a particular transaction had a particular result. You can replay the close of the appropriate ledger with logging at trace level and even try modifying the code to get more information or see the effects of code changes on particular transactions.

Share this post


Link to post
Share on other sites
The "replay" option exists to do exactly this. If you do "--a -load --ledger --replay" rippled will load the specified ledger and its predecessor, then extract the state of the predecessor and the transactions from that ledger, it will wait for a "ledger_accept" command, and then replay the close of the specified ledger;. You can use "ledger_request" in a prior run to ensure both ledgers are fully resident to ensure the replay will succeed.

This is a great way to understand precisely why and how a particular transaction had a particular result. You can replay the close of the appropriate ledger with logging at trace level and even try modifying the code to get more information or see the effects of code changes on particular transactions.

Forensic accountants will love this.

Sent from my iPad using Tapatalk

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×
×
  • Create New...