The "200" limit

[Malloy]

Hi, 

Anyone can tell me why I can only see the first 200 orders in the order book?

I have more than 200 orders and it's really a pain to get to the latest ones when I can't see them.

I have the Rippex desktop wallet (topped at 200), same with Gatehub online wallet. I don't know if it's the same with Kraken and Polo, I never got to this point "yet".

Thanks

 

@gregor @RafOlP

[velmet]

Dont spam f-ing orderbook!

[Professor Hantzen]

27 minutes ago, Malloy said:

Anyone can tell me why I can only see the first 200 orders in the order book?

The underlying rippled API request that is used to retrieve RCL orderbooks you see in any app or service has a parameter called "limit" which limits the amount of offers the rippled server will include in the response to the request.  It's there to ensure the server being queried doesn't have to send any more data than is necessary (orderbook data on RCL is hard on bandwidth).  Presumably whatever service or app you're using has set that limit to 200.  This is actually quite high.  In my experience at least, the public rippled servers run by Ripple for example, won't send more than 300 offers for any given book.

Kraken and Polo won't help you here.  They don't show offers from the Ripple network. They have their own, separate orderbooks formed of offers placed within their proprietary exchange platforms only. Confusingly, Bitstamp also does this on their platform, and yet has a separate set of orderbooks for their USD and BTC IOU's on RCL.

[Malloy]

49 minutes ago, Professor Hantzen said:

Presumably whatever service or app you're using has set that limit to 200. 

 

Anyway to change that number?

[ripplerm]

Try this wallet: https://ripplerm.github.io/ripple-wallet/

the wallet-client does not set any limit number when making offers-query to server...

but if u still see only 200 then it's the limit imposed from server side...

[Malloy]

15 minutes ago, ripplerm said:

Try this wallet: https://ripplerm.github.io/ripple-wallet/

the wallet-client does not set any limit number when making offers-query to server...

but if u still see only 200 then it's the limit imposed from server side...

Hey @ripplerm, I really want to try your wallet...I'll setup something with an alternate account for test purpose asap. Until then, I'd be nice to know if it's a server side issue.

[JoelKatz]

There's a server default limit and a server maximum limit. If you run your own server, you can set the limit higher.

We talked about using the marker technique that we use with other calls to allow you to retrieve offers past the limit. But the problem is that the server has a lot of context that it maintains during that call and there's really no good way to store that context to allow the call to pick up from where it left off.

We had two ideas for how to work around this:

1) We add a new call to get the order book where the client is responsible for all context tracking. This could trivially use a marker so that a follow up call could pick up where you left off.

2) We allow each client connection to have one "order retrieval in progress" context associated with it. This would let the client pick up the last order retrieval request where it left off.

The problem is that both of these require significant client support (1 more than 2). The advantage of option 1 would be that it would let clients reliably track changes in order books over long periods of time if they wanted to.

There just wasn't much interest in implementing any of these.

[Malloy]

1 hour ago, JoelKatz said:

There's a server default limit and a server maximum limit. If you run your own server, you can set the limit higher.

So if Gatehub uses their own server, they could set the limit higher (how high max - just wondering?). But I guess the Rippex app. retrieves info from your server so it's capped by default?

 

Quote

There just wasn't much interest in implementing any of these.

With the Market Maker program, that could be something to revisit.

[JoelKatz]

3 hours ago, Malloy said:

So if Gatehub uses their own server, they could set the limit higher (how high max - just wondering?). But I guess the Rippex app. retrieves info from your server so it's capped by default?

That's correct. If they point their client to their own server, they can set the default and the limit to whatever values they want. If the client doesn't specify a limit and lets you configure what server you point it to, you can point it to your own server and change the default to be much higher.

The reason this is more complicated than retrieving anything else is that the funding status of an offer is dependent on the offers ahead of it in the book. As the server produces the response to the query, it keeps tabs of the effect of previous offers on account's funding status. To "pick up where it left off" it would need to recover that context. Or the whole design would have to change such that the client keeps track of the funding status of offers.

[Professor Hantzen]

6 hours ago, JoelKatz said:

That's correct. If they point their client to their own server, they can set the default and the limit to whatever values they want. If the client doesn't specify a limit and lets you configure what server you point it to, you can point it to your own server and change the default to be much higher.

The reason this is more complicated than retrieving anything else is that the funding status of an offer is dependent on the offers ahead of it in the book. As the server produces the response to the query, it keeps tabs of the effect of previous offers on account's funding status. To "pick up where it left off" it would need to recover that context. Or the whole design would have to change such that the client keeps track of the funding status of offers.

How about a flag on the book_offers request (say "limit: -1"), that requests the server to dynamically set a temporary limit as high it is able in excess of it's own maximum, based on load?  So, the server would endeavour to send the whole book in one hit, but the client knows the amount of offers returned may fall short of the total, yet can rely on receiving at least the servers usual maximum.

Would an optional "light" book_offers request be possible?  The main reason clients request the book is to place trades against it.  To do this, clients require only two numbers per offer: the quality, and the funded amount of taker_gets.  In most cases the other material rippled supplies is of no use.  The issuer and currency for each offer is redundant for example, as it doesn't change and is known prior to sending the request.  A light request could reduce book_offers bandwidth by around 95% (~800bytes vs ~40bytes per offer).

[JoelKatz]

We actually could use the new(ish) coroutine code to raise the limit. The tricky bit would be that we would need to stream the results to the client as the coroutine was running. We've built all the support for this but never used all of its pieces together in one thing before. The coroutine code is used to enable old-style path find (ripple_path_find) to use the same code as the new-style path finding. But in that case, we don't need to stream out the output.

I like the idea of a "light" book_offers request. It could even summarize the order book into levels, showing in full detail only the placer's own offers.

[Professor Hantzen]

Cool! And the levels idea is great - specifying an account to show detailed offers from would be the best of both worlds.

[Malloy]

20 hours ago, JoelKatz said:

I like the idea of a "light" book_offers request. It could even summarize the order book into levels, showing in full detail only the placer's own offers.

If you mean by "light" book only type/currencies/price/amount...that's all I need. If more info is available, it could be layered and retrieved only if asked.

I'm a power user not a programmer...hope it make sens.