which ledger is used to check validity of candidate transaction for inclusion into open ledger

[Armor009]

Against which type of ledger do servers check the validity of transaction to be included in candidate set (inclusion into open ledger)?

Is it last closed ledger or last validated ledger?

What I could find so far is this: https://xrpl.org/consensus.html

Quote

The servers that receive, relay and process transactions may be either tracking servers or validators. The major functions of tracking servers include distributing transactions from clients and responding to queries about the ledger. Validating servers perform the same functions as tracking servers and also contribute to advancing the ledger history. 3.

While accepting transactions submitted by client applications, each tracking server uses the last validated ledger as a starting point. The accepted transactions are candidates.

 

Terms:

1. last closed ledger (lcl)

2. last validated ledger (lvl)

 

Reason for confusion- my understanding: The account sequence $s$ is incremented once a transaction is validated. All transactions that are in the LCL but not in the LVL will have the same transaction sequence $s+1$. The state of the account after the transaction with sequence $s$ will be used to check the validity of the transaction $s+1$. The transaction $s$ is in a VL $L_i$. Since no validated transactions are added to the VLs after $L_i$, the LVL will have the same state of the account as $L_i$. Thus it is correct to state that the transaction is based on the LVL.

 

What if there were three closed ledgers. Would the candidate transaction refer to previous transactions in the last closed ledger

[jn_r]

It will refer to previous transactions in last validated ledger. 

In general a validator will only have 0 or 1 closed ledger. I actually can't see how it would have more than 1 closed ledger at one moment in time, but it could create more closed ledgers during the determination of 1 ledger.

At this page the difference between close and validated ledger is better explained: https://xrpl.org/ledgers.html#open-closed-and-validated-ledger

Edited July 22, 2022 by jn_r

[Armor009]

Thank you for your response.

A. If validation fails due to network partitioning but consensus goes through, it is possible to have more than 0 or 1 closed ledgers. In that case would it fallback on the last closed ledger or last validated ledger? The network would stop making progress in such a situation if last validated ledger is to be referenced.

 

B. Since account sequence is increased only when validation is completed, the account will be able to make only one transaction and wait for validation completion. This is a good measure from a safety/ financial services standpoint. However from a product perspective this delay can cause frustration.

 

C. Also, since

1. a new closed  ledger is applied on the previous closed ledger.

2. open ledger is created using last closed ledger as a base

shouldn't the validity of a transaction be based on the last closed ledger.

Edited July 23, 2022 by Armor009

[jn_r]

13 minutes ago, Armor009 said:

A. If validation fails due to network partitioning but consensus goes through, it is possible to have more than 0 or 1 closed ledgers. In that case would it fallback on the last closed ledger or last validated ledger? The network would stop making progress in such a situation.

Partitioning will not occur. I'll try to explain. During a round, validator participants (you and the validators in your chosen UNL) decide on which transactions they want to include in the next ledger. You receive all kind of candidate transactions. If they are valid you share them with the others. If you reach a certain tresholds between the UNL validators to add a transaction, then you add that transaction to your 'closed ledger' (or 'open ledger', not exactly sure the difference there, as 'open ledger' is described as the working ledger, it seems to me 'closed ledger' is just a possible end-state of your current 'open ledger'). So you continuously update your current open ledger which leads to a certain closed ledger. Up to the point where the treshold for inclusion of a transaction is above a certain level and it is time for another ledger. Only then will the transaction be ordened and you will get a validated ledger. Obviously better explained here:

https://xrpl.org/consensus-principles-and-rules.html#consensus-rounds

24 minutes ago, Armor009 said:

B. Since account sequence is increased only when validation is completed, the account will be able to make only one transaction and wait for validation completion. This is a good measure. However from a product perspective this delay can cause frustration.

 

As an account holder you can request the sequence number you are at. That sequence number is determined from the last 'validated ledger'. But then you can add multiple new transactions to be included in the next 1 or more ledgers, just make sure you use successive sequence numbers in your transactions . The XRPL will make sure that the ordening of transactions is done in such a way that the transaction sequence numbers per account are chronological in order (remember that transaction ordening is done after determining the set of transactions in a ledger). So it is possible that one account has more than 1 transaction in 1 ledger.

27 minutes ago, Armor009 said:

C. Also, since

1. a new closed  ledger is applied on the previous closed ledger.

2. open ledger is created using Last closed ledger as a base

shouldn't the validity of a transaction be based on the last closed ledger.

As I read it, (but I am a little guessing here)

1. The transaction set from the closed ledger has reached the treshold, so all transactions in that ledger have already been agreed on with superconsensus. There is a small chance that also other sets of transactions have reached superconsensus (80%). In that case the previous closed ledger is expanded with the other transactions that also reached super-consensus. The chance that you missed a transaction that is not in your closed ledger is small, but is possible, i.e. not in your 80% but apparently in another 80%. But again, I'm a bit guessing here.

2.  Yes, so open ledger is your working ledger. You use all transactions from the proposal - the closed ledger. And you expand that with the transaction(s) that you had apparently missed. That will lead to a new proposed closed ledger.

[Armor009]

Thank you for your response.

1 hour ago, jn_r said:

Partitioning will not occur.

Partially agree with you.

It is very difficult to cause partitioning if the UNL/ list of validators is predefined and has 90% overlap. Unless BGP Hijack or DDOS occurs. The network is highly centralised with 90% overlap.

Partitioning can be affected via malicious (byzantine) nodes throttling validations and/ or sending different validations to different partitions.

 

1 hour ago, jn_r said:

But then you can add multiple new transactions to be included in the next 1 or more ledgers, just make sure you use successive sequence numbers in your transactions

Can a client send transactions with increasing Transaction Sequence number in transaction field?

This is what I found wrt Account Sequence https://xrpl.org/basic-data-types.html#account-sequence

Every account in the XRP Ledger has a sequence number in its Sequence field, which increases by 1 whenever that account sends a transaction and that transaction gets included in a validated ledger. Each transaction also has a sequence number in its Sequence field, which must match the account's current sequence number when the transaction executes

It is possible for multiple unconfirmed transactions to have the same sender and sequence number. Such transactions are mutually exclusive, and at most one of them can be included in a validated ledger. (Any others ultimately have no effect.)

1 hour ago, jn_r said:

Yes, so open ledger is your working ledger. You use all transactions from the proposal - the closed ledger. And you expand that with the transaction(s) that you had apparently missed. That will lead to a new proposed closed ledger.

Bringing clarity on open and closed ledgers: At the end of Consensus Voting, the server discards the Open Ledger and creates a new Closed Ledger (CL). The most recent CL at a server is known as its Last Closed Ledger (LCL). The LCL is created by applying the transactions approved via consensus to the previous CL. The CL proposes the next state of the ledger. After creating the LCL, the server creates a new Open Ledger with the LCL as its base. https://xrpl.org/ledgers.html#open-closed-and-validated-ledgers

Query: If the LCL is the base for the Open Ledger, shouldn't transaction validity be dependent on last closed ledger. The validity of last closed ledger depends on it previous closed ledgers which finally depend on the last validated ledger. So, If there were two CLs, then the validity of transactions in LCL with ledger index s+1 should depend on CL with ledger index s. This in turn would depend on the LVL with ledger index less than s.

Edited July 23, 2022 by Armor009

[jn_r]

3 minutes ago, Armor009 said:

It is very difficult to cause partitioning if the UNL/ list of validators is predefined and has 90% overlap. Unless BGP Hijack or DDOS occurs. The network is highly centralised with 90% overlap.

Partitioning can be affected via malicious (byzantine) nodes throttling validations and/ or sending different validations to different partitions.

Yes. So that's why it is preferred that you use the UNL from XRPL foundation or Coil. You must however realise what exactly the UNL does, it only determines which transactions will be part of the next round. All other nodes also validate all the transactions. The worst that could happen is either a halt of the network because of byzantine nodes in your UNL. Never will a faulty transaction be accepted by the network. The actions to follow are simple to be solved, you either agree on another UNL, or if it is a temporary situation you resolve it using https://xrpl.org/known-amendments.html#negativeunl

In case of censorship of transactions would occur (all validators UNL would have to work together) there is detection that is used by all nodes in the network: https://xrpl.org/transaction-censorship-detection.html#transaction-censorship-detection , so also that would be noticed by the entire network.

12 minutes ago, Armor009 said:

Can a client send transactions with increasing sequence number?

This is what I found wrt Account Sequence https://xrpl.org/basic-data-types.html#account-sequence

Every account in the XRP Ledger has a sequence number in its Sequence field, which increases by 1 whenever that account sends a transaction and that transaction gets included in a validated ledger. Each transaction also has a sequence number in its Sequence field, which must match the account's current sequence number when the transaction executes

 

It is possible for multiple unconfirmed transactions to have the same sender and sequence number. Such transactions are mutually exclusive, and at most one of them can be included in a validated ledger. (Any others ultimately have no effect.)

1 hour ago, jn_r said:

Yes you can send transactions with increasing number. My market maker bots usually sends 6 txs at once (3 below the spot price and 3 above) and they usually all end up in the same ledger. 

You can not send 2 transactions with the same transaction number. That would not make a valid transaction set, so the first transaction is accepted in the set, then the 2nd is invalid.

15 minutes ago, Armor009 said:

Query: If the LCL is the base for the Open Ledger, shouldn't transaction validity be dependent on last closed ledger. The validity of last closed ledger depends on it previous closed ledgers which finally depend on the last validated ledger. So, If there were two CLs, then the validity of transactions in LCL with ledger index s+1 should depend on CL with ledger index s. This in turn would depend on the validity of LVL with ledger index less than s.

Reading it again it seems that the difference is simply that the LOL (😂 last open ledger) transactions are in the order that they appear, whereas the LCL (last closed ledger) orders these transactions in the deterministic canonical order.

Wether a transaction is valid or not has nothing to do with the order in which it will be executed. It just has to be well-formed and the signature should be correct. Compare a failed transaction in Ethereum, which can perfectly valid, but yet fails because of some state in the network. Another example, 2 transactions sending XRP out of your account, the first transaction succeeds and sends everything out, the 2nd fails because there is no XRP in your account (but it is a valid transaction nevertheless, it is included in the ledger transactions to be executed and fail)

Quote

For an open ledger, servers apply transactions in the order those transactions appear, but different servers may see transactions in different orders. Since there is no central timekeeper to decide which transaction was actually first, servers may disagree on the exact order of transactions that were sent around the same time. Thus, the process for calculating a closed ledger version that is eligible for validation is different than the process of building an open ledger from proposed transactions in the order they arrive. To create a "closed" ledger, each XRP Ledger server starts with a set of transactions and a previous, or "parent", ledger version. The server puts the transactions in a canonical order, then applies them to the previous ledger in that order. The canonical order is designed to be deterministic and efficient, but hard to game, to increase the difficulty of front-running Offers in the decentralized exchange.

 

[Armor009]

Thank you for your response

1 hour ago, jn_r said:

Yes you can send transactions with increasing number.

Thank you

 

1 hour ago, jn_r said:

Wether a transaction is valid or not has nothing to do with the order in which it will be executed. It just has to be well-formed and the signature should be correct. Compare a failed transaction in Ethereum, which can perfectly valid, but yet fails because of some state in the network. Another example, 2 transactions sending XRP out of your account, the first transaction succeeds and sends everything out, the 2nd fails because there is no XRP in your account (but it is a valid transaction nevertheless, it is included in the ledger transactions to be executed and fail)

A. If I am not wrong the validity (whether the account has enough balance to pay for the transaction) of a transaction is checked at candidate transaction set phase (phase open). It is checked against the LVL or LCL. If a transaction is not executable (not enough balance in account), there would be no point in taking it through consensus. It is a waste of resources.

This validity is also checked while applying the transaction to the LCL and LVL

This is different from the valid (well-formed) transaction concept.

Please correct me if I am wrong.

Let's share references and citations to clear this.

1 hour ago, jn_r said:

For an open ledger, servers apply transactions in the order those transactions appear, but different servers may see transactions in different orders. Since there is no central timekeeper to decide which transaction was actually first, servers may disagree on the exact order of transactions that were sent around the same time. Thus, the process for calculating a closed ledger version that is eligible for validation is different than the process of building an open ledger from proposed transactions in the order they arrive. To create a "closed" ledger, each XRP Ledger server starts with a set of transactions and a previous, or "parent", ledger version. The server puts the transactions in a canonical order, then applies them to the previous ledger in that order. The canonical order is designed to be deterministic and efficient, but hard to game, to increase the difficulty of front-running Offers in the decentralized exchange.

Indeed

B. In the open ledger, ordering is as per transactions received. In LCL and LVL, it is as per canonical order. However if I am not wrong the validity of transaction (whether the account has money) is checked at all phases of ledger generation (open ledger, LCL and LVL generation).

Transactions that were initially valid (at candidate set generation phase) as the account had balance might be invalid (while applying the transaction to the LCL) as there was a prior transaction in the same ledger that exhausted the balance.

Please correct me if I am wrong.

Edited July 23, 2022 by Armor009

[jn_r]

1 hour ago, Armor009 said:

A. If I am not wrong the validity (whether the account has enough balance to pay for the transaction) of a transaction is checked at candidate transaction set phase (phase open). It is checked against the LVL or LCL. If a transaction is not executable (not enough balance in account), there would be no point in taking it through consensus. It is a waste of resources.

This validity is also checked while applying the transaction to the LCL and LVL

This is different from the valid (well-formed) transaction concept.

Please correct me if I am wrong.

If you have not enough balance your transaction is executed, but will fail. You will have to pay a fee though.

One example is from the famous Tacostand, namely

https://bithomp.com/explorer/6DFF8964F961D13DB9DEED3CA27CF7C26BB3A031E538ED59E6E884180D7C4659

I would not see it as a waste of resources, you will have to pay transaction fee still, and it was at a valid request after all ..

1 hour ago, Armor009 said:

B. In the open ledger, ordering is as per transactions received. In LCL and LVL, it is as per canonical order. However if I am not wrong the validity of transaction (whether the account has money) is checked at all phases of ledger generation (open ledger, LCL and LVL generation).

Transactions that were initially valid (at candidate set generation phase) as the account had balance might be invalid (while applying the transaction to the LCL) as there was a prior transaction in the same ledger that exhausted the balance.

 

This is the same assumption as the previous, your  definition of 'valid' is not correct. The correct way is to see 'valid' independent from the state on-chain. Basically means, a well-formed transaction with a correct signature.

edit: well not entirely, sequence number must be correct, which is a state dependent on the last validated ledger. If the sequence number is not correct, the transaction will not be included in the ledger. Trying to find a quote that explains exactly correctness, but I could only find this: https://xrpl.org/consensus-principles-and-rules.html#consensus-rules

Quote

Every participant’s top priority is correctness. They must first enforce the rules to be sure nothing violates the integrity of the shared ledger. For example, a transaction that is not properly signed must never be processed (even if other participants want it to be processed). However, every honest participant’s second priority is agreement. A network with possible double spends has no utility at all, so every honest participant values agreement above everything but correctness.

And for reference here all the TEC codes, transactions that failed, but were applied to the ledger: https://xrpl.org/tec-codes.html#tec-codes 

or this https://xrpl.org/transaction-results.html even better, also check out the TER codes .. 

Edited July 23, 2022 by jn_r
I should stop editing previous posts :-)

[Armor009]

Thank you so much.

This was very useful.

On a lighter note (wrt your edit comment):

19 hours ago, jn_r said:

I should stop editing previous posts :-)

There is no harm in editing/ updating the last post to provide richer information. :-)

[Armor009]

 

22 hours ago, jn_r said:

edit: well not entirely, sequence number must be correct, which is a state dependent on the last validated ledger. If the sequence number is not correct, the transaction will not be included in the ledger.

Wrt this, I am guessing that

1. the transaction sequence number should be equal to or greater than the account sequence (s) for inclusion into open ledger

2. the transaction sequence number (say, s+4) should be equal to or greater than the account sequence for inclusion into closed ledger;

a. there should be a valid (well-formed) series of transactions from s+1  to the transaction (s+4) that are added to the closed ledger.

b. Once consensus is reached and before transactions are applied to the LCL, it is checked if the account has adequate amount.

c. This is done against the LCL.

 

https://xrpl.org/ledgers.html. Applies to previous ledger, which implies it checks account balance?

Quote

To create a "closed" ledger, each XRP Ledger server starts with a set of transactions and a previous, or "parent", ledger version. The server puts the transactions in a canonical order, then applies them to the previous ledger in that order. The canonical order is designed to be deterministic and efficient, but hard to game, to increase the difficulty of front-running Offers in the decentralized exchange

https://xrpl.org/consensus-principles-and-rules.html#consensus-rules

Quote

A network with possible double spends has no utility at all, so every honest participant values agreement above everything but correctness.

 

3. the transaction sequence number (say, s+4) should be equal to or greater than the account sequence for inclusion into closed ledger;

a. there should be a valid (well-formed) series of transactions from s+1  to the transaction (s+4) that are added to the last validated ledger.

b. Once consensus is reached and before transactions are applied to the LCL, it is checked if the account has  adequate amount.

 

c. This is done against LVL.

----------------------------------------------------------------------------------------

Is this accurate?

I've added a video explaining the old RPCA. Couldn't find any explaining the XRP LCP

For your ease, I've labelled wit numbering and sub points. One could mention which sub point is inaccurate and the accurate version for it.

----------------------------------------------------------------------------------------

https://xrpl.org/consensus.html

Quote

All transactions included in a ledger destroy some XRP as a transaction cost, regardless of whether they had a tes or tec code. The exact amount of XRP to destroy is defined by the signed transaction instructions.

 

Quote

While accepting transactions submitted by client applications, each tracking server uses the last validated ledger as a starting point

4. Why is the LVL taken as a starting point if all well formed and signed transactions are to be included

 

Edited July 24, 2022 by Armor009

[jn_r]

'kay will try to answer those:

2 hours ago, Armor009 said:

1. the transaction sequence number should be equal to or greater than the account sequence (s) for inclusion into open ledger

I would think not, I think only the current sequence number is accepted, after which the next can be accepted

2 hours ago, Armor009 said:

2. the transaction sequence number (say, s+4) should be equal to or greater than the account sequence for inclusion into closed ledger;

open ledger leads to closed ledger, so I don't see that as an issue to be discussed. What matters is that the open ledger contains valid transactions and that is handled by 1.

2 hours ago, Armor009 said:

a. there should be a valid (well-formed) series of transactions from s+1  to the transaction (s+4) that are added to the closed ledger.

Yes, but that follows from these transactions gathered as an unordered set in openledger, as in 1., but with this property that you mention under a.

2 hours ago, Armor009 said:

b. Once consensus is reached and before transactions are applied to the LCL, it is checked if the account has adequate amount.

no, that check is not performed, unless you mean that the the transactions are 'executed'

2 hours ago, Armor009 said:

c. This is done against the LCL.

I like that video, surprised however how they mention closed ledgers. Apparently a node can gather closed ledgers that would still need final validation. Not sure how exactly that works

2 hours ago, Armor009 said:

3. the transaction sequence number (say, s+4) should be equal to or greater than the account sequence for inclusion into closed ledger;

It should be the exact correct number to be included in the open ledger. Not sure why you keep repeating this. Say for example (out of practical experience) I would put in a transaction with sequence number s+1 and sequence number s is missing, then that transaction will not be included in a closed ledger. It will however be executed after I add at a later time a transaction with sequence s. So I am not sure if that transaction is then kept in the open ledger. I would assume not, I would assume it is kept in a queue of candidate transactions

2 hours ago, Armor009 said:

a. there should be a valid (well-formed) series of transactions from s+1  to the transaction (s+4) that are added to the last validated ledger.

of course

2 hours ago, Armor009 said:

b. Once consensus is reached and before transactions are applied to the LCL, it is checked if the account has  adequate amount.

that's a repeat question I think

2 hours ago, Armor009 said:

c. This is done against LVL.

I think it is done against the LCL if I understand correct, but again, I'm a bit off with how exactly it works with validated versus closed ledger.

2 hours ago, Armor009 said:

4. Why is the LVL taken as a starting point if all well formed and signed transactions are to be included

Again there seems confusion about LCL and LVL. To me it seems most logical to start building from a LVL, but apparently there could theoretically be a gap with some LCL's that have not become LVL's yet.

[Armor009]

Thank you for your response

Points 5 and 6 are inter-related. Made 6 first and then 5. Please bear with me. _/\_

I'm kinda skinning the hair to get perfect clarity. This because I am currently writing a description of the consensus algorithm. Want to be crystal clear on the concepts.

You're a gem of a person

  5.

22 hours ago, jn_r said:

 

On 7/24/2022 at 6:39 PM, Armor009 said:

1. the transaction sequence number should be equal to or greater than the account sequence (s) for inclusion into open ledger

I would think not, I think only the current sequence number is accepted, after which the next can be accepted

a. Is it like the next transaction can be accepted in the same open ledger in the order you mentioned. So transaction s is followed by transaction s+1 in the same open ledger.

b. Again what is to be noted is that the open ledger does not have transaction ordering. So how would this transaction sequence number ordering be enforced.

I'm slightly confused.

6. You had mentioned that in the past you had 3 transactions with increasing transaction sequence number included in the same ledger.

(5 b and 6 a are inter-related)

On 7/23/2022 at 6:39 PM, jn_r said:

Yes you can send transactions with increasing number. My market maker bots usually sends 6 txs at once (3 below the spot price and 3 above) and they usually all end up in the same ledger.

a. If this is the case, I am guessing that transactions with transaction sequences s->s+1->s+2->s+3 are accepted in a sequence in the LVL. Since the LVL is basically a vote on the validity of LCL, these 3-4 transactions would be admitted in the LCL. An LCL is created by transactions in the open ledger. So it must follow through that these transactions can be included in the same open ledger.

Since there is no ordering in the Open Ledger, there can be transactions which would be greater than or equal to account sequence number and not necessarily in the right transaction sequence order.

Couple of things other things to notice are that

b. the account sequence is only incremented from s to s+1 when a transaction s is included in the LVL. So. when the three transactions with increasing transaction sequence number are added to LCL, which then gets voted/ validated by the validators, the account sequence will increase as and when the transactions get 'applied' in the LVL.

c. Importantly, in the video of the Ripple Protocol Consensus Algorithm (RPCA), it says  the validity of a transaction is checked after consensus and while applying to the LCL.

d. The RPCA paper mentions that "Initially, each server takes all valid transactions it has seen prior to the beginning of the consensus round that have not already been applied (these may include new transactions initiated by end-users of the server, transactions held over from a previous consensus process, etc.), and makes them public in the form of a list known as the “candidate set”."

https://ripple.com/files/ripple_consensus_whitepaper.pdf

This means that there is a validity of transaction that is checked before inclusion into open ledger.

e. Wondering what are the two different/ same validity checks in the two different phases.

Note: This might be applicable to the XRP LCP as well. The XRP Ledger Consensus Protocol (XRP LCP) is built on top of the Ripple Protocol Consensus Algorithm (RPCA) basis what I have seen. It makes sense since the development work would be linear and built on top of each other. Cobalt is a slightly different  as it merges the different phases of RPCA/ XRP LCP and executes all of them simultaneously. However the building blocks for cobalt are the same.

7.

22 hours ago, jn_r said:

On 7/24/2022 at 6:39 PM, Armor009 said:

2. the transaction sequence number (say, s+4) should be equal to or greater than the account sequence for inclusion into closed ledger;

open ledger leads to closed ledger, so I don't see that as an issue to be discussed. What matters is that the open ledger contains valid transactions and that is handled by 1.

Yes, in the closed ledger, the transaction ordering has to be there. Agreed. Thanks

8.

22 hours ago, jn_r said:

On 7/24/2022 at 6:39 PM, Armor009 said:

a. there should be a valid (well-formed) series of transactions from s+1  to the transaction (s+4) that are added to the closed ledger.

Yes, but that follows from these transactions gathered as an unordered set in openledger, as in 1., but with this property that you mention under a.

Yes. This got cleared. Thank you.

9.

22 hours ago, jn_r said:

On 7/24/2022 at 6:39 PM, Armor009 said:

b. Once consensus is reached and before transactions are applied to the LCL, it is checked if the account has adequate amount.

no, that check is not performed, unless you mean that the the transactions are 'executed'

a. This follows from 6. c.

b. When is a transaction executed? Is it when it is added to the LVL/ is it when added to LCL.

To add on to it. The transaction is proposed for execution in the LCL. The LVL finally executes it. Am I correct. Am I correct?

c. So LCL just does transaction ordering but no check on adequate amount?

d. When are tes, tec, and different codes added to a transaction.

e. If LVL is the 'network's vote' on the LCL, shouldn't the validity of transaction from an account balance be checked at LCL state.

f. Would it be correct to describe the LCL and LVL. The LCL is a proposal on the execution of the transaction.The LVL is the final execution of the transaction

 

10.

22 hours ago, jn_r said:

On 7/24/2022 at 6:39 PM, Armor009 said:

 

I like that video, surprised however how they mention closed ledgers. Apparently a node can gather closed ledgers that would still need final validation. Not sure how exactly that works

Yes. . The 0-1 LCL count exists is in happy flow of the RPCA/ XRP LCP.

 

11.

22 hours ago, jn_r said:

On 7/24/2022 at 6:39 PM, Armor009 said:

3. the transaction sequence number (say, s+4) should be equal to or greater than the account sequence for inclusion into closed ledger;

It should be the exact correct number to be included in the open ledger. Not sure why you keep repeating this. Say for example (out of practical experience) I would put in a transaction with sequence number s+1 and sequence number s is missing, then that transaction will not be included in a closed ledger. It will however be executed after I add at a later time a transaction with sequence s. So I am not sure if that transaction is then kept in the open ledger. I would assume not, I would assume it is kept in a queue of candidate transactions

Yes, it remains in the candidate set. I meant the 3. point as a base for 3.a. and 3. b. The queries are generally the sub point if it exists, else it would be the number. The repeat of the sequence number is to formally mention it in a more mathematical manner to avoid confusion. Thank you.

12.

22 hours ago, jn_r said:

On 7/24/2022 at 6:39 PM, Armor009 said:

b. Once consensus is reached and before transactions are applied to the LCL, it is checked if the account has  adequate amount.

that's a repeat question I think

It's a error on my part. Sorry. I should have updated the copy pasted content. I missed that.

the right one for 3 b. would have been. Once validation is reached and before transactions are applied to the LVL, it is checked if the account has adequate amount.

13. If you see the video, when the LCL is formed and applied across the network, the ledger has accounts and balance written on it.

Does this mean balances are calculated then also. In that case the validity at close of LCL might be account status also. This makes so much sense since the LVL is a network validated version of the LCL.

14.

22 hours ago, jn_r said:

On 7/24/2022 at 6:39 PM, Armor009 said:

4. Why is the LVL taken as a starting point if all well formed and signed transactions are to be included

Again there seems confusion about LCL and LVL. To me it seems most logical to start building from a LVL, but apparently there could theoretically be a gap with some LCL's that have not become LVL's yet.

Exactly, the gap between LVL and LCL is the reason why my query is coming up.

 

15.

On 7/24/2022 at 6:39 PM, Armor009 said:

While accepting transactions submitted by client applications, each tracking server uses the last validated ledger as a starting point

This also means that the state of a transaction (account balance) is also considered while checking validity at candidate set generation phase. Which adds to the confusion.

Edited July 25, 2022 by Armor009
It's cleaned

[Armor009]

Came across this

https://xrpl.org/reliable-transaction-submission.html#transaction-timeline

It speaks about candidate set generation.

It merges consensus and validation together for transaction timeline

Edited July 25, 2022 by Armor009

[jn_r]

6 hours ago, Armor009 said:

a. Is it like the next transaction can be accepted in the same open ledger in the order you mentioned. So transaction s is followed by transaction s+1 in the same open ledger.

b. Again what is to be noted is that the open ledger does not have transaction ordering. So how would this transaction sequence number ordering be enforced.

I'm slightly confused.

It is sort of special within he open ledger. The transactions are unordered, but since multiple transactions from one account can be included in one ledger, per account these checks are performed. The logical way to implement it, is first allow s, then allow s+1, etc. Transactions that are not yet proposed in the open ledger, are kept in the transaction queue: https://xrpl.org/transaction-queue.html

Quote

6 abcde

Would it make sense if summarize it as follows?

1. Incoming transactions enter the transaction queue
2. Transactions are selected from the transaction queue and placed in the transaction set from the open ledger
3. The transaction set from the open ledger is used to form the closed ledger
4. The closed ledger can become a validated ledger. Only validated ledgers are final and form the (block)chain

Quote

9 bcdef

9b. A transaction is executed when added to a closed ledger. Actually not 100% sure it could also be only applying the ordering, but my guess is that a creating the closed ledger it also executes them so it know what the ledger will look like (and determine the ledger hash) https://xrpl.org/consensus.html#validation At final validation validators can compare the hash to determine the validated ledger.

9c. see above. at execution it will check everything it needs

9d. also at 'execution'. With execution meaning just calculating what the ledger will look like. It is only final after it is validated by consensus

9e yes

9f a validated ledger is just the consensus on what has been executed, so LCL executes (proposes a new ledger) but the LVL gives you the final verdict.

6 hours ago, Armor009 said:

13. If you see the video, when the LCL is formed and applied across the network, the ledger has accounts and balance written on it.

Does this mean balances are calculated then also. In that case the validity at close of LCL might be account status also. This makes so much sense since the LVL is a network validated version of the LCL.

yes I agree and as explained above

6 hours ago, Armor009 said:

This also means that the state of a transaction (account balance) is also considered while checking validity at candidate set generation phase. Which adds to the confusion.

No, that is not correct. See the order I gave at 6. It would be checked at determining the LCL when the transactions are 'executed' on the last validated ledger. There is only a very small chance that there will be more than 1 closed ledgers, in that case (that is how I interpret it) the next closed ledger is built on the LCL, which would mean the validator is building new ledgers, but has not reached final consensus on several closed ledgers .. imo normally a closed ledger would be built on top of the last validated ledger, so LCL = OL transaction set ordered canonical and executed one by one on top of the LVL

Edited July 25, 2022 by jn_r

[Armor009]

19 hours ago, jn_r said:

Quote

6 abcde

Would it make sense if summarize it as follows?

1. Incoming transactions enter the transaction queue
2. Transactions are selected from the transaction queue and placed in the transaction set from the open ledger
3. The transaction set from the open ledger is used to form the closed ledger
4. The closed ledger can become a validated ledger. Only validated ledgers are final and form the (block)chain

16. In continuation of the the thread on (6 abcde).

 

This was enlightening. I didn't know that a transaction queue existed as it is not mentioned in the RPCA white paper and I believe it is not mentioned in the XRP LCP paper either. It makes intuitive sense that it exists as shown in the video shared earlier.

Sub points 3 and 4 are accurate.

Sub point 1 is partially accurate.

a. Consider a scenario where the server is in candidate set generation phase and transaction queue is empty. In this case the potential transaction goes directly into the candidate transaction set if it is well formed and has adequate fee for consensus.

TxQ.h in 'rippled' states-

"Transaction Queue. Used to manage transactions in conjunction with fee escalation. Once enough transactions are added to the open ledger, the required fee will jump dramatically. If additional transactions are added, the fee will grow exponentially from there. Transactions that don't have a high enough fee to be applied to the ledger are added to the queue in order from highest fee level to lowest. Whenever a new ledger is accepted as validated, transactions are first applied from the queue to the open ledger in fee level order until either all transactions are applied or the fee again jumps too high for the remaining transactions. For further information and a high-level overview of how transactions are processed with the `TxQ`, see FeeEscalation.md"

 

Fee escalation.md in 'rippled' states-

"There is a limit on the number of transactions that can get into an open ledger for that base fee level. The limit will vary based on the health of the consensus process, but will be at least 5."

"Once the queue is empty, or the required fee level rises too high for the remaining transactions in the queue, the ledger is opened up for normal transaction processing."

 

b. The transaction queue is filled with incoming transactions when

i. transaction queue is not empty

or

ii. the node is in consensus voting phase - as shown in the consensus video

or

iii. The node is in candidate generation phase, the transaction queue is empty and the transaction is not able to meet the current fee requirement of the node/ network

-> This is possible if the open ledger has reached the limit for the maximum number of transactions with base fee and now fee escalation has started. Here, the transaction does not meet the fee requirement

-> This is also possible if the transaction is not able to meet the current base fee requirement, when the open ledger has not reached the limit for the maximum number of transactions with base fee.

In case a. when the transactions are directly entering transaction queue, would the quote below be accurate.

On 7/25/2022 at 3:54 PM, Armor009 said:

Since there is no ordering in the Open Ledger, there can be transactions which would be greater than or equal to account sequence number and not necessarily in the right transaction sequence order.

For case b. The transaction queue applies sequencing of transactions. This sequencing is different from the canonical ordering.

17.

19 hours ago, jn_r said:

On 7/25/2022 at 3:54 PM, Armor009 said:

a. Is it like the next transaction can be accepted in the same open ledger in the order you mentioned. So transaction s is followed by transaction s+1 in the same open ledger.

b. Again what is to be noted is that the open ledger does not have transaction ordering. So how would this transaction sequence number ordering be enforced.

I'm slightly confused.

It is sort of special within he open ledger. The transactions are unordered, but since multiple transactions from one account can be included in one ledger, per account these checks are performed. The logical way to implement it, is first allow s, then allow s+1, etc.

a. The transactions are ordered basis the sequence in which they are received. So if there are two transactions with the same transaction sequence number, the one which was received first is applied to the open ledger. This is iff transaction queue is empty. When transaction queue is not empty, the transactions are sequenced as per transaction queue's sequencing rules

b. What happens when transaction queue is empty and there are two transactions with increasing sequence number

T1 -> transaction sequence number s+1

T2 -> transaction sequence number s+2

account sequence number is s

and T2 is received before T1 at the node.

There is no mention of checks of sequence number in the RPCA white paper and XRP LCP (afaik).

c. What happens transaction queue has transactions and

there are two transactions with increasing sequence number

T1 -> transaction sequence number s+1

T2 -> transaction sequence number s+2

T3 -> transaction sequence number s+4

account sequence number is s

and T2 is received, then T3 and then T1 at the node

and there is no transaction with sequence number s

Does T2, then T3 and T1 get added to the transaction queue?

Asking since I don't see any restriction on the transaction sequence number in https://xrpl.org/transaction-queue.html

18.

19 hours ago, jn_r said:

9d. also at 'execution'. With execution meaning just calculating what the ledger will look like. It is only final after it is validated by consensus

Thank you.

Is this definiton of execution mentioned any where?

19.

19 hours ago, jn_r said:

On 7/25/2022 at 3:54 PM, Armor009 said:

13. If you see the video, when the LCL is formed and applied across the network, the ledger has accounts and balance written on it.

Does this mean balances are calculated then also. In that case the validity at close of LCL might be account status also. This makes so much sense since the LVL is a network validated version of the LCL.

yes I agree and as explained above

Thank you.

20.

19 hours ago, jn_r said:

On 7/25/2022 at 3:54 PM, Armor009 said:

This also means that the state of a transaction (account balance) is also considered while checking validity at candidate set generation phase. Which adds to the confusion.

No, that is not correct. See the order I gave at 6. It would be checked at determining the LCL when the transactions are 'executed' on the last validated ledger. There is only a very small chance that there will be more than 1 closed ledgers, in that case (that is how I interpret it) the next closed ledger is built on the LCL, which would mean the validator is building new ledgers, but has not reached final consensus on several closed ledgers .. imo normally a closed ledger would be built on top of the last validated ledger, so LCL = OL transaction set ordered canonical and executed one by one on top of the LVL

Thank you.

a. The assumption that only 0-1 LCL exists is for happy case. I'm trying to solve for the eventuality where the system breaks and there are more than 1 LCLs. In that case would the LCL state be based on the previous closed ledger? (Given that we've established that the account state is checked at the LCL phase as well as later at LVL state.)

b.

Quote

LCL = OL transaction set ordered canonical and executed one by one on top of the LVL.

Imo the LCL has more weightage than canonical ordering + executed 1 by 1 on top of LVL. This is because the representative of the entire network (UNL) has voted on the inclusion of transaction in the ledger. (

I just realised - vote on inclusion of transaction in the ledger is different from vote on executability of transaction. Thanks

Then again the validity [account balance] of transaction is checked when a transaction is finally applied to LCL

)

21.

Could you please have a look at the queries. Parallelly can we get someone from Ripple Labs to have a look at this and double check the points. Given that this thread clears a lot of queries, it can be used to further improve the website and explanations. You might know someone?

Thanks. This is becoming an awesome thread that clarifies a lot of things.

 

22.

Quote

While accepting transactions submitted by client applications, each tracking server uses the last validated ledger as a starting point. https://xrpl.org/consensus.html#the-xrp-ledger-protocol-consensus-and-validation

a. This is due to the fact that tracking servers do not participate in consensus and don't have an LCL?

b. For rippled servers, the base is the LCL?

Edited July 26, 2022 by Armor009