Jump to content
devnullprod

Key / Account Generation: Did we do it right?!

Recommended Posts

Hello and happy Tuesday everyone! (or Wednesday depending where you are in the world!) We hope you're having a pleasant day/night wherever you are! Just a quick post today pertaining to key and account generation. We've been diving into alot of specifics w/ rippled data structures recently, now that our nodestore reader is mostly wrapped up (at least for the rocksdb backend), we've begun looking at other components and subsystems. We're not looking to do a full fledged rippled implementation, but rather just implement specific components for easier integration into other systems without having to go through costly network / foreign-interface / other calls. Also since our implementation is written in ruby a language designed to be very programmer friendly and human-understandable, perhaps it can be used as a terse "reference implementation" for the various standards that make up the XRPL. It's not there yet, but we can dream....

 

In any case, back to $topic. As part of our library we threw together a few modules to generate keys & subsequently account, node, validator, etc IDs that are compliant with the network. We mostly digged through the rippled source to understand specifics, but this page was also instrumental in completing the picture (again many thanks to the ripple-devs & others for putting all this awesome documentation together, many open source projects are lacking this!!!). AFAIK the logic is complete & compliant with the rippled implementation, and we even verified it by writing a few test modules, utilizing parsers in the C++ rippled codebase to test our generated addresses.


We just wanted to throw this out here to make sure that it's looks good before we started advertising it / sharing it with the general public. I'd hate to hear that a bug in our key generation code resulted in funds being sent to an invalid address! To simply things, I also threw together a gist with the serial list of calls from the relevant XRBP modules. If anyone had a few moments to look at it, it would really be appreciated. (there isn't anything too ruby-specific here, perhaps the only thing being the pack / unpack methods)

 

Many thanks!

 

370265074_Screenshotfrom2019-04-3020-15-40.png.85a6d9bf71b4c74201d99696c93c912a.png

 

Share this post


Link to post
Share on other sites

Great that you've done this.  I don't know much (anything) about Ruby, but in general when implementing such things, watch out for how the system might fallback on some other source of randomness in the case the intended source isn't available.  This could result in something less than secure and as such may be important to understand how differing systems running the same code can serve different (but still "valid") results.

On from there, for the secret_seed --> secret_human_readable part, there's a roughly-parallel node.js implementation, here.

Share this post


Link to post
Share on other sites
1 hour ago, Professor Hantzen said:

Great that you've done this.  I don't know much (anything) about Ruby, but in general when implementing such things, watch out for how the system might fallback on some other source of randomness in the case the intended source isn't available.  This could result in something less than secure and as such may be important to understand how differing systems running the same code can serve different (but still "valid") results.


On from there, for the secret_seed --> secret_human_readable part, there's a roughly-parallel node.js implementation, here.

+1 good point, right now this modules uses the default 'securerandom' interface to generate the default seed, but we'll look into how exactly this is implemented & the fallbck cases. Thanks for the link to the nodejs module, its great to have several parallel implementations to use as a reference.

Share this post


Link to post
Share on other sites
22 minutes ago, Sukrim said:

Why do you take 32 bytes from a secure(?) source just to hash them with SHA256?

This is was mostly to handle the case where secure random wasn't used and the developer / user chose to go with a fixed seed. But agree that in the former case, it's unecessary.

 

11 minutes ago, Sukrim said:

Also it might be useful to implement deriving key pairs from seeds instead of just generating bare keys.

The line above the securerandom call is a seed which if uncommented (and the following securerandom call removed) is used as the basis for the private key digest. One thing that's puzzling me (or perhaps this is a final edge case that we have yet to address) is the lack of a 'salt' variable, eg if two users so happen to randomly use the same seed won't they both generate the same account?

 

Share this post


Link to post
Share on other sites

I'm talking about the "account family seed" (https://xrpcommunity.blog/keys-are-key-secret-keys-signing-transactions/), not some seed value to a PRNG.

2 minutes ago, devnullprod said:

One thing that's puzzling me (or perhaps this is a final edge case that we have yet to address) is the lack of a 'salt' variable, eg if two users so happen to randomly use the same seed won't they both generate the same account?

:blink:

If you want to, I can explain it to you, but first I'd like to know if you have experience in implementing and designing cryptosystems and if you have some examples to show or if this is your first stab at the topic...

Share this post


Link to post
Share on other sites
Posted (edited)
15 minutes ago, Sukrim said:

I'm talking about the "account family seed" (https://xrpcommunity.blog/keys-are-key-secret-keys-signing-transactions/), not some seed value to a PRNG.

:blink:

If you want to, I can explain it to you, but first I'd like to know if you have experience in implementing and designing cryptosystems and if you have some examples to show or if this is your first stab at the topic...

Of course random seed collision is of low probability given a sufficient minimum seed length, but we didn't find this constraint in the source. Again, there's alot there so it might just be something we missed

Edited by devnullprod

Share this post


Link to post
Share on other sites
Posted (edited)
15 minutes ago, Sukrim said:

I'm talking about the "account family seed" (https://xrpcommunity.blog/keys-are-key-secret-keys-signing-transactions/), not some seed value to a PRNG.

 

Agree. This effort mostly started off to generate node keys' for RTXP communication, but expanded into more general key / entity generation. In the former case the seed family doesn't apply as we just need a random private key, but yes for account generation we'll look into the seed family soon.

Edited by devnullprod

Share this post


Link to post
Share on other sites

I want to start with a cryptography comment on your implementation: I'd strongly caution you against deriving a key directly from a seed using a single SHA-256 operation. Doing that leaves your users vulnerable to attacks; notice how rippled doesn't just blindly hash the seed and use it as a secret key. There are ways to strengthen derivations of a key from a passphrase/password and you should use them (e.g. PBKDF2).

More broadly, as @Sukrim pointed out, your key derivation isn't "standard" in that you don't derive the key from a seed in the same way as rippled does. Strictly speaking, that's fine, but it may confuse some users who expect to have the "familiar" sWhatever secret; more importantly, it makes keys generated using your script not portable which can result in lock-in.

For details on how rippled derives a keypair from a 128-bit seed, check out Generator.

With that said, I personally find the derivation we use to be somewhat obscure and wouldn't mind simplifying it going forward. Perhaps a "new" standard, a-la BIP-39 which would also encode the key type (secp256k1 or Ed25519 for now, but maybe more in the future) would be a good idea.

Lastly, I'd be very wary of calls like SecureRandom.random_bytes(32). Maybe Ruby has a great PRNG that is cryptographically secure and does the right thing in all cases—the docs suggests it does. But you probably don't want to depend on just that. Consider additional randomness sources.

Share this post


Link to post
Share on other sites
4 hours ago, nikb said:

I want to start with a cryptography comment on your implementation: I'd strongly caution you against deriving a key directly from a seed using a single SHA-256 operation. Doing that leaves your users vulnerable to attacks; notice how rippled doesn't just blindly hash the seed and use it as a secret key. There are ways to strengthen derivations of a key from a passphrase/password and you should use them (e.g. PBKDF2).

More broadly, as @Sukrim pointed out, your key derivation isn't "standard" in that you don't derive the key from a seed in the same way as rippled does. Strictly speaking, that's fine, but it may confuse some users who expect to have the "familiar" sWhatever secret; more importantly, it makes keys generated using your script not portable which can result in lock-in.

For details on how rippled derives a keypair from a 128-bit seed, check out Generator.

With that said, I personally find the derivation we use to be somewhat obscure and wouldn't mind simplifying it going forward. Perhaps a "new" standard, a-la BIP-39 which would also encode the key type (secp256k1 or Ed25519 for now, but maybe more in the future) would be a good idea.

Lastly, I'd be very wary of calls like SecureRandom.random_bytes(32). Maybe Ruby has a great PRNG that is cryptographically secure and does the right thing in all cases—the docs suggests it does. But you probably don't want to depend on just that. Consider additional randomness sources.

Cool, thanks for the info Nik. I understand your concerns pertaining to seed security and the "standard way" for deriving the key from the seed. We will refine our approach to incorporate your suggestions going forward (while trying to keep it easy to understand for reference purposes).

Stay tuned for updates!

 

 

 

Share this post


Link to post
Share on other sites

Honestly, I think that the existing generation scheme is a pain, and the way we conflate the 'seed' with a 'secret' (as Wietse points out) is a problem. My preference would be to define a new mechanism encode the key generation parameters.

I came up with a new way of representing a seed, which encoded both the type (Ed25519 or secp256k1) and was more "user friendly".

Here's an example 128-bit secp256k1 seed in my new encoding:
    002992-186054-564520-197527-479336-662101-018205-239030-454003

Here's an example 256-bit Ed25519 seed in my new encoding:
    005984-207295-416900-415811-559911-544368-575157-236995-404965-416339-352957-496848-476597-518793-247610-248468

Advantages of this format:

  1. It's only numbers! More universal than English, easier to write and less prone to mess up!
  2. Each 6 digit group has a check digit. If you make a typo in a group, you know which group the typo was in.

The obvious disadvantage:

    WTF IS THIS?! IT DOESN'T LOOK WHAT I'VE USING UNTIL NOW!:ireful1:

Share this post


Link to post
Share on other sites
Posted (edited)
15 hours ago, nikb said:

Honestly, I think that the existing generation scheme is a pain, and the way we conflate the 'seed' with a 'secret' (as Wietse points out) is a problem. My preference would be to define a new mechanism encode the key generation parameters.

I came up with a new way of representing a seed, which encoded both the type (Ed25519 or secp256k1) and was more "user friendly".

Here's an example 128-bit secp256k1 seed in my new encoding:
    002992-186054-564520-197527-479336-662101-018205-239030-454003

Here's an example 256-bit Ed25519 seed in my new encoding:
    005984-207295-416900-415811-559911-544368-575157-236995-404965-416339-352957-496848-476597-518793-247610-248468

Advantages of this format:

  1. It's only numbers! More universal than English, easier to write and less prone to mess up!
  2. Each 6 digit group has a check digit. If you make a typo in a group, you know which group the typo was in.

The obvious disadvantage:

    WTF IS THIS?! IT DOESN'T LOOK WHAT I'VE USING UNTIL NOW!:ireful1:

 

This new format does have an advantage in the sense that it can be parsed in both an incremental / serial manner as well as in parallel chunks. Also i'd imaging it could be encoded in a traditional 'barcode' format pretty easily. How would the application parse this format / generate the keypairs?

 

Getting back to the original $topic, we bit the bullet and pulled in the ruby bindings to the bitcoin secp256k1 library and are just using that as an interim solution for key generation & bindings. At some point, it would be nice to implement this in pure ruby to remove the c-dependency, but for now efforts are best focused elsewhere, and not reinventing the wheel.

 

Glad to say we reached our goal with the original motivation behind all this, groking and implementing a RTXP endpoint which we have successfully used to receive P2P messages from the overlay network. We're planning on using this, our nodestore implementation, and additional analysis for the next section of our codebase analysis in the near future.

 

Screenshot from 2019-05-02 12-47-19.png

Edited by devnullprod

Share this post


Link to post
Share on other sites
1 hour ago, Sukrim said:

Oh nice, I never got around implementing the weird handshake to dive deeper into the P2P protocol...

I’d like to migrate away from that and use TLS 1.3 functionality.

Alternatively, perhaps we can eschew TLS altogether for peer to peer use, opting to adopt BIP-151 instead. Although, we’d still need TLS for user endpoints, so it’s not like we’d remove a dependency.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...