Jump to content
theGlitchKing

Personal Validator / Server

Recommended Posts

I'm a long time reader, and first time poster.  I'm a data scientist, that designs and runs Big Data architectures all over the world, so I have some solid knowledge of technology, though the engineering components to Block chain are still somewhat of a mystery to me. 

I just wanted to ask if there would be some advantage to setting up a validator in AWS that i'm not seeing? 

 

Let's say the cost is 300 /mo for the machine to run the validator software.  

How does that impact RippleNet (pros/cons) 

Is there an advantage to an individual doing it, as opposed to an enterprise? 

 

I know there's no mining component to Ripple, but Connectivity, and exposure seems to be something that's lacking for Ripple. I also know (at least I think I do)  that adding a validator to RippleNet may impact TPS positively, though I don't know if that's something worth the expense for an individual to invest 300 /mo. 

With so many lovers of Ripple (me included) I'm wondering if either 1) there's an underground engineering community that does build validators / servers for a distinct purpose. In that case i'm curious as to what that purpose would be, and/or 2) what, if any, incentives are there for an individual to take on a monthly cost to spin up their own validator / server?  Even if that means using python or java on log data to develop something that has been needed for a long time, and would be huge for the Ripple community, but no one has done it yet for whatever reason. 

Share this post


Link to post
Share on other sites

1) Not very underground, but I run my full history node to have local access to the full ledger history. The difference is for example when querying transactions from a single account from s2.ripple.com taking 20 seconds vs. maybe half a second from my node.

2) For me its curiosity mostly and also the fact that I like to support decentralization (my node is running for close to 5 years by now). I don't see what Python/Java/whatever have to do with running a node, could you elaborate what's that something that noone has done yet?

Share this post


Link to post
Share on other sites
2 hours ago, theGlitchKing said:

 I also know (at least I think I do)  that adding a validator to RippleNet may impact TPS positively

You may want to do some more reading. Adding a validating server wouldn't impact TPS positively. (I believe the main limits on TPS are disagreements between servers, and the slowest trusted validator's time doing things like verifying signatures... assuming the trusted validators all have sufficient I/O to meet their needs.) Most likely, adding a new validator will have no effect, although it could in theory decrease TPS by increasing the communication overhead with the other servers on the network. Right now the XRP Ledger isn't that close to capacity on throughput, though, so the easiest way to increase TPS is by... sending transactions. :P

Right now, most people who're running their own rippled server are either businesses like exchanges that offer XRP, or individuals running arbitrage bots to trade in the decentralized exchange that's built into the XRP Ledger—or trade in off-ledger exchanges and bridge the gaps between them. (I know there are least a couple people who make a tidy income doing that kind of stuff.) I'd love to see more creative, interesting uses of the XRP Ledger.

If you're a data scientist, you may want to look at the Data API and see if the stuff they're collecting interests you. That code's mostly open source, so you can even run your own historical database if you want to pull the data locally and analyze it yourself (although I think some of the import scripts for certain types of data are closed-source).

Share this post


Link to post
Share on other sites

@Sukrim  To answer your question:

Quote

I don't see what Python/Java/whatever have to do with running a node, could you elaborate what's that something that noone has done yet?

I honestly don't know, i'm just asking questions to probe the community, to see if there's a list of needs that I may be able to help with. I suppose this would be a case of 'i'd like to help, and I have many skills' . i'm just uncertain what use-case would be the best fit, and impact Ripple/RippleNet positively, or... even if I can.

Python and Java can be used to tie into things like Hadoop (think Spark here) or Splunk (to name a couple off the top of my head), I just can't see what data I would use, or to what end. I used to work for pretty much every major US bank, specifically with market data, and the only thing I've ever seen historical transaction data used for is either charting, or audit.

I followed the thread about creating an algorithm to determine xRapid transactions based on historical data. However from how I understand it, the task is quite daunting to be able to separate standard transactions from xRapid transactions within a combined historical data set. While this is a use case that would have positive impact, I don't think I could figure that one out. any others I simply am unaware of.

Share this post


Link to post
Share on other sites

It would be relatively easy to find potential xRapid transactions once you know what to look for. They would be either direct exchange --> exchange transactions or (less likely, but still consistent with the "less than 10 seconds" reports instead of "less than 5 seconds") exchange --> account --> exchange transactions. Aggregate virtual accounts by destination tag per exchange, look for transactions during business hours and calculate the total value transacted while looking at historic ticker data from these exchanges and then look for clusters that are only doing transactions that are more profitable than the FOREX rates at that time.

If you already know Spark, you could do this with Spark.

Data you'll need:

* FOREX rates

* Exchange trade tickers of a few exchanges where XRP is listed

* Ripple historic transactions, especially payments to/from exchanges and their hot wallets (I wouldn't recommend the data API because it can be flaky, but it might be an easier start)

Happy hunting!

The monthly cost to run just a random rippled node by the way is far below 300 USD (maybe about a tenth of that), if you want full history on a cloud  provider though be prepared to get out the big bucks because you'll need a couple TB of SSD storage. On AWS you can expect to pay about 1200-1500 USD/month last I checked their prices. It'll take you a few weeks or months to even get that data from other nodes, though it might get better with history sharding now.

Share this post


Link to post
Share on other sites

@theGlitchKing Welcome! This video by 3Blue1Brown is the clearest overview of blockchain architecture I've seen.  It doesn't all apply exactly to Ripple, but much of what's presented is analogous or otherwise related.  This plus his other video on sha256 forms a pretty solid ground.

As to server costs - Digital Ocean is the most competitive I've found for setting up a rippled server.  At the moment you could operate a full-history node for under $1000/month total including the SSD's for nudb.  It's also much easier to use and configure than AWS, and you could very easily resize your SSD's on the fly as you populate to keep the cost down initially.  Obviously the price for running at full history would continue to increase as more ledgers are added to the db, maybe you could go for the next three months at under $1000. It may even out somewhat as SSD's continue to drop in price and cloud providers reflect that in their price - though you pay for the lag.  Then again increased usage of the ledger could see an increase in the size of ledgers added to the DB daily that could outweigh any of that (perhaps ultimately significantly...).  I've wondered if it may be more cost-efficient to simply buy your own hardware and add SSD's at the market price as required.

Share this post


Link to post
Share on other sites
8 minutes ago, Professor Hantzen said:

I've wondered if it may be more cost-efficient to simply buy your own hardware and add SSD's at the market price as required.

I do this and so far it is much cheaper than renting someone else's computer. The downsides are that it uses a lot of bandwidth, something my ISP doesn't seem to care about, but that probably puts me in the very tail end of their customers I guess.

Share this post


Link to post
Share on other sites
On 7/3/2018 at 5:37 PM, mDuo13 said:

You may want to do some more reading. Adding a validating server wouldn't impact TPS positively. (I believe the main limits on TPS are disagreements between servers, and the slowest trusted validator's time doing things like verifying signatures... assuming the trusted validators all have sufficient I/O to meet their needs.) Most likely, adding a new validator will have no effect, although it could in theory decrease TPS by increasing the communication overhead with the other servers on the network. Right now the XRP Ledger isn't that close to capacity on throughput, though, so the easiest way to increase TPS is by... sending transactions. :P

Right now, most people who're running their own rippled server are either businesses like exchanges that offer XRP, or individuals running arbitrage bots to trade in the decentralized exchange that's built into the XRP Ledger—or trade in off-ledger exchanges and bridge the gaps between them. (I know there are least a couple people who make a tidy income doing that kind of stuff.) I'd love to see more creative, interesting uses of the XRP Ledger.

If you're a data scientist, you may want to look at the Data API and see if the stuff they're collecting interests you. That code's mostly open source, so you can even run your own historical database if you want to pull the data locally and analyze it yourself (although I think some of the import scripts for certain types of data are closed-source).

Can I get in touch with theese people settings up the arbitrage bots?what 's the name of the bots they use?

Share this post


Link to post
Share on other sites
15 hours ago, bodytexture said:

Can I get in touch with theese people settings up the arbitrage bots?what 's the name of the bots they use?

There is no name and there are not public bots available (as far as I know). It's mainly self-made bots.

Share this post


Link to post
Share on other sites
On 02/04/2018 at 3:33 AM, bodytexture said:

Can I get in touch with theese people settings up the arbitrage bots?what 's the name of the bots they use?

PM me if you want

Share this post


Link to post
Share on other sites

×