Ricevi gli ultimi aggiornamenti da Hortonworks tramite e-mail

Una volta al mese, ricevi gli approfondimenti, le tendenze, le informazioni analitiche e la conoscenza approfondita dei big data.

Sign up for the Developers Newsletter

Una volta al mese, ricevi gli approfondimenti, le tendenze, le informazioni analitiche e la conoscenza approfondita dei big data.

invito all'azione

Per iniziare

cloud

Sei pronto per cominciare?

Scarica Sandbox

Come possiamo aiutarti?

* Ho compreso che posso disdire in qualsiasi momento. Sono inoltre a conoscenza delle informazioni aggiuntive presenti nella informativa sulla privacy di Hortonworks.
chiudiPulsante di chiusura
August 09, 2018
diapositiva precedentediapositiva successiva

Distributed Pricing Engine using Dockerized Spark on YARN w/ HDP 3.0 [Part 4/4]

This is the finale of the blog series (see part 1, part 2, part 3) where having discussed the problem domain, looked at the functional and architectural aspects and prepared the environment, we are now ready to execute a few pricing calculations.

SSH into the cluster gateway node and download the following from repo:

  • compute/compute-engine-spark-1.0.0.jar
  • compute/scripts/compute-price.sh

Notice the directives around using Docker as executor env for Spark on YARN in client mode.

You should now be ready to simulate a distributed pricing compute using the following command:

./compute-price.sh <metric> <numTrades> <numSplits>

where metric:

  • FwdRate: Spot Price of Forward Rate Agreement (FRA)
  • NPV: Net Present Value of a vanilla fixed-float Interest Rate Swap (IRS)
  • OptionPV: Net Present Value of a European Equity Put Option average over multiple algorithmic calcs (Black-Scholes, Binomial, Monte Carlo)

eg: ./compute-price.sh OptionPV 5000 20

And see the job execute as follows:

Wrapping up …

HDP 3.0 is pretty awesome right!!! It is now GA and I can tell you, if a decade ago you thought Hadoop was exciting, this will blow your mind away!!

In this blog series, we’ve just scratched the surface and looked at only one of the myriad compute centric aspects of innovation in the platform. For a more detailed read on platform capabilities, direction and unbound possibilities I urge you to read the blog series from folks behind this.

Comments

Roshni Khanna says:

I recently found much useful information in your website especially this blog page. Among the lots of comments on your articles. Thanks for sharing. http://www.feeltheheaven.com/

Lascia una risposta

L'indirizzo email non verrà reso pubblico. I campi obbligatori sono segnalati con *

If you have specific technical questions, please post them in the Forums