GridGain Community Edition

Back to discussions
Expand all | Collapse all

Spark Ignite embedded

Jump to Best Answer

  • 1.  Spark Ignite embedded

     
    Posted 07-16-2020 10:10 AM
    Hello,

    I am trying to build a very simple spark job with Apache Ignite embedded on Yarn cluster for Test purposes

    I am getting : TcpDiscoverySpi: Failed to connect to any address from IP finder

    Do you have  an example to run on Yarn cluster ?

    Thanks for your help


    ------------------------------
    pascal ka
    Senior Big Data developper
    Bank Leumi
    ------------------------------


  • 2.  RE: Spark Ignite embedded
    Best Answer

    Posted 07-17-2020 06:02 AM
    Hello!

    Unfortunately, it's hard to say. Normally YARN will take care of discovery on its side. But it's possible that Spark brings up client nodes only. Please provide more detail.

    Regards,

    ------------------------------
    Ilya Kasnacheev
    Community Support Specialist
    GridGain
    ------------------------------

    Original Message


  • 3.  RE: Spark Ignite embedded

    Posted 07-17-2020 08:57 AM
      |   view attached
    check the attached document, I have used jdbc thin client.

    Thanks
    Sri

    ------------------------------
    Sri Tummala
    ------------------------------

    Attachment(s)

    docx
    IgniteOnAWS.docx   32 KB 1 version
    Original Message


  • 4.  RE: Spark Ignite embedded

    Posted 07-30-2020 06:46 AM
    Hello!

    TcpDiscoverySpi is entirely unrelated to jdbc thin client, so I don't understand how your cluster is formed.

    The document also has nothing about YARN.
    Can you elaborate?

    Regards,

    ------------------------------
    Ilya Kasnacheev
    Community Support Specialist
    GridGain
    ------------------------------

    Original Message