Follow

Spark Workers Do Not Appear In Web Interface

When accessing the Spark web interface (e.g. http://<ANALYTICS>:8080), the remote Spark Workers do not appear except any running on the Master server.

Spark relies on the slaves.template file to manage instances in its environment. This file is located in /opt/interset/spark/conf, must be renamed to slaves, and must be copied to this directory on all remote Spark installations. The slaves file contains the hostnames of all Workers in the environment. These values should be listed 1 per line, and should correspond with the entries in the /etc/hosts file*.

If these files have been copied to all servers, also ensure that they are owned by the proper user (interset) and have appropriate permissions (minimum 400/r--).

 

*Apache best practices note that hostnames used in its configuration files should be hard coded in the systems' /etc/hosts file. More information on this can be found @ https://wiki.apache.org/hadoop/UnknownHost.

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments

Powered by Zendesk