Follow

Change interval for Application Reports when running Analytics

By default, when running Analytics (analytics.sh), a report on the status of the currently running Spark job will be printed every second. This can result in extremely large amounts of output being generated, especially when redirected to a log file.

To change this, add the following parameter under Custom spark-defaults in Ambari:

key: spark.yarn.report.interval
value: 30000

Note that this value is specified in milliseconds, and can be set to any desired value.

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments

Powered by Zendesk