Follow

Error outputted in Flume log "_org.apache.kafka.common.config.ConfigException_ No resolvable bootstrap urls given in bootstrap.servers_"

Issue

The following error can be outputted in Flume log(s):

  • apache.kafka.common.config.ConfigException: No resolvable bootstrap urls given in bootstrap.servers

 The error can be thrown in the following log(s):

  • flume-interset_<ds>_events_<did>_<tid>_csv_transform.log
  • flume-interset_<ds>_raw_<did>_<tid>_csv_multiline_extract.log 

Cause

This is generally caused by the Flume agent not being able to connect to the Kafka Broker(s). 

Resolution Steps

NOTE: This information is only useful for CSV data ingest using Flume

There are a two steps to take in order to resolve the issue:

  • Validate the Kafka Broker(s) port accessibility
  • Validate STREAM NODE FQDN in Flume configuration/Regenerate Flume configuration

Validate Kafka Broker(s) port accessibility

  1. SSH to the STREAM NODE as the Interset User.
  2. Type in the following command telnet to the Kafka Broker on port 6667:
    • sudo telnet <STREAM_NODE_FQDN> 6667
  3. If the telnet command succeeds, please continue to STREAM NODE FQDN in Flume configuration or Regenerate Flume configuration section
  4. If the telnet command fails, it will output would be similar to the following:
    • telnet: connect to address XXX.XXX.XXX.XXX: Connection refused
  5. Type in the following command to verify if iptables/ip6tables/firewalld is running:
    • For EL6:
      • sudo service iptables status
      • sudo service ip6tables status
    • For EL7:
      • sudo systemctl status firewalld
      • sudo systemctl status iptables
  6. If iptables/ip6tables/firewalld is running, type in the following command to stop the daemon(s).
    1. NOTE: Please verify with your network security team if this is appropriate
      • For EL6:
        • sudo service iptables stop
        • sudo service ip6tables stop
      • For EL7:
        • sudo systemctl stop firewalld
        • sudo systemctl stop iptables
  7. To prevent this from occurring in the future, iptables/ip6tables/firewalld can be disabled on the Linux systems by typing in the following command:
    • NOTE: Please verify with your network security team if this is appropriate
      • For EL6:
        • sudo chkconfig iptables off
        • sudo chkconfig ip6tables off
      • For EL7:
        • sudo systemctl disable firewalld
        • sudo systemctl disable iptables
  8. Please repeat Step 2 to ensure port 6667 is opened on the STREAM NODE

Below are two options that may resolve the STREAM NODE FQDN and port issues:

  • Validate STREAM NODE FQDN in Flume configuration
  • Regenerate Flume configuration

NOTE: If “Validate Elasticsearch cluster name in Flume configuration” option becomes too difficult, it is recommended to regenerate the Flume config to minimize mistakes.

Validate STREAM NODE FQDN in Flume configuration

  1. Open up a web browser and navigate to the Ambari UI URL:
  2. Log in to the Ambari UI as the Ambari admin. The default credentials for the Ambari admin user are as follow:
    • Username: admin
    • Password: admin
  3. Once logged in, click on Flume (from component list)
  4. In Flume, click on the Configs tab
  5. Click the Groups dropdown and select Ingest
  6. Under the flume.conf section, copy the Flume configuration to a text editor.
  7. In the text editor, look for the following 2 parameters:
    • interset_<ds>_raw_<did>_<tid>_csv_multiline_extract.channels.kafkaChannel.brokerList
    • interset_<ds>_events_<did>_<tid>_csv_transform.channels.kafkaChannel.brokerList
  8. Modify/validate the value specified for each parameter set to the STREAM NODE FQDN along with the proper port.
    • EXAMPLE: stream.acme.com:6667
      • NOTE: If there are multiple STREAM NODES, the value for each parameter will resemble the following:
        • acme.com:6667,stream2.acme.com:6667
  9. After the STREAM NODE FQDN(s) value is/are modified/validated, copy the flume configuration from the text editor and replace the flume configuration in the flume.conf section in the Ambari UI.
  10. Once the configuration has been replaced, click Save to continue, and then OK.
  11. Click the Restart button at top of the webpage and select Restart All Affected.
  12. Click Confirm Restart All to restart Flume and use the new configuration
  13. Click OK once Flume restarts successfully

Regenerate Flume configuration

  1. To regenerate the Flume ingest configuration, please see the Configure a New Data Source section in the Interset <version> Installation and Configuration Guide guide.

Applies To

  • Interset 5.4.x or higher 
Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments