Follow

How To: Configure Elasticsearch Security with X-Pack

Add certificates to nodes

If a user does not already have signed certificates for each node, the following steps will create an interset CA and register it in the java keystore on all nodes. It will also generate, on each node, a certificate signed by the interset CA:

  1. On the ambari node, run 
    /opt/interset/bin/sysprep/scripts/security_create_ca
  2. Copy the resulting /opt/interset/ssl/interset_ca.tar.gz to each node in the cluster. 
  3. On each node (including ambari)
    • Untar interset_ca.tar.gz to location /opt/interset/ssl/ca. (can be skipped on ambari node):
      tar xzf interset_ca.tar.gz && mv ca /opt/interset/ssl/ca

    • Create a signed certificate:
      /opt/interset/bin/sysprep/scripts/security_create_signed_cert

      The output should be similar to:
      Add certificate to node keystore by running /opt/interset/bin/sysprep/scripts/security_add_cert --cert /opt/interset/ssl/ca/certs/hostname-signed.crt

      NOTE: In case of JAVA upgrade performed on the system afterwards, you would need to re-run commands listed above and restart the Elasticsearch service!
    • To add the certificate to the node keystore, run the following with the --cert value provided as output at the end of step above:
      /opt/interset/bin/sysprep/scripts/security_add_cert --cert /opt/interset/ssl/ca/certs/hostname-signed.crt

    • The resulting contents of the keystore should be that each node has it's own signed certificate, and the interset_ca, for example if you've got 3 nodes:
      node1 keystore - node_1_cert, interset_ca
      node2 keystore - node_2_cert, interset_ca
      node3 keysore - node_3_cert, interset_ca
      etc...

If a user already has a set of signed certificates for each node:

  1. Copy each certificate to its correspondent node.
  2. Run:
    /opt/interset/bin/sysprep/scripts/security_add_cert --cert /path/to/cert
    to create the node jks and add the certificate to it.

 

Install XPack on Search Nodes

Xpack offers additional features for Elasticsearch and Kibana. See https://www.elastic.co/subscriptions for more details. Note that this installation assumes an internet connected system.

On each Elasticsearch node: 

  1. Install x-pack
    bin/elasticsearch-plugin install x-pack
    b
    in/kibana-plugin install x-pack
  2. Copy the keystore to a location where Elasticsearch can read it:

    sudo cp /opt/interset/ssl/certs/interset.jks /etc/elasticsearch/x-pack/interset.jks
    sudo chown -R elasticsearch:elasticsearch /etc/elasticsearch/x-pack/

  3.  Add the following to /etc/elasticsearch/elasticsearch.yml:

    # create an anoynomous user to allow interaction without auth
    xpack.security.authc:
      anonymous:
        username: anonymous
        roles: superuser
        authz_exception: true

    # Setup ssl
    xpack.security:
      http.ssl:
        keystore.path: /etc/elasticsearch/x-pack/interset.jks
        keystore.password: interset
        enabled: true
      transport.ssl:
        keystore.path: /etc/elasticsearch/x-pack/interset.jks
        keystore.password: interset
        enabled: true

  4. If there are any specific xpack features that you want to turn off in Elasticsearch, add and adjust the following in /etc/elasticsearch/elasticsearch.yml on each Elasticsearch node:

    # Xpack features
    xpack.security.enabled: true
    xpack.monitoring.enabled: false
    xpack.graph.enabled: false
    xpack.watcher.enabled: false

  5. Do a rolling restart of Elasticsearch by executing the following on each search node, replacing <esHost> with the hostname of the node you wish to issue the command to:

    • curl -XPUT <esHost>:9200/_cluster/settings -d '{ "persistent": { "cluster.routing.allocation.enable": "none" } }'

    • curl -XPOST <esHost>:9200/_flush

    • Restart Elasticsearch

      # EL6 servers
      sudo service elasticsearch restart
      # EL7 servers
      sudo systemctl restart elasticsearch

    • Be sure to send this query to the node being restarted so that the allocation isn't accidentally changed before the restarted node has joined the cluster again:

      curl -XPUT https://<nodeBeingRestarted>:9200/_cluster/settings?pretty --cacert /opt/interset/ssl/ca/certs/cacert.pem -d '{ "persistent": { "cluster.routing.allocation.enable": "all" } }

    • Wait for the cluster status to be green again before continuing on to the next node:

      curl -XGET https://<esHost>::9200/_cluster/health?pretty --cacert /opt/interset/ssl/ca/certs/cacert.pem

  6. Security is available for free for a month, and then requires a paid Gold or Platinum licence. Contact the Elastic sales team to obtain one.

  7. Add the licence to Elasticsearch:
    • curl -XPUT -u elastic:changeme https://<esHost>:9200/_xpack/license?acknowledge=true --insecure -d @/data/licence.json

      NOTE: Because the license is installed through curl, the command only ever needs to be run once. The cluster will share it with all nodes.

    • You can check that the licence was properly installed with the following:

      curl -XGET https://<esHost>:9200/_xpack?pretty --cacert /opt/interset/ssl/ca/certs/cacert.pem
      curl -XGET https://<esHost>:9200/_xpack/license?pretty --cacert /opt/interset/ssl/ca/certs/cacert.pem

 

Configuring Kibana with x-pack

Note that this installation assumes an internet connected system.

  1. On the reporting node, install x-pack on Kibana if you want to have access to the search profiler, monitoring UI, user management UI, etc.

    /usr/share/kibana/bin/kibana-plugin install xpack
  2. Include the following settings in kibana.yml (/opt/kibana/config/kibana.yml):

    elasticsearch.url: "https://<host>:9200"
    elasticsearch.username: "elastic"
    elasticsearch.password: "changeme"
    elasticsearch.ssl.ca: /path/to/your/CA.pem

  3. If there are any specific xpack features that you want to turn off in Kibana, add and adjust the following in /etc/kibana/kibana.yml on the reporting node:

    #Kibana xpack features
    xpack.security.enabled: true
    xpack.monitoring.enabled: false
    xpack.graph.enabled: false
    xpack.reporting.enabled: false

  4. Restart Kibana:

    # EL6 servers
    sudo service kibana restart
    # EL7 servers
    sudo systemctl restart kibana

Configuring Reporting to interact with Elasticsearch over SSL

  1. On the Reporting node, edit /opt/interset/etc/investigator.yml to include:

    esSearch:

      esUser: elastic
      esPassword: changeme
      esEnableSSL: true
      esKeystorePath: /opt/interset/ssl/certs/interset.jks
      esKeystorePassword: interset

  2. Restart Reporting

    sudo monit -g reporting restart

Configuring Analytics to interact with Elasticsearch over SSL

  1. On the analytics node, edit /opt/interset/analytics/conf/interset.conf to include:

    esXPackUser=elastic:changeme
    keystorePath=/opt/interset/ssl/certs/interset.jks
    keystorePassword=interset
    sslEnabled=true

Configuring Flume to Interact with Elasticsearch over SSL

  1. Stop Flume.
  2. Add the following to the Elasticsearch sink section of each flume configuration, replacing <TYPE> with the type of data source used in the config (e.g. ad, repo, etc...), and <DID> and <TID> with the specified DID and TID for the config, respectively:

    interset_<TYPE>_events_<DID>_<TID>_es.sinks.esSink.client.xpack.security.transport.ssl.enabled = true
    interset_<TYPE>_events_<DID>_<TID>_es.sinks.esSink.client.xpack.security.transport.ssl.keystore.path = /opt/interset/ssl/certs/interset.jks
    interset_<TYPE>_events_<DID>_<TID>_es.sinks.esSink.client.xpack.security.transport.ssl.keystore.password = interset
    interset_<TYPE>_events_<DID>_<TID>_es.sinks.esSink.client.xpack.security.transport.ssl.truststore.path = /opt/interset/ssl/certs/interset.jks
    interset_<TYPE>_events_<DID>_<TID>_es.sinks.esSink.client.xpack.security.transport.ssl.truststore.password = interset

    interset_<TYPE>_events_<DID>_<TID>_es.sinks.esSink.serializer.client.xpack.security.transport.ssl.enabled = true
    interset_<TYPE>_events_<DID>_<TID>_es.sinks.esSink.serializer.client.xpack.security.transport.ssl.keystore.path = /opt/interset/ssl/certs/interset.jks
    interset_<TYPE>_events_<DID>_<TID>_es.sinks.esSink.serializer.client.xpack.security.transport.ssl.keystore.password = interset
    interset_<TYPE>_events_<DID>_<TID>_es.sinks.esSink.serializer.client.xpack.security.transport.ssl.truststore.path = /opt/interset/ssl/certs/interset.jks
    interset_<TYPE>_events_<DID>_<TID>_es.sinks.esSink.serializer.client.xpack.security.transport.ssl.truststore.password = interset


    # If violations are configured as well, add the following to esSink flume configurations for violations:

    interset_violations_<TID>_es.sinks.esSink.client.xpack.security.transport.ssl.enabled = true
    interset_violations_<TID>_es.sinks.esSink.client.xpack.security.transport.ssl.keystore.path = /opt/interset/ssl/certs/interset.jks
    interset_violations_<TID>_es.sinks.esSink.client.xpack.security.transport.ssl.keystore.password = interset
    interset_violations_<TID>_es.sinks.esSink.client.xpack.security.transport.ssl.truststore.path = /opt/interset/ssl/certs/interset.jks
    interset_violations_<TID>_es.sinks.esSink.client.xpack.security.transport.ssl.truststore.password = interset

    interset_violations_<TID>_es.sinks.esSink.serializer.client.xpack.security.transport.ssl.enabled = true
    interset_violations_<TID>_es.sinks.esSink.serializer.client.xpack.security.transport.ssl.keystore.path = /opt/interset/ssl/certs/interset.jks
    interset_violations_<TID>_es.sinks.esSink.serializer.client.xpack.security.transport.ssl.keystore.password = interset
    interset_violations_<TID>_es.sinks.esSink.serializer.client.xpack.security.transport.ssl.truststore.path = /opt/interset/ssl/certs/interset.jks
    interset_violations_<TID>_es.sinks.esSink.serializer.client.xpack.security.transport.ssl.truststore.password = interset

    Note: If you wish to specify credentials to access the Elasticsearch cluster from Flume, also include the following line:

    interset_<TYPE>_events_<DID>_<TID>_es.sinks.esSink.client.xpack.security.user = username:password
  3. Start Flume.
Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments

Powered by Zendesk