Follow

Purge all data for a tenant from a Interset Cluster

Applies To

  • Interset 5.4.x or higher

Summary

This is a How To article on how to purge all data for a tenant from a Interset Cluster. 

WARNING: THIS WILL PERMANENTLY REMOVE ALL DATA FROM YOUR INTERSET SYSTEM. PLEASE BE ABSOLUTELY CERTAIN THAT THIS IS WHAT YOU WANT TO DO. USE THIS ONLY UNDER THE GUIDANCE OF INTERSET SUPPORT.

Steps

Global Reset:

# Completely purge and recreate analytics db in HBase

  • ./sql.sh --action purge --force true --dbServer <zookeeper-server:2181:/hbase-unsecure>
  • ./sql.sh --action create --dbServer <zookeeper-server:2181:/hbase-unsecure>

 # Delete and recreate topics in Kafka

  • ./kafka-topics.sh --topic <topicName> --delete --zookeeper <zookeeperURL>
  • ./kafka-topics.sh --topic <topicName> --create --replication-factor 1 --partitions 8 --zookeeper <zookeeperURL>

 # Delete all data from ES

  • curl -XDELETE 'http://<elasticsearch-url>:9200/_all'

TID (Tenant) Specific:

Delete analytics data from HBase

  • ./sql.sh --action clean --tenantID <TID> --dbServer <zookeeper-server:2181:/hbase-unsecure>

 Delete raw data from HBase

  • ./sql.sh --dbServer<zookeeper-server:2181:/hbase-unsecure> --action console
  • DELETE FROM OBSERVED_ENTITY_RELATION_MINUTELY_COUNTS WHERE TID = '<TID>';

 Recreate schema in HBase

  • ./sql.sh --action migrate --tenantID <TID> --dbServer <zookeeper-server:2181:/hbase-unsecure>

 Delete and recreate topics in Kafka

  • ./kafka-topics.sh --topic <topicName> --delete --zookeeper <zookeeperURL>
  • ./kafka-topics.sh --topic <topicName> --create --replication-factor 1 --partitions 8 --zookeeper <zookeeperURL>

Delete data from ES

  • curl -XDELETE 'http://<elasticsearch-url>:9200/interset_*_<TID>-*'

Delete river sync data from ES

  • curl -XDELETE 'http://<elasticsearch-url>:9200/<TID>_*’ 

If using non-Endpoint data, the data will need to be re-ingested into the system from the source.

You can use the attached renameCompleted.sh script, updated for your specific path to log files, in order to rename all of your files that have been renamed to .COMPLETED after being previously ingested.

If using Endpoint data, you will need to run flowdtokafka.py to push the raw data from Cassandra back into Kafka for re-ingest. Perform the following steps on the Endpoint node.

  • source /opt/interset/flow/flowd_environment/bin/activate
    cd /opt/interset/flow/tools/flow_to_kafka
    python flowdtokafka.py --kafka-server <comma separated list of Kafka brokers>

 

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments

Powered by Zendesk