Follow

How To: Purge all Data from a System (or from a specific Tenant ID)

WARNING: THIS WILL PERMANENTLY REMOVE ALL DATA FROM YOUR INTERSET SYSTEM. PLEASE BE ABSOLUTELY CERTAIN THAT THIS IS WHAT YOU WANT TO DO. USE THIS ONLY UNDER THE  GUIDANCE OF INTERSET SUPPORT.

 

Global Reset:

# Completely purge and recreate analytics db in HBase

./sql.sh --action purge --force true --dbServer <zookeeper-server:2181:/hbase-unsecure>

./sql.sh --action create --dbServer <zookeeper-server:2181:/hbase-unsecure>

 

# Delete and recreate topics in Kafka

./kafka-topics.sh --topic <topicName> --delete --zookeeper <zookeeperURL>

./kafka-topics.sh --topic <topicName> --create --replication-factor 1 --partitions 8 --zookeeper <zookeeperURL>

 

# Delete all data from ES

curl -XDELETE 'http://<elasticsearch-url>:9200/_all'

 

TID (Tenant) Specific:

Delete analytics data from HBase

./sql.sh --action clean --tenantID <TID> --dbServer <zookeeper-server:2181:/hbase-unsecure>

 

Delete raw data from HBase

./sql.sh --dbServer<zookeeper-server:2181:/hbase-unsecure> --action console

DELETE FROM OBSERVED_ENTITY_RELATION_MINUTELY_COUNTS WHERE TID = '<TID>';

 

Recreate schema in HBase

./sql.sh --action migrate --tenantID <TID> --dbServer <zookeeper-server:2181:/hbase-unsecure>

 

Delete and recreate topics in Kafka

./kafka-topics.sh --topic <topicName> --delete --zookeeper <zookeeperURL>

./kafka-topics.sh --topic <topicName> --create --replication-factor 1 --partitions 8 --zookeeper <zookeeperURL>

 

Delete data from ES

curl -XDELETE 'http://<elasticsearch-url>:9200/interset_*_<TID>-*'

 

Delete river sync data from ES

curl -XDELETE 'http://<elasticsearch-url>:9200/<TID>_*’

 

 

If using non-Endpoint data, the data will need to be re-ingested into the system from the source.

You can use the attached renameCompleted.sh script, updated for your specific path to log files, in order to rename all of your files that have been renamed to .COMPLETED after being previously ingested.

 

If using Endpoint data, you will need to run flowdtokafka.py to push the raw data from Cassandra back into Kafka for re-ingest. Perform the following steps on the Endpoint node.

source /opt/interset/flow/flowd_environment/bin/activate
cd /opt/interset/flow/tools/flow_to_kafka
python flowdtokafka.py --kafka-server <comma separated list of Kafka brokers>

 

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments

Powered by Zendesk