Follow

Ingesting Squid Data as a Datasource

Issue

When attempting to ingest Squid data into the Interset system for analysis, there are several changes that need to be made to the flume config to get this data ingested correctly for processing.

Cause

Due to differences in the log structure and timestamp, standard configs will not work.

Resolution Step

The attached flume config will need to be used to replace the existing flume config in the environment in order for Squid data to be ingested and prepared for analysis.

1.  Using the attached config, open in a text editor and copy the content.

2.  Open the Ambari console and click on Flume on the left had side.

3.  Click on Configs link at the top of the Flume section

4.  Open the Ingest config and edit.

5.  Paste the content of the flume config that was copied in step one into the existing config.  Please ensure that you select all of the content of the existing config and delete it before you paste the new content.

6.  Save the config and restart Flume.

7.  Provided that the file to be ingested is in the source location, once the processes restart the file should be picked up and processed as per normal operation.

Applies To

  • Interset 5.5.X or higher 
Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments