This document outlines how to configure a Yum repository for use with the Interset 4.2 installation. Note that many of these instructions will need to be adapted based on your actual environment.
An example script is provided attached to this article. Please note that this script is provided as-is with no implicit support, and will need modifications to function in any given environment.
There are two options for configuring a local mirrored repository - mirroring the Interset hosted repository (recommended), or configuring your own entirely from scratch.
Mirroring Interset's Repository
1) Download the attached configureMirror.sh script.
2) Modify the script and update the "localrepopath" variable to point to the directory that will be used for the mirror on your server (default: /var/www/html as per the default Apache web server config)
3) Run the configureMirror.sh script
Configuring your own repository
1) Create the Apache HTTP server
If you already have Apache HTTP server up and running, proceed to the next section.
As the root user on the mirror server machine install the Apache httpd server
yum install httpd
chkconfig httpd on
Create web server directories:
mkdir -p /var/www/html
2) Download the Hadoop repository tarballs from the web to the web server directory and untar.
cd /var/www/html
Amazon:
wget https://s3-us-west-1.amazonaws.com/theta-deployment/infrastructure/hadoop/ambari-2.2.1.1-centos6.tar.gz
tar xvf ambari-2.2.1.1-centos6.tar.gz
Amazon:
wget https://s3-us-west-1.amazonaws.com/theta-deployment/infrastructure/hadoop/HDP-2.3.4.7-centos6-rpm.tar.gz
tar xvf HDP-2.3.4.7-centos6-rpm.tar.gz
Amazon:
wget https://s3-us-west-1.amazonaws.com/theta-deployment/infrastructure/hadoop/HDP-UTILS-1.1.0.20-centos6.tar.gz
tar xvf HDP-UTILS-1.1.0.20-centos6.tar.gz
3) Configure network access
Open editor and add new line after the line with "dport 22 -j ACCEPT"
vim /etc/sysconfig/iptables
-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
service iptables restart
4) Start the web server and verify access to the newly created local repositories:
service httpd start
Ambari Base URL: http://<web_server_hostname>/AMBARI-2.2.1.1/centos6/2.2.1.1-70
HDP Base URL: http://<web_server_hostname>/HDP/centos6/2.x/updates/2.3.4.7
HDP-UTILS Base URL: http://<web_server_hostname>/HDP-UTILS-1.1.0.20/repos/centos6
Ambari PGP Key URL: http://<web_server_hostname>/AMBARI-2.2.1.1/centos6/2.2.1.1-70/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
5) Create web directory for the Interset software.
mkdir -p /var/www/html/interset-4.2
cd /var/www/html/interset-4.2
6) Download the latest Interset software archive.
Download files into an 'interset-4.2' folder:
wget <URLs to be provided by Interset>
7) Create Interset repodata:
Verify that createrepo utility is installed on your web server.
If not, then run:
yum install createrepo
Navigate to directory:
cd /var/www/html/interset-4.2/
Execute command:
createrepo .
8) Verify that you can browse to the newly created local repository and see repodata directory
at the same hierarchy level as rpm packages:
Interset Base URL: http://<web_server_hostname>/interset-4.2/
9) On the web server create web directory for the Elasticsearch software.
mkdir -p /var/www/html/elasticsearch-1.7
10) On the web server create remote elasticsearch repository as:
[remote:elasticsearch-1.7]
name=Elasticsearch Repository for 1.7.X Packages
baseurl=http://packages.elastic.co/elasticsearch/1.7/centos
gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
gpgcheck=1
enabled=0
11) Download rpms using remote elasticsearch repository:
reposync -p /var/www/html/elasticsearch-1.7 -n --norepopath -r 'remote:elasticsearch-1.7'
12) Create repodata for elasticsearch-1.7 repository:
( cd /var/www/html/elasticsearch-1.7 && createrepo . )
13) Download gpgkey into elasticsearch-1.7 repository:
cd /var/www/html/elasticsearch-1.7
wget http://packages.elastic.co/GPG-KEY-elasticsearch
14) Check local elasticsearch repo content and compare structure with original source:
Local: http://<web_server_hostname>/elasticsearch-1.7/
Source: http://packages.elastic.co/elasticsearch/1.7/centos http://packages.elastic.co/elasticsearch-1.7/centos
Local gpgkey: http://<web_server_hostname>/GPG-KEY-elasticsearch
Source gpgkey: http://packages.elastic.co/GPG-KEY-elasticsearch
15) On the web server create web directory for the Datastax software (Cassandra).
mkdir -p /var/www/html/datastax
16) On the web server create remote datastax repository as:
vim /etc/yum.repos.d/remote:datastax.repo
[remote:datastax]
name = DataStax Repo for Apache Cassandra
baseurl = http://rpm.datastax.com/community
gpgcheck = 0
enabled = 0
17) Download Datastax rpms using remote datastax repository
reposync -p /var/www/html/datastax -n --norepopath -r 'remote:datastax'
18) Create repodata for datastax repository:
( cd /var/www/html/datastax && createrepo . )
Downloading gpgkey into datastax repository is not required as datastax has no gpgkey
19) Check local datastax repo content and compare structure with original source:
Local: http://<web_server_hostname>/datastax/
Source: http://rpm.datastax.com/community/
Local repository structure must be identical to the source.
However, the number of files on local will be less than on the source as only new packages were downloaded.
20) On the web server create web directory for the Nginx software.
mkdir -p /var/www/html/nginx
21) On the web server create remote nginx repository as:
vim /etc/yum.repos.d/remote:nginx.repo
[remote:nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/6/x86_64/
gpgcheck=0
enabled=0
22) Download rpms using remote nginx repository
reposync -p /var/www/html/nginx -n --norepopath -r 'remote:nginx'
23) Create repodata for nginx repository:
( cd /var/www/html/nginx && createrepo . )
24) Check local nginx repo content and compare structure with original source:
Local: http://<web_server_hostname>/nginx/
Source: http://nginx.org/packages/centos/6/x86_64/
Local nginx repository structure must be identical to the source
25) For production implementation of local repositories mirror, CentOS base and CentOS updates
should be created on the web server as well.
Instructions for setting up CentOS local repositories on the web server are provided in "Appendix" section of this document.
- Configuring Data Node (Ambari platform), Client Node (Analytics / Reporting) and Endpoint Node to use local repos on a web server.
On the client systems configure yum to use local mirror repositories:
1) On all 3 nodes:
Create interset.repo file and configure it to use mirror:
vim /etc/yum.repos.d/interset.repo
[interset-4.1]
name=interset-4.1
baseurl=http://<web_server_hostname>/interset-4.1/el6
gpgcheck=0
enabled=1
priority=1
2) On Data Node:
Create ambari.repo file and configure it to use mirror:
vim /etc/yum.repos.d/ambari.repo
[Updates-ambari-2.2.1.1]
name=ambari-2.2.1.1-Updates
baseurl=http://<web_server_hostname>/AMBARI-2.2.1.1/centos6/2.2.1.1-70
gpgcheck=1
gpgkey=http://<web_server_hostname>/AMBARI-2.2.1.1/centos6/2.2.1.1-70/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
3) On Client Node (Analytics / Reporting):
- a) Create elasticsearch.repo file and configure it to use mirror:
vim /etc/yum.repos.d/elasticsearch.repo
[elasticsearch-1.7]
name=Elasticsearch Repository for 1.7.X Packages
baseurl=http://<web_server_hostname>/elasticsearch-1.7
gpgkey=http://<web_server_hostname>/GPG-KEY-elasticsearch
gpgcheck=1
enabled=1
- b) Create nginx.repo file and configure it to use mirror:
vim /etc/yum.repos.d/nginx.repo
[nginx]
name=nginx repo
baseurl=http://<web_server_hostname>/nginx/
gpgcheck=0
enabled = 1
4) On Endpoint Node:
- a) Create datastax.repo file and configure it to use mirror:
vim /etc/yum.repos.d/datastax.repo
[datastax]
name = DataStax Repo for Apache Cassandra
baseurl = http://<web_server_hostname>/datastax/
gpgcheck = 0
enabled = 1
- b) Create nginx.repo file and configure it to use mirror:
vim /etc/yum.repos.d/nginx.repo
[nginx]
name=nginx repo
baseurl=http://<web_server_hostname>/nginx/
gpgcheck=0
enabled = 1
5) To clear yum repositories and verify local repo status, execute:
yum clean all
yum repolist
Note: All configured local repositories should not have status 0!
6) If local repositories for CentOS base and CentOS updates were created on the web server then on all 3 nodes:
- a) Verify that all other CentOS-*.repo files are disabled:
CentOS-Debuginfo.repo
CentOS-fasttrack.repo
CentOS-Media.repo
CentOS-Vault.repo
If not then set:
enabled=0
- b) Backup existing CentOS-Base.repo file:
mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.bak
- c) Create new CentOS-Base.repo file and configure it to use mirror:
vim /etc/yum.repos.d/CentOS-Base.repo
[base]
name=CentOS-$releasever - Base
baseurl=http://<web_server_hostname>/centos/6/os/x86_64
gpgcheck=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
#released updates
[updates]
name=CentOS-$releasever - Updates
baseurl=http://<web_server_hostname>/centos/6/updates/x86_64/
gpgcheck=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6
- d) To clear yum repositories and verify local repo status, execute:
yum clean all
yum repolist
Comments