Page History

Recipe instructions for building your own Avalon setup from scratch on CentOS or Red Hat Enterprise Linux

Install EPEL

Code Block
rpm -ivh http://linux.mirrors.es.net/fedora-epel/6/i386/epel-release-6-8.noarch.rpm

This package has libyaml-devel which is required by ruby and not provided by Redhat.

Make sure a valid hostname is resolvable

The default hostname is “avalon.dev”, so name the machine this and enter it into /etc/hosts

Code Block
% hostname avalon.dev % cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 avalon.dev

Add the NUL repository
Anchor
NUL
NUL

Create the NUL repository config file:

Code Block
vi /etc/yum.repos.d/nul-public.repo

Append the following code:

Code Block
[nul_public] name=NUL Library Public RHEL repository baseurl=http://yumrepo-public.library.northwestern.edu/x86_64 enabled=1 gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-nul cost=150

Install and place the NUL GPG key in the proper location:

Code Block
curl http://yumrepo-public.library.northwestern.edu/RPM-GPG-KEY-nul -o /etc/pki/rpm-gpg/RPM-GPG-KEY-nul

Disable SELinux

Code Block
echo 0 > /selinux/enforce vi /etc/selinux/config #change the value of `SELINUX` from `enforcing` to `permissive`

Stop iptables
Anchor
iptables
iptables

Code Block
service iptables stop

Install Development Libraries

Code Block
yum groupinstall "Development Tools"

Install development packages for building Ruby

Code Block
yum install readline-devel zlib-devel libyaml-devel libffi-devel openssl-devel libxml2-devel libxslt-devel

Tomcat

Install Apache Tomcat with the nulrepo-public repository configured and enabled.

Code Block
yum install tomcat service tomcat start vi /usr/local/tomcat/conf/server.xml

on line 70, change the Tomcat connector port from 8080 to 8983:

<Connector port="8983" protocol="HTTP/1.1"

Add Tomcat manager user

By default, no user has access to the Tomcat Manager App.

Code Block
vi /usr/local/tomcat/conf/tomcat-users.xml

Define a role and a user in tomcat-users.xml with a manager-gui role. Below is a very basic example. Use more secure log in credentials than the example please.

Code Block
<tomcat-users> <role rolename="manager-gui"/> <user username="admin" password="tomcat" roles="manager-gui"/> </tomcat-users>

Restart Tomcat

Code Block
service tomcat restart

See if you can log in to the manager app: http://<server host name>:8983/manager/html

Try it out on your local machine and on another machine. If you can't reach the app from another machine, your iptables might need to be changed to allow access.

Solr

Avalon makes use of Solr through the Blacklight gem for faceting and relevance-based searching.

Download the solr tarball and unpack it to /usr/local/solr/:

from http://www.apache.org/dyn/closer.cgi/lucene/solr/4.2.0

Code Block

wget http://archive.apache.org/dist/lucene/solr/4.2.0/solr-4.2.0.tgz
tar xvf solr-4.2.0.tgz
mv solr-4.2.0 /usr/local/solr
cd /usr/local/solr/dist/
cp solr-4.2.0.war /usr/local/tomcat/webapps/solr.war
mv solr-4.2.0.war /usr/local/solr/solr.war

Place into /usr/local/tomcat/conf/Catalina/localhost/solr.xml:

Code Block
<Context docBase="/usr/local/solr/solr.war" debug="0" crossContext="true"> <Environment name="solr/home" type="java.lang.String" value="/usr/local/solr" override="true"/> </Context>

Place into /usr/local/solr/solr.xml:

Code Block

<?xml version="1.0" encoding="UTF-8" ?>
  <solr persistent="true">
    <cores defaultCoreName="avalon" adminPath="/admin/cores" zkClientTimeout="${zkClientTimeout:15000}" hostPort="8983" hostContext="solr">
    <core instanceDir="avalon/" name="avalon"/>
  </cores>
</solr>

Create directories and move files around

Code Block

mkdir /usr/local/solr/lib
mv /usr/local/solr/dist/* /usr/local/solr/lib/
mv /usr/local/solr/contrib /usr/local/solr/lib/
 
wget https://github.com/avalonmediasystem/puppet-solr/archive/master.zip
unzip master
mv puppet-solr-master/files/avalon /usr/local/solr
chown -R tomcat7:tomcat /usr/local/solr/
chown -R tomcat7:tomcat /usr/local/tomcat
service tomcat restart

If you have successfully installed Solr you should be able to access the dashboard page:

http://<server host name>:8983/solr

If you can't access the dashboard, check the tomcat logs in /usr/local/tomcat/logs/. Catalina.out and localhost.<date>.log have been the most helpful.

MySQL

Avalon uses MySQL for storing search queries, user data and roles, and as a back end for asynchronously sending requests to Matterhorn.

Install MySQL server

Code Block
yum install mysql-server service mysqld start

Create databases and users

Enter the mysql monitor:

Code Block
#mysql Welcome to the MySQL monitor. Commands end with ; or \g. ...etc... mysql>

Create a database for the Fedora Commons Repository software and add a user to it.

Code Block
create database fedora3; create user 'fcrepo'@'localhost' identified by 'fcrepo'; grant all privileges on fedora3.* to 'fcrepo'@'localhost'; flush privileges;

Create a database for the Avalon web application and add a user to it.

Code Block
create database rails; create user 'rails'@'localhost' identified by 'rails'; grant all privileges on rails.* to 'rails'@'localhost'; flush privileges;

Check your work and exit

Code Block

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| fedora3            |
| mysql              |
| rails              |
| test               |
+--------------------+
5 rows in set (0.00 sec)
mysql> select user, host from mysql.user;
+--------+--------------+
| user   | host         |
+--------+--------------+
| root   | 127.0.0.1    |
|        | 129.79.32.87 |
| root   | 129.79.32.87 |
|        | localhost    |
| fcrepo | localhost    |
| rails  | localhost    |
| root   | localhost    |
+--------+--------------+
7 rows in set (0.00 sec)
 
mysql> exit;
Bye

Please see documentation for your version of MySQL Server for detailed syntax (http://dev.mysql.com/doc/refman/5.1/en/create-database.html )

Fedora Commons repository

Download and run the fcrepo installer

Code Block
wget http://sourceforge.net/projects/fedora-commons/files/fedora/3.6.2/fcrepo-installer-3.6.2.jar/download java -jar fcrepo-installer-3.6.2.jar

Enter values:

Setting	Value
Installation type	custom
Fedora home directory	/usr/local/fedora
Fedora administrator password	fedoraPassword
Fedora server host	localhost
Fedora application server context	fedora
Authentication requirement for API-A	false
SSL availability	true
SSL required for API-A	false
SSL required for API-M	true
Servlet engine	existingTomcat
Tomcat home directory	/usr/local/tomcat
Tomcat HTTP port	8983
Tomcat shutdown port	8005
Tomcat Secure HTTP port	8443
Keystore file	default
Keystore password	changeit
Keystore type	JKS
Database	mysql
MySQL JDBC driver	included
Database username	fcrepo
Database password	fcrepo

accept remaining defaults then add permissions for Tomcat and restart Tomcat

Code Block
chown -R tomcat7:tomcat /usr/local/fedora service tomcat restart

Install Red5 media server

Red5 is an open source alternative to Adobe Flash Media Server.

Code Block

useradd red5

wget http://red5.org/downloads/red5/1_0_1/red5-1.0.1.tar.gz
tar xvf red5-1.0.1.tar.gz
mv red5-server-1.0 /usr/local/red5

wget https://raw.github.com/avalonmediasystem/avalon-installer/master/modules/red5/templates/red5_init_script.erb -O red5_init_script.sh
mv red5_init_script.sh /etc/rc.d/init.d/red5

chown -R red5:red5 /usr/local/red5
chmod +x /etc/rc.d/init.d/red5

Install FFmpeg

Install prerequisite packages using yum and the NUL-public repository:

Code Block

SDL-devel
a52dec-devel 
bzip2-devel 
faad2-devel 
freetype-devel 
frei0r-plugins-devel 
gsm-devel 
imlib2-devel 
lame-devel 
libdc1394-devel
libraw1394-devel 
librtmp-devel  
libtheora-devel 
libva-devel 
libfaac-devel 
libvdpau-devel 
libstdc++-devel 
libvorbis-devel 
libvpx-devel
mediainfo  
opencore-amr-devel 
opencv-devel 
openjpeg-devel 
openssl-devel 
schroedinger-devel 
speex-devel 
texi2html
vo-aacenc-devel 
x264-devel 
xvidcore-devel 
yasm  
zlib-devel

for the lazy:

Code Block

yum install SDL-devel a52dec-devel bzip2-devel faad2-devel freetype-devel frei0r-plugins-devel \
	gsm-devel imlib2-devel lame-devel libdc1394-devel libraw1394-devel librtmp-devel libtheora-devel \
	libva-devel libfaac-devel libvdpau-devel libstdc++-devel libvorbis-devel libvpx-devel \
	mediainfo opencore-amr-devel opencv-devel openjpeg-devel openssl-devel schroedinger-devel \
	speex-devel texi2html vo-aacenc-devel x264-devel xvidcore-devel yasm zlib-devel

Install rpmdev-setuptree

Code Block
yum install rpmdevtools

Install ffmpeg srpm

Log in as the user red5, because the following commands need to run under a user other than root.

Code Block
su - red5

Run the ffmpeg install.

Code Block

#rpmdev-setuptree
#rpm -ivh https://github.com/avalonmediasystem/avalon-installer/raw/master/files/ffmpeg/ffmpeg-1.2-59a.el6.src.rpm
Retrieving https://github.com/avalonmediasystem/avalon-installer/raw/master/files/ffmpeg/ffmpeg-1.2-59a.el6.src.rpm
   1:ffmpeg                 warning: user makerpm does not exist - using root
warning: group makerpm does not exist - using root
########################################### [100%]
warning: user makerpm does not exist - using root
warning: group makerpm does not exist - using root
Build ffmpeg binary as non-root and install as root
#rpmbuild -bb rpmbuild/SPECS/ffmpeg12.spec

Log back in as root and finish the install.

Code Block
su - root rpm -ivh /home/red5/rpmbuild/RPMS/x86_64/ffmpeg-*.rpm

Install httpd

Code Block
yum install httpd

Add avalon user

Code Block
useradd avalon mkdir /var/www/avalon chown -R avalon:avalon /var/www/avalon

Install Matterhorn

Create a user for Matterhorn and then install Matterhorn.

Code Block

useradd matterhorn
 
wget https://github.com/avalonmediasystem/avalon-felix/archive/1.4.x.tar.gz
tar xvf 1.4.x
mv avalon-felix-1.4.x /usr/local/matterhorn

wget https://raw.github.com/avalonmediasystem/config-files/master/matterhorn/matterhorn_init.sh
mv matterhorn_init.sh /etc/init.d/matterhorn

chmod +x /etc/init.d/matterhorn
chown -R matterhorn:matterhorn /usr/local/matterhorn

Create and configure streaming directories.

Code Block

mkdir -p /usr/local/red5/webapps/avalon/streams
mkdir /var/avalon
mkdir -p /var/www/avalon/public/streams

chown red5:avalon /usr/local/red5/webapps/avalon/streams
chmod 0775 /usr/local/red5/webapp/avalon/streams
ln -s /usr/local/red5/webapps/avalon/streams /var/avalon/rtmp_streams
 
mkdir /var/avalon/hls_streams
chown root:root /var/avalon/hls_streams/
ln -s /var/avalon/hls_streams/ /var/www/avalon/public/streams

chmod 0775 /var/avalon/hls_streams/

Download Matterhorn config and verify property values.

Code Block
wget https://raw.github.com/avalonmediasystem/config-files/master/matterhorn/config.properties vim config.properties

Change the server url port from 18080 to 8080 on line 19:

org.opencastproject.server.url=http://localhost:8080

And verify these values:

org.opencastproject.streaming.directory=/var/avalon/rtmp_streams
org.opencastproject.hls.directory=/var/avalon/hls_streams

Then move the config to the appropriate spot.

Code Block
mv config.properties /usr/local/matterhorn/etc/

Add matterhorn user to the avalon group.

Code Block
usermod -G avalon matterhorn

Install sqlite-devel

Code Block
yum install sqlite-devel

Change current user to avalon then install RVM and ruby 1.9.3-p448.

Code Block
su - avalon curl -L https://get.rvm.io \| bash -s stable --ruby=1.9.3

Source the RVM shell (as avalon user).

Code Block
source /home/avalon/.rvm/scripts/rvm

Install Passenger via Gem (as avalon user).

Code Block
gem install passenger

Check to make sure passenger installed in the expected location (as avalon user).

Code Block
$ passenger-config --root /home/avalon/.rvm/gems/ruby-1.9.3-p429/gems/passenger-3.0.19

Install Passenger apache module requirements (as root).

Code Block
su - root yum install curl-devel httpd httpd-devel apr-devel apr-util-devel

Build passenger for your version of Apache and Ruby (as avalon user).

Code Block
su - avalon passenger-install-apache2-module #copy the suggested Apache configuration file settings for later

Create an apache configuration file.

Code Block
su - root vi /etc/httpd/conf.d/passenger.conf

Example contents of /etc/httpd/conf.d/passenger.conf:

Code Block

LoadModule passenger_module /home/avalon/.rvm/gems/ruby-1.9.3-p448/gems/passenger-4.0.10/buildout/apache2/mod_passenger.so
<IfModule passenger_module>
  PassengerRoot /home/avalon/.rvm/gems/ruby-1.9.3-p448/gems/passenger-4.0.10
  PassengerDefaultRuby /home/avalon/.rvm/wrappers/ruby-1.9.3-p448/ruby
  PassengerMaxPoolSize 30
  PassengerPoolIdleTime 300
  PassengerMaxInstancesPerApp 0
  PassengerMinInstances 3
  PassengerSpawnMethod smart-lv2
</IfModule>

Apache security configuration

Code Block

wget https://raw.github.com/avalonmediasystem/config-files/master/sbin/avalon_auth -O /usr/local/sbin/avalon_auth
chmod +x /usr/local/sbin/avalon_auth
wget https://raw.github.com/avalonmediasystem/config-files/master/apache/10-mod_rewrite.conf -P /etc/httpd/conf.d/

Create a virtual host for avalon in /etc/httpd/conf.d/avalon.conf

Code Block
wget https://raw.github.com/avalonmediasystem/config-files/master/apache/20-avalon.conf -P /etc/httpd/conf.d/ vi /etc/httpd/conf.d/20-avalon.conf

Restart apache. With apache running, check passenger-status:

Code Block
[root@localhost ~]# /etc/init.d/httpd restart [root@localhost ~]# su - avalon [avalon@localhost ~]$ which passenger-status ~/.rvm/gems/ruby-1.9.3-p448/bin/passenger-status

Grab Avalon code from github

Code Block

cd ~
git clone git://github.com/avalonmediasystem/avalon.git
cd avalon
git checkout master #make sure you are in the master branch (should be by default)
rm .rvmrc
mv public/* /var/www/avalon/public/
mv  * /var/www/avalon/

Create /var/www/avalon/.ruby-version and add:

Code Block
1.9.3-p448

Create /var/www/avalon/config/setup_load_paths.rb and add:

Code Block

if ENV['MY_RUBY_HOME'] && ENV['MY_RUBY_HOME'].include?('rvm')
   begin
     gems_path = ENV['MY_RUBY_HOME'].split(/@/)[0].sub(/rubies/,'gems')
     ENV['GEM_PATH'] = "#{gems_path}:#{gems_path}@global"
     require 'rvm'
     RVM.use_from_path! File.dirname(File.dirname(__FILE__))
   rescue LoadError
     raise "RVM gem is currently unavailable."
   end
 end
 # If you're not using Bundler at all, remove lines bellow
 ENV['BUNDLE_GEMFILE'] = File.expand_path('../Gemfile', File.dirname(__FILE__))
 require 'bundler/setup'

Code Block
cd /var/www/avalon/config cp authentication.yml.example config/authentication.yml vi database.yml

Replace what's in database.yml with:

Code Block
development: adapter: mysql2 host: localhost database: rails username: rails password: rails pool: 5 timeout: 5000

Code Block
yum install mysql-devel su - avalon gem install activerecord-mysql2-adapter gem install mysql2 su - root vi /var/www/avalon/Gemfile

Add to the Gemfile:

Code Block
gem 'mysql2', '~>0.3.11'

Run the bundle install

Code Block
su - avalon cd /var/www/avalon gem update debugger-ruby_core_source bundle install

Edit /var/www/avalon/config/solr.yml

Code Block
development: url: http://localhost:8983/solr/

Edit /var/www/avalon/config/fedora.yml

Code Block
development: user: fedoraAdmin password: FedoraPassword url: http://127.0.0.1:8983/fedora

Create /var/www/avalon/config/matterhorn.yml

Code Block
development: url: http://matterhorn_system_account:CHANGE_ME@localhost:8080/

Create /var/www/avalon/config/avalon.yml

Code Block
development: dropbox: path: '/var/avalon/dropbox/' upload_uri: 'sftp://localhost/var/avalon/dropbox' username: 'test' password: 'test' notification_email_address: ''

Create the database using rake

Code Block
su - avalon cd /var/www/avalon rake db:create

If you get an error message saying that you can't connect to the database, take a look at this post and follow some of the troubleshooting steps:

http://stackoverflow.com/questions/5376427/cant-connect-to-local-mysql-server-through-socket-var-mysql-mysql-sock-38

Run the database migrations

Code Block
rake db:migrate

set rails environment to development

Visit port 80

Sign in

Create identity:

archivist1@example.com

password/password

This is a known identity with administrative privileges.

Start delayed_job:

in /var/www/avalon (as avalon user)

bundle exec rake delayed_job:start

Delayed Job handles background jobs. DelayedJob logs to log/delayed_job.log in the avalon directory.

Red5 Avalon Security Webapp

cd /usr/local/red5/webapps

wget https://github.com/avalonmediasystem/config-files/raw/master/red5/red5-avalon.tar.gz

tar xvzf red5-avalon.tar.gz

chown -R red5:red5 avalon/

edit /usr/local/red5/webapps/avalon/WEB-INF/red5-web.properties:

avalon.serverUrl=http://localhost/

Restart red5

sudo /sbin/service red5 restart

If Red5 is installed and running you should be able to access this page.

Dropbox

groupadd -r dropbox

useradd -r avalondrop

mkdir -p /var/avalon/dropbox

chown avalondrop:dropbox /var/avalon/dropbox

chmod 2775 /var/avalon/dropbox

Add this to /etc/ssh/sshd_config:

Code Block

# override default of no subsystems
Subsystem sftp internal-sftp
 
# Example of overriding settings on a per-user basis
#Match User anoncvs
# X11Forwarding no
# AllowTcpForwarding no
# ForceCommand cvs server
Match Group dropbox
ChrootDirectory /var/avalon
X11Forwarding no
AllowTcpForwarding no
ForceCommand internal-sftp

/sbin/service sshd restart

Using the System

You can find specific information about using the system in the Collection Manager's Guide. Upload items individually or via batch. Batch demo content is available for your convenience, which includes the media files and an excel document necessary to run a batch. You can submit a batch directly via SFTP using the avalondrop account you created above.

Known Issues - Release 1.0 - a list of bugs, workarounds, and cautions for using Release 1.0.

Space shortcuts

Page tree

Versions Compared

Old Version 61

New Version 62

Key

Install EPEL

Make sure a valid hostname is resolvable

Add the NUL repository
Anchor
NUL
NUL

Disable SELinux

Stop iptables
Anchor
iptables
iptables

Install Development Libraries

Install development packages for building Ruby

Tomcat

Add Tomcat manager user

Solr

Download the solr tarball and unpack it to /usr/local/solr/:

Place into /usr/local/tomcat/conf/Catalina/localhost/solr.xml:

Place into /usr/local/solr/solr.xml:

Create directories and move files around

MySQL

Install MySQL server

Create databases and users

Download and run the fcrepo installer

Install Red5 media server

Install FFmpeg

Install rpmdev-setuptree

Install ffmpeg srpm

Install httpd

Add avalon user

Install sqlite-devel

Grab Avalon code from github

Create the database using rake

Run the database migrations

Using the System

Space shortcuts

Page tree

Page History

Versions Compared

Old Version 61

New Version 62

Key

Install EPEL

Make sure a valid hostname is resolvable

Add the NUL repository AnchorNULNUL

Disable SELinux

Stop iptables Anchoriptablesiptables

Install Development Libraries

Install development packages for building Ruby

Tomcat

Add Tomcat manager user

Solr

Download the solr tarball and unpack it to /usr/local/solr/:

Place into /usr/local/tomcat/conf/Catalina/localhost/solr.xml:

Place into /usr/local/solr/solr.xml:

Create directories and move files around

MySQL

Install MySQL server

Create databases and users

Download and run the fcrepo installer

Install Red5 media server

Install FFmpeg

Install rpmdev-setuptree

Install ffmpeg srpm

Install httpd

Add avalon user

Install sqlite-devel

Grab Avalon code from github

Create the database using rake

Run the database migrations

Using the System

Add the NUL repository
Anchor
NUL
NUL

Stop iptables
Anchor
iptables
iptables