Recipe instructions for building your own Avalon setup from scratch on CentOS or Red Hat Enterprise Linux
Install EPEL
rpm -ivh http://linux.mirrors.es.net/fedora-epel/6/i386/epel-release-6-8.noarch.rpm
This package has libyaml-devel which is required by ruby and not provided by Redhat.
Make sure a valid hostname is resolvable
The default hostname is “avalon.dev”, so name the machine this and enter it into /etc/hosts
% hostname avalon.dev % cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 avalon.dev
Add the NUL repository
Create the NUL repository config file:
vi /etc/yum.repos.d/nul-public.repo
Append the following code:
[nul_public] name=NUL Library Public RHEL repository baseurl=http://yumrepo-public.library.northwestern.edu/x86_64 enabled=1 gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-nul cost=150
Install and place the NUL GPG key in the proper location:
curl http://yumrepo-public.library.northwestern.edu/RPM-GPG-KEY-nul -o /etc/pki/rpm-gpg/RPM-GPG-KEY-nul
Disable SELinux
echo 0 > /selinux/enforce vi /etc/selinux/config #change the value of `SELINUX` from `enforcing` to `permissive`
Stop iptables
service iptables stop
Install Development Libraries
yum groupinstall "Development Tools"
Install development packages for building Ruby:
yum install readline-devel zlib-devel libyaml-devel libffi-devel openssl-devel libxml2-devel libxslt-devel
Tomcat
Install Apache Tomcat with the nulrepo-public repository configured and enabled.
yum install tomcat service tomcat start vi /usr/local/tomcat/conf/server.xml
on line 70, change the Tomcat connector port from 8080 to 8983:
<Connector port="8983" protocol="HTTP/1.1"
Add admin user to Tomcat
By default, no user has access to the Tomcat admin page.
vi /usr/local/tomcat/conf/tomcat-users.xml
Define a role and a user in tomcat-users.xml with a manager-gui role. Below is a very basic example. Use more secure log in credentials than the example please.
<tomcat-users> <role rolename="manager-gui"/> <user username="admin" password="tomcat" roles="manager-gui"/> </tomcat-users>
Restart Tomcat
service tomcat restart
See if you can log in to the manager app: http://<server host name>:8983/manager/html
Try it out on your local machine and on another machine. If you can't reach the app from another machine, your iptables might need to be changed to allow access.
Solr
Avalon makes use of Solr through the Blacklight gem for faceting and relevance-based searching.
Download the solr tarball and unpack it to /usr/local/solr/:
from http://www.apache.org/dyn/closer.cgi/lucene/solr/4.2.0
wget http://archive.apache.org/dist/lucene/solr/4.2.0/solr-4.2.0.tgz tar xvf solr-4.2.0.tgz mv solr-4.2.0 /usr/local/solr cd /usr/local/solr/dist/ cp solr-4.2.0.war /usr/local/tomcat/webapps/solr.war mv solr-4.2.0.war /usr/local/solr/solr.war
Place into /usr/local/tomcat/conf/Catalina/localhost/solr.xml:
<Context docBase="/usr/local/solr/solr.war" debug="0" crossContext="true"> <Environment name="solr/home" type="java.lang.String" value="/usr/local/solr" override="true"/> </Context>
Place into /usr/local/solr/solr.xml:
<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="true">
<cores defaultCoreName="avalon" adminPath="/admin/cores" zkClientTimeout="${zkClientTimeout:15000}" hostPort="8983" hostContext="solr">
<core instanceDir="avalon/" name="avalon"/>
</cores>
</solr>
Create directories and move files around
mkdir /usr/local/solr/lib mv /usr/local/solr/dist/* /usr/local/solr/lib/ mv /usr/local/solr/contrib /usr/local/solr/lib/ wget https://github.com/avalonmediasystem/puppet-solr/archive/master.zip unzip master mv puppet-solr-master/files/avalon /usr/local/solr chown -R tomcat7:tomcat /usr/local/solr/ chown -R tomcat7:tomcat /usr/local/tomcat service tomcat restart
If you have successfully installed Solr you should be able to access the dashboard page:
http://<server host name>:8983/solr
If you can't access the dashboard, check the tomcat logs in /usr/local/tomcat/logs/. Catalina.out and localhost.<date>.log have been the most helpful.
MySQL
Avalon uses MySQL for storing search queries, user data and roles, and as a backend for asynchronously sending requests to Matterhorn.
Install MySQL server
yum install mysql-server service mysqld start
and create 1 user with ALL privileges to 1 database for Fedora Commons Repository software, and 1 user with 1 database for the Avalon web application.
mysql
create database fedora3; create user 'fcrepo'@'localhost' identified by 'fcrepo'; grant all privileges on fedora3.* to 'fcrepo'@'localhost'; flush privileges; create database rails; create user 'rails'@'localhost' identified by 'rails'; grant all privileges on rails.* to 'rails'@'localhost'; flush privileges;
Please see documentation for your version of MySQL Server for detailed syntax (http://dev.mysql.com/doc/refman/5.0/en/create-database.html)
Fedora Commons repository
wget http://sourceforge.net/projects/fedora-commons/files/fedora/3.6.2/fcrepo-installer-3.6.2.jar/download
java -jar fcrepo-installer-3.6.2.jar
Enter values:
Installation type: custom
Fedora home directory: /usr/local/fedora
fedoraAdmin password: fedoraPassword
server host: localhost
Application context: fedora
require authentication for api-a: false
ssl availability: true
ssl required for api-a: false
ssl required for api-m: true
servlet engine: existingTomcat
tomcat home: /usr/local/tomcat
tomcat http port: 8983
tomcat shutdown port: 8005
tomcat secure port: 8443
keystore file: default
password for keystore: changeit
keystore type: jks
database: mysql
mysql jdbc driver: included
database username:fcrepo
database password: fcrepo
jdbc url: default jdbc:mysql://localhost/fedora3?useUnicode=true&characterEncoding=UTF-8&autoReconnect=true
jdbc driverclass: default
accept remaining defaults
chown -R tomcat7:tomcat /usr/local/fedora
service tomcat restart
Red5 media server
Red5 is an open source alternative to Adobe Flash Media Server.
useradd red5
wget http://red5.org/downloads/red5/1_0_1/red5-1.0.1.tar.gz
tar xvf red5-1.0.1.tar.gz
mv red5-server-1.0 /usr/local/red5
wget https://raw.github.com/avalonmediasystem/avalon-installer/master/modules/red5/templates/red5_init_script.erb -O red5_init_script.sh
mv red5_init_script.sh /etc/rc.d/init.d/red5
chown -R red5:red5 /usr/local/red5
chmod +x /etc/rc.d/init.d/red5
ffmpeg
Install prerequisite packages using yum and the NUL-public repository:
SDL-devel a52dec-devel bzip2-devel faad2-devel freetype-devel frei0r-plugins-devel gsm-devel imlib2-devel lame-devel libdc1394-devel libfaac libfaac-devel libraw1394-devel librtmp-devel libstdc++-devel libvorbis-devel libtheora-devel libva-devel libvdpau-devel libvpx-devel mediainfo opencore-amr-devel opencv-devel openjpeg-devel openssl-devel schroedinger-devel speex-devel texi2html vo-aacenc-devel x264-devel xvidcore-devel yasm zlib-devel
Install rpmdev-setuptree
yum install rpmdevtools
As any non-root user, install ffmpeg srpm
su - red5 rpmdev-setuptree rpm -ivh https://github.com/avalonmediasystem/avalon-installer/raw/master/files/ffmpeg/ffmpeg-1.2-59a.el6.src.rpm Retrieving https://github.com/avalonmediasystem/avalon-installer/raw/master/files/ffmpeg/ffmpeg-1.2-59a.el6.src.rpm 1:ffmpeg warning: user makerpm does not exist - using root warning: group makerpm does not exist - using root ########################################### [100%] warning: user makerpm does not exist - using root warning: group makerpm does not exist - using root Build ffmpeg binary as non-root and install as root [as not root] rpmbuild -bb rpmbuild/SPECS/ffmpeg12.spec [as root] rpm -ivh /home/red5/rpmbuild/RPMS/x86_64/ffmpeg-*.rpm
Create user for matterhorn
useradd matterhorn
Install httpd
yum install httpd
Create avalon user
[root@localhost avalon]# useradd avalon
mkdir /var/www/avalon
[root@localhost avalon]# chown -R avalon:avalon /var/www/avalon
[root@localhost avalon]# su - avalon
Matterhorn
wget https://github.com/avalonmediasystem/avalon-felix/archive/1.4.x.tar.gz
tar xvf 1.4.x
mv avalon-felix-1.4.x /usr/local/matterhorn
wget https://raw.github.com/avalonmediasystem/config-files/master/matterhorn/matterhorn_init.sh
mv matterhorn_init.sh /etc/init.d/matterhorn
chmod +x /etc/init.d/matterhorn
chown -R matterhorn:matterhorn /usr/local/matterhorn
Install config.properties in /usr/local/matterhorn/etc:
wget https://raw.github.com/avalonmediasystem/config-files/master/matterhorn/config.properties
mv config.properties /usr/local/matterhorn/etc/
edit config.properties:
org.opencastproject.server.url=http://localhost:8080
Matterhorn for streaming files
[security.pp ~line 37]
mkdir /usr/local/red5/webapps/avalon/streams
chown red5:avalon
chmod 0775 /usr/local/red5/webapp/avalon/streams
ln -s /usr/local/red5/webapps/avalon/streams /var/avalon/rtmp_streams
mkdir /var/avalon/hls_streams
chown root:root /var/avalon/hls_streams/
ln -s /var/avalon/hls_streams/ /var/www/avalon/public/streams
chmod 0775 /var/avalon/hls_streams/
add matterhorn user into the avalon group
org.opencastproject.streaming.directory=/var/avalon/rtmp_streams
org.opencastproject.hls.directory=/var/avalon/hls_streams
Install sqlite-devel
yum install sqlite-devel
As Avalon user, Install RVM and ruby 1.9.3-p429
# curl -L https://get.rvm.io | bash -s stable --ruby=1.9.3
As Avalon user, Source the RVM shell
# source /home/avalon/.rvm/scripts/rvm
As Avalon user, Install Passenger via Gem
gem install passenger
As Avalon user, Check to make sure passenger installed in the expected location
$ passenger-config --root
/home/avalon/.rvm/gems/ruby-1.9.3-p429/gems/passenger-3.0.19
Install Passenger apache module requirements (as root)
yum install curl-devel httpd httpd-devel apr-devel apr-util-devel
As Avalon user, build passenger for your version of Apache and Ruby.
[avalon@localhost ~]$ passenger-install-apache2-module
create an appropriate apache configuration file passenger.conf with appropriate settings. Example contents of /etc/httpd/conf.d/passenger.conf
LoadModule passenger_module /home/avalon/.rvm/gems/ruby-1.9.3-p429/gems/passenger-3.0.19/ext/apache2/mod_passenger.so <IfModule passenger_module> PassengerRoot /home/avalon/.rvm/gems/ruby-1.9.3-p429/gems/passenger-3.0.19 PassengerRuby /home/avalon/.rvm/wrappers/ruby-1.9.3-p429/ruby PassengerMaxPoolSize 30 PassengerPoolIdleTime 300 PassengerMaxInstancesPerApp 0 PassengerMinInstances 3 PassengerSpawnMethod smart-lv2 </IfModule>
Apache security configuration
Retrieve https://raw.github.com/avalonmediasystem/config-files/master/sbin/avalon_auth and place into /usr/local/sbin
chmod +x /usr/local/sbin/avalon_auth
Retrieve https://raw.github.com/avalonmediasystem/config-files/master/apache/10-mod_rewrite.conf
and place into /etc/httpd/conf.d/
Create a virtual host for avalon in /etc/httpd/conf.d/avalon.conf
Retrieve https://raw.github.com/avalonmediasystem/config-files/master/apache/20-avalon.conf and place into /etc/httpd/conf.d/ Edit to set RailsEnv to development:
RailsEnv Development
Restart apache. With apache running, check passenger-status:
[avalon@localhost ~]$ which passenger-status
~/.rvm/gems/ruby-1.9.3-p429/bin/passenger-status
As Root
mkdir /var/www/avalon
chown avalon:avalon /var/www/avalon
As avalon user:
cd /var/www/avalon
git clone git://github.com/avalonmediasystem/avalon.git
git checkout master
rm /var/www/avalon/.rvmrc
Add to /var/www/avalon/config/setup_load_paths.rb
if ENV['MY_RUBY_HOME'] && ENV['MY_RUBY_HOME'].include?('rvm')
begin
gems_path = ENV['MY_RUBY_HOME'].split(/@/)[0].sub(/rubies/,'gems')
ENV['GEM_PATH'] = "#{gems_path}:#{gems_path}@global"
require 'rvm'
RVM.use_from_path! File.dirname(File.dirname(__FILE__))
rescue LoadError
raise "RVM gem is currently unavailable."
end
end
# If you're not using Bundler at all, remove lines bellow
ENV['BUNDLE_GEMFILE'] = File.expand_path('../Gemfile', File.dirname(__FILE__))
require 'bundler/setup'edit /var/www/avalon/.ruby-version and enter 1.9.3-p429
cp config/authentication.yml.example config/authentication.yml
edit config/database.yml
development:
adapter: mysql2
database: rails
username: rails
password: rails
pool: 5
timeout: 5000
yum install mysql-devel
gem install activerecord-mysql2-adapter
gem install mysql2
edit Gemfile.local:
gem 'mysql2', '~>0.3.11'
Run bundle install
edit solr.yml
development:
url: http://localhost:8983/solr/
edit fedora.yml
development:
user: fedoraAdmin
password: FedoraPassword
url: http://127.0.0.1:8983/fedora
create config/matterhorn.yml
development:
url: http://matterhorn_system_account:CHANGE_ME@localhost:8080/
create config/avalon.yml
development:
dropbox:
path: '/var/avalon/dropbox/'
upload_uri: 'sftp://localhost/var/avalon/dropbox'
username: 'test'
password: 'test'
notification_email_address: ''
rake db:create
This command creates the database.
rake db:migrate
This command runs the database migrations.
set rails environment to development
Visit port 80
Sign in
Create identity:
password/password
This is a known identity with administrative privileges.
Start delayed_job:
in /var/www/avalon (as avalon user)
bundle exec rake delayed_job:start
Delayed Job handles background jobs. DelayedJob logs to log/delayed_job.log in the avalon directory.
Red5 Avalon Security Webapp
cd /usr/local/red5/webapps
wget https://github.com/avalonmediasystem/config-files/raw/master/red5/red5-avalon.tar.gz
tar xvzf red5-avalon.tar.gz
chown -R red5:red5 avalon/
edit /usr/local/red5/webapps/avalon/WEB-INF/red5-web.properties:
avalon.serverUrl=http://localhost/
Restart red5
sudo /sbin/service red5 restart
If Red5 is installed and running you should be able to access this page.
Dropbox
groupadd -r dropbox
useradd -r avalondrop
mkdir -p /var/avalon/dropbox
chown avalondrop:dropbox /var/avalon/dropbox
chmod 2775 /var/avalon/dropbox
Add this to /etc/ssh/sshd_config:
# override default of no subsystems Subsystem sftp internal-sftp # Example of overriding settings on a per-user basis #Match User anoncvs # X11Forwarding no # AllowTcpForwarding no # ForceCommand cvs server Match Group dropbox ChrootDirectory /var/avalon X11Forwarding no AllowTcpForwarding no ForceCommand internal-sftp
/sbin/service sshd restart
Using the System
You can find specific information about using the system in the Collection Manager's Guide. Upload items individually or via batch. Batch demo content is available for your convenience, which includes the media files and an excel document necessary to run a batch. You can submit a batch directly via SFTP using the avalondrop account you created above.
Known Issues - Release 1.0 - a list of bugs, workarounds, and cautions for using Release 1.0.