This documentation is for Release 3.0.0. For the Release 1 version of this page, see v.81. For the Release 2 version of this page, see v.87. |
Recipe instructions for building your own Avalon setup from scratch on CentOS or Red Hat Enterprise Linux.
rpm -ivh http://linux.mirrors.es.net/fedora-epel/6/i386/epel-release-6-8.noarch.rpm |
This package has libyaml-devel which is required by ruby and not provided by Redhat.
The default hostname is “avalon.dev”, so name the machine this and enter it into /etc/hosts
# hostname avalon.dev # cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 avalon.dev |
The Avalon Media System requires several ports to be open to client browsers.
Here are the port settings that will need to be configured:
| Port | Purpose | External? |
|---|---|---|
| 80 | HTTP (Avalon) | Yes |
| 1935 | RTMP (red5) | Yes |
| 5080 | HTTP (red5) | No |
| 8983 | HTTP (Fedora/Solr) | No |
| 18080 | HTTP (Matterhorn) | Yes |
The preferred method is to create a shell script that will do the work for you. Here is an example script that you should look through and customize as needed: avalon-iptables-config.sh
If you're connected over ssh, it might kick you off. |
Save your script to /etc/sysconfig/avalon-iptables-config.sh, make it executable and run it.
chmod +x /etc/sysconfig/avalon-iptables-config.sh /etc/sysconfig/avalon-iptables-config.sh |
If you run into connection issues you can disable the iptables, by running "service iptables stop". This will completely drop your firewall. When finished troubleshooting run "service iptables start".
echo 0 > /selinux/enforce vim /etc/selinux/config #change the value of `SELINUX` from `enforcing` to `permissive` |
Create the NUL repository config file:
vim /etc/yum.repos.d/nul-public.repo |
Append the following code:
[nul_public] name=NUL Library Public RHEL repository baseurl=http://yumrepo-public.library.northwestern.edu/x86_64 enabled=1 gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-nul cost=150 |
Install and place the NUL GPG key in the proper location:
curl http://yumrepo-public.library.northwestern.edu/RPM-GPG-KEY-nul -o /etc/pki/rpm-gpg/RPM-GPG-KEY-nul |
Install development libraries and packages for building Ruby
yum groupinstall "Development Tools" yum install readline-devel zlib-devel libyaml-devel libffi-devel openssl-devel libxml2-devel libxslt-devel |
Install Apache Tomcat with the nulrepo-public repository configured and enabled.
yum install tomcat service tomcat start vim /usr/local/tomcat/conf/server.xml |
on line 70, change the Tomcat connector port from 8080 to 8983:
<Connector port="8983" protocol="HTTP/1.1" |
By default, no user has access to the Tomcat Manager App.
vim /usr/local/tomcat/conf/tomcat-users.xml |
Define a user in tomcat-users.xml with access to the manager-gui role. Below is a very basic example.
<tomcat-users> <role rolename="manager-gui"/> <user username="admin" password="tomcat" roles="manager-gui"/> </tomcat-users> |
Restart Tomcat
service tomcat restart |
See if you can log in to the manager app at http://<server host name>:8983/manager/html
Try it out on your local machine and on another machine. If you can't reach the app from another machine, your iptables might need to be changed to allow access.
Avalon makes use of Solr through the Blacklight gem for faceting and relevance-based searching.
Download Solr from http://www.apache.org/dyn/closer.cgi/lucene/solr/4.2.0
wget http://archive.apache.org/dist/lucene/solr/4.2.0/solr-4.2.0.tgz tar xvf solr-4.2.0.tgz mv solr-4.2.0 /usr/local/solr cd /usr/local/solr/dist/ cp solr-4.2.0.war /usr/local/tomcat/webapps/solr.war mv solr-4.2.0.war /usr/local/solr/solr.war |
Edit /usr/local/tomcat/conf/Catalina/localhost/solr.xml and add
<Context docBase="/usr/local/solr/solr.war" debug="0" crossContext="true"> <Environment name="solr/home" type="java.lang.String" value="/usr/local/solr" override="true"/> </Context> |
Edit /usr/local/solr/solr.xml and insert
<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="true">
<cores defaultCoreName="avalon" adminPath="/admin/cores" zkClientTimeout="${zkClientTimeout:15000}" hostPort="8983" hostContext="solr">
<core instanceDir="avalon/" name="avalon"/>
</cores>
</solr> |
mkdir /usr/local/solr/lib mv /usr/local/solr/dist/* /usr/local/solr/lib/ mv /usr/local/solr/contrib /usr/local/solr/lib/ wget https://github.com/avalonmediasystem/puppet-solr/archive/master.zip unzip master mv puppet-solr-master/files/avalon /usr/local/solr chown -R tomcat7:tomcat /usr/local/solr/ chown -R tomcat7:tomcat /usr/local/tomcat service tomcat restart |
If you have successfully installed Solr you should be able to access the dashboard page at http://<server host name>:8983/solr
If you can't access the dashboard, check the tomcat logs in /usr/local/tomcat/logs/. Catalina.out and localhost.<date>.log usually provide the best information.
Avalon uses MySQL for storing search queries, user data and roles, and as a back end for asynchronously sending requests to Matterhorn.
yum install mysql-server service mysqld start |
Enter the mysql monitor
#mysql Welcome to the MySQL monitor. Commands end with ; or \g. ...etc... mysql> |
Create a database for the Fedora Commons Repository software and add a user to it
create database fedora3; create user 'fcrepo'@'localhost' identified by 'fcrepo'; grant all privileges on fedora3.* to 'fcrepo'@'localhost'; flush privileges; |
Create a database for the Avalon web application and add a user to it
create database rails; create user 'rails'@'localhost' identified by 'rails'; grant all privileges on rails.* to 'rails'@'localhost'; flush privileges; |
Check your work and exit
mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | fedora3 | | mysql | | rails | | test | +--------------------+ 5 rows in set (0.00 sec) mysql> select user, host from mysql.user; +--------+--------------+ | user | host | +--------+--------------+ | root | 127.0.0.1 | | | 129.79.32.87 | | root | 129.79.32.87 | | | localhost | | fcrepo | localhost | | rails | localhost | | root | localhost | +--------+--------------+ 7 rows in set (0.00 sec) mysql> exit; Bye |
See documentation for your version of MySQL Server for detailed syntax (http://dev.mysql.com/doc/refman/5.1/en/create-database.html )
wget http://sourceforge.net/projects/fedora-commons/files/fedora/3.6.2/fcrepo-installer-3.6.2.jar/download java -jar fcrepo-installer-3.6.2.jar |
Use the following values in place of defaults
| Setting | Value |
|---|---|
Installation type | custom |
Fedora home directory | /usr/local/fedora |
Fedora administrator password | fedoraPassword |
Fedora server host | localhost |
Fedora application server context | fedora |
Authentication requirement for API-A | false |
SSL availability | true |
SSL required for API-A | false |
SSL required for API-M | true |
Servlet engine | existingTomcat |
Tomcat home directory | /usr/local/tomcat |
Tomcat HTTP port | 8983 |
Tomcat shutdown port | 8005 |
Tomcat Secure HTTP port | 8443 |
Keystore file | default |
Keystore password | changeit |
Keystore type | JKS |
Database | mysql |
MySQL JDBC driver | included |
Database username | fcrepo |
Database password | fcrepo |
Accept remaining defaults then add permissions for Tomcat and restart Tomcat
chown -R tomcat7:tomcat /usr/local/fedora service tomcat restart |
Red5 is an open source alternative to Adobe Media Server. If using the Adobe Media Server you can skip to the next step.
Create a red5 user.
useradd red5 |
Download and install Red5.
wget http://red5.org/downloads/red5/1_0_1/red5-1.0.1.tar.gz tar xvf red5-1.0.1.tar.gz mv red5-server-1.0 /usr/local/red5 |
Download the init script and add it to the init.d directory.
wget https://raw.github.com/avalonmediasystem/avalon-installer/master/modules/red5/templates/red5_init_script.erb -O red5_init_script.sh mv red5_init_script.sh /etc/rc.d/init.d/red5 |
Give Red5 permissions to the red5 directory and the init script.
chown -R red5:red5 /usr/local/red5 chmod +x /etc/rc.d/init.d/red5 |
Install prerequisite packages using yum and the NUL-public repository
yum install SDL-devel a52dec-devel bzip2-devel faad2-devel freetype-devel frei0r-plugins-devel \ gsm-devel imlib2-devel lame-devel libdc1394-devel libraw1394-devel librtmp-devel libtheora-devel \ libva-devel libfaac-devel libvdpau-devel libstdc++-devel libvorbis-devel libvpx-devel \ mediainfo opencore-amr-devel opencv-devel openjpeg-devel openssl-devel schroedinger-devel \ speex-devel texi2html vo-aacenc-devel x264-devel xvidcore-devel yasm zlib-devel |
Install rpmdev-setuptree
yum install rpmdevtools |
The following commands need to run under a user other than root. Change to the red5 user to continue.
su - red5 |
Run the ffmpeg installer
#rpmdev-setuptree #rpm -ivh https://github.com/avalonmediasystem/avalon-installer/raw/master/files/ffmpeg/ffmpeg-1.2-59a.el6.src.rpm Retrieving https://github.com/avalonmediasystem/avalon-installer/raw/master/files/ffmpeg/ffmpeg-1.2-59a.el6.src.rpm 1:ffmpeg warning: user makerpm does not exist - using root warning: group makerpm does not exist - using root ########################################### [100%] warning: user makerpm does not exist - using root warning: group makerpm does not exist - using root Build ffmpeg binary as non-root and install as root #rpmbuild -bb rpmbuild/SPECS/ffmpeg12.spec |
Log back in as root and finish the install.
su - root rpm -ivh /home/red5/rpmbuild/RPMS/x86_64/ffmpeg-*.rpm |
Install and start the httpd service.
yum install httpd service httpd start |
Create a user for Matterhorn and then install Matterhorn
useradd matterhorn wget https://github.com/avalonmediasystem/avalon-felix/archive/release/3.0.0.tar.gz mv avalon-felix-1.4.x /usr/local/matterhorn wget https://raw.github.com/avalonmediasystem/config-files/master/matterhorn/matterhorn_init.sh mv matterhorn_init.sh /etc/init.d/matterhorn chmod +x /etc/init.d/matterhorn chown -R matterhorn:matterhorn /usr/local/matterhorn |
Add avalon user and create avalon directory.
useradd avalon mkdir /var/www/avalon chown -R avalon:avalon /var/www/avalon |
Create and configure streaming directories.
mkdir -p /usr/local/red5/webapps/avalon/streams mkdir /var/avalon mkdir -p /var/www/avalon/public/streams chown red5:avalon /usr/local/red5/webapps/avalon/streams chmod 0775 /usr/local/red5/webapps/avalon/streams ln -s /usr/local/red5/webapps/avalon/streams /var/avalon/rtmp_streams mkdir /var/avalon/hls_streams chown root:root /var/avalon/hls_streams/ ln -s /var/avalon/hls_streams/ /var/www/avalon/public/streams chmod 0775 /var/avalon/hls_streams/ |
Download Matterhorn config and verify property values.
wget https://raw.github.com/avalonmediasystem/config-files/master/matterhorn/config.properties vim config.properties |
And verify the configuration of the streaming directoriesorg.opencastproject.streaming.directory=/var/avalon/rtmp_streams org.opencastproject.hls.directory=/var/avalon/hls_streams |
Move the config to the appropriate spot
mv config.properties /usr/local/matterhorn/etc/ |
Add matterhorn user to the avalon group.
usermod -G avalon matterhorn |
Change current user to avalon then install RVM and ruby 1.9.3.
su - avalon curl -L https://get.rvm.io | bash -s stable --ruby=1.9.3 |
Source the RVM shell (as avalon user)
source /home/avalon/.rvm/scripts/rvm |
Install Passenger via Gem (as avalon user)
gem install passenger |
Check to make sure passenger installed in the expected location (as avalon user)
$ passenger-config --root /home/avalon/.rvm/gems/ruby-1.9.3-p545/gems/passenger-4.0.40 |
Install Passenger apache module requirements (as root)
su - root yum install curl-devel httpd httpd-devel apr-devel apr-util-devel |
Build passenger for your version of Apache and Ruby (as avalon user)
su - avalon passenger-install-apache2-module #copy the suggested Apache configuration file settings for later |
Create an apache configuration file (as root)
su - root vim /etc/httpd/conf.d/passenger.conf |
Example contents of /etc/httpd/conf.d/passenger.conf, which may need to be changed based off of the current version of ruby and passenger:
LoadModule passenger_module /home/avalon/.rvm/gems/ruby-1.9.3-p545/gems/passenger-4.0.40/buildout/apache2/mod_passenger.so <IfModule passenger_module> PassengerRoot /home/avalon/.rvm/gems/ruby-1.9.3-p545/gems/passenger-4.0.40 PassengerDefaultRuby /home/avalon/.rvm/wrappers/ruby-1.9.3-p545/ruby PassengerMaxPoolSize 30 PassengerPoolIdleTime 300 PassengerMaxInstancesPerApp 0 PassengerMinInstances 3 PassengerSpawnMethod smart-lv2 </IfModule> |
Apache security configuration
wget https://raw.github.com/avalonmediasystem/config-files/master/sbin/avalon_auth -O /usr/local/sbin/avalon_auth chmod +x /usr/local/sbin/avalon_auth wget https://raw.github.com/avalonmediasystem/config-files/master/apache/10-mod_rewrite.conf -P /etc/httpd/conf.d/ |
Create a virtual host for avalon in /etc/httpd/conf.d/avalon.conf
wget https://raw.github.com/avalonmediasystem/config-files/master/apache/20-avalon.conf -P /etc/httpd/conf.d/ vim /etc/httpd/conf.d/20-avalon.conf |
Add this line to 20-avalon under the VirtualHost tag:
RailsEnv development |
Restart apache. With apache running, check passenger-status
/etc/init.d/httpd restart su - avalon which passenger-status #> ~/.rvm/gems/ruby-1.9.3-p448/bin/passenger-status |
cd ~ git clone git://github.com/avalonmediasystem/avalon.git cd avalon git checkout master #make sure you are in the master branch (should be by default) mv public/* /var/www/avalon/public/ mv * /var/www/avalon/ |
If using vim with default settings and pasting the the code below, it will automatically comment out the last line. To prevent that, enable paste using the command :set paste and then use just ctrl+shift+v instead of going into insert mode. |
Create /var/www/avalon/config/setup_load_paths.rb and add:
if ENV['MY_RUBY_HOME'] && ENV['MY_RUBY_HOME'].include?('rvm')
begin
gems_path = ENV['MY_RUBY_HOME'].split(/@/)[0].sub(/rubies/,'gems')
ENV['GEM_PATH'] = "#{gems_path}:#{gems_path}@global"
require 'rvm'
RVM.use_from_path! File.dirname(File.dirname(__FILE__))
rescue LoadError
raise "RVM gem is currently unavailable."
end
end
# If you're not using Bundler at all, remove lines bellow
ENV['BUNDLE_GEMFILE'] = File.expand_path('../Gemfile', File.dirname(__FILE__))
require 'bundler/setup' |
cd /var/www/avalon/config cp authentication.yml.example config/authentication.yml vim database.yml |
vim database.yml |
Replace database.yml with the correct values for your development environment
development: adapter: mysql2 host: localhost database: rails username: rails password: rails pool: 5 timeout: 5000 |
Install the mysql2 adapter
yum install mysql-devel su - avalon gem install activerecord-mysql2-adapter gem install mysql2 su - root vim /var/www/avalon/Gemfile |
Add this line to the Gemfile
gem 'mysql2', '~>0.3.11' |
If you are using mysql instead of sqllite (as is done throughout this document), then comment out the following lines in the Gemfile
# gem 'activerecord-jdbcsqlite3-adapter' # gem 'jdbc-sqlite3' # gem 'sqlite3' |
su - avalon cd /var/www/avalon gem update debugger-ruby_core_source bundle install |
Edit /var/www/avalon/config/solr.yml
development:
url: http://localhost:8983/solr/ |
Edit /var/www/avalon/config/fedora.yml
development:
user: fedoraAdmin
password: fedoraPassword
url: http://127.0.0.1:8983/fedora |
Create /var/www/avalon/config/matterhorn.yml
development: url: http://matterhorn_system_account:CHANGE_ME@localhost:18080/ |
Create /var/www/avalon/config/avalon.yml and base it off of /var/www/avalon/config/avalon.yml.example. Consult the documentation to customize this file for your installation.
development:
dropbox:
path: '/var/avalon/dropbox/'
upload_uri: 'sftp://localhost/var/avalon/dropbox'
username: 'test'
password: 'test'
notification_email_address: '' |
su - avalon cd /var/www/avalon rake db:create |
If you get an error message saying that you can't connect to the database, take a look at this post and follow some of the troubleshooting steps.
rake db:migrate |
Set rails environment to development, if it has not defaulted to this. On the first line of /var/www/avalon/config/environment.rb make sure it says 'development'
ENV['RAILS_ENV'] ||= 'development' |
You should be able to visit the webpage with just the hostname (ie http://localhost)
Click on "Sign in" in the upper right corner of the website main page. Set up a default identity with the following properties.
archivist1@example.com password/password |
This is a known identity with administrative privileges.
As avalon run
cd /var/www/avalon/ bundle exec rake delayed_job:start |
Delayed Job handles background jobs. DelayedJob logs to log/delayed_job.log in the avalon directory.
cd /usr/local/red5/webapps wget https://github.com/avalonmediasystem/config-files/raw/master/red5/red5-avalon.tar.gz tar xvzf red5-avalon.tar.gz chown -R red5:red5 avalon/ |
Edit /usr/local/red5/webapps/avalon/WEB-INF/red5-web.properties
avalon.serverUrl=http://localhost/ |
Restart Red5
service red5 restart |
If Red5 is installed and running you should be able to access http://avalon.dev:5080/
groupadd -r dropbox useradd -r avalondrop usermod -G dropbox avalon mkdir -p /var/avalon/dropbox chown avalondrop:dropbox /var/avalon/dropbox chmod 2775 /var/avalon/dropbox |
Edit /etc/ssh/sshd_config
# override default of no subsystems Subsystem sftp internal-sftp # Example of overriding settings on a per-user basis #Match User anoncvs # X11Forwarding no # AllowTcpForwarding no # ForceCommand cvs server Match Group dropbox ChrootDirectory /var/avalon X11Forwarding no AllowTcpForwarding no ForceCommand internal-sftp |
Restart SSH
service sshd restart |
You can find specific information about using the system in the Collection Manager's Guide. Upload items individually or via batch. Batch demo content is available for your convenience, which includes the media files and an excel document necessary to run a batch. You can submit a batch directly via SFTP using the avalondrop account you created above.
Known Issues - a list of bugs, workarounds, and cautions.
Before you restart your Avalon server, you'll want to make sure all of the services necessary to run Avalon will start automatically after the restart. Run these commands once and you should be set:
chkconfig --level 345 tomcat on chkconfig --level 345 mysqld on chkconfig --level 345 sshd on chkconfig --level 345 red5 on chkconfig --level 345 httpd on chkconfig --level 345 matterhorn on |