Recipe instructions for building your own Avalon setup from scratch on CentOS or Red Hat Enterprise Linux
Install EPEL
Code Block |
---|
rpm -ivh http://linux.mirrors.es.net/fedora-epel/6/i386/epel-release-6-8.noarch.rpm |
This package has libyaml-devel which is required by ruby and not provided by Redhat.
Make sure a valid hostname is resolvable
The default hostname is “avalon.dev”, so name the machine this and enter it into /etc/hosts
Code Block |
---|
% hostname avalon.dev % cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 avalon.dev |
Add the NUL repository AnchorNUL NUL
NUL | |
NUL |
Create the NUL repository config file:
Code Block |
---|
vi /etc/yum.repos.d/nul-public.repo |
Append the following code:
Code Block |
---|
[nul_public] name=NUL Library Public RHEL repository baseurl=http://yumrepo-public.library.northwestern.edu/x86_64 enabled=1 gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-nul cost=150 |
Install and place the NUL GPG key in the proper location:
Code Block |
---|
curl http://yumrepo-public.library.northwestern.edu/RPM-GPG-KEY-nul -o /etc/pki/rpm-gpg/RPM-GPG-KEY-nul |
Disable SELinux
Code Block |
---|
echo 0 > /selinux/enforce vi /etc/selinux/config #change the value of `SELINUX` from `enforcing` to `permissive` |
Stop iptables Anchoriptables iptables
iptables | |
iptables |
Code Block |
---|
service iptables stop |
Install Development Libraries
Code Block |
---|
yum groupinstall "Development Tools" |
Install development packages for building Ruby
Code Block |
---|
yum install readline-devel zlib-devel libyaml-devel libffi-devel openssl-devel libxml2-devel libxslt-devel |
Tomcat
Install Apache Tomcat with the nulrepo-public repository configured and enabled.
Code Block |
---|
yum install tomcat service tomcat start vi /usr/local/tomcat/conf/server.xml |
on line 70, change the Tomcat connector port from 8080 to 8983:
<Connector port="8983" protocol="HTTP/1.1"
Add Tomcat manager user
By default, no user has access to the Tomcat Manager App.
Code Block |
---|
vi /usr/local/tomcat/conf/tomcat-users.xml |
Define a role and a user in tomcat-users.xml with a manager-gui role. Below is a very basic example. Use more secure log in credentials than the example please.
Code Block |
---|
<tomcat-users> <role rolename="manager-gui"/> <user username="admin" password="tomcat" roles="manager-gui"/> </tomcat-users> |
Restart Tomcat
Code Block |
---|
service tomcat restart |
See if you can log in to the manager app: http://<server host name>:8983/manager/html
Try it out on your local machine and on another machine. If you can't reach the app from another machine, your iptables might need to be changed to allow access.
Solr
Avalon makes use of Solr through the Blacklight gem for faceting and relevance-based searching.
Download the solr tarball and unpack it to /usr/local/solr/:
from http://www.apache.org/dyn/closer.cgi/lucene/solr/4.2.0
Code Block |
---|
wget http://archive.apache.org/dist/lucene/solr/4.2.0/solr-4.2.0.tgz tar xvf solr-4.2.0.tgz mv solr-4.2.0 /usr/local/solr cd /usr/local/solr/dist/ cp solr-4.2.0.war /usr/local/tomcat/webapps/solr.war mv solr-4.2.0.war /usr/local/solr/solr.war |
Place into /usr/local/tomcat/conf/Catalina/localhost/solr.xml:
Code Block |
---|
<Context docBase="/usr/local/solr/solr.war" debug="0" crossContext="true"> <Environment name="solr/home" type="java.lang.String" value="/usr/local/solr" override="true"/> </Context> |
Place into /usr/local/solr/solr.xml:
Code Block |
---|
<?xml version="1.0" encoding="UTF-8" ?> <solr persistent="true"> <cores defaultCoreName="avalon" adminPath="/admin/cores" zkClientTimeout="${zkClientTimeout:15000}" hostPort="8983" hostContext="solr"> <core instanceDir="avalon/" name="avalon"/> </cores> </solr> |
Create directories and move files around
Code Block |
---|
mkdir /usr/local/solr/lib mv /usr/local/solr/dist/* /usr/local/solr/lib/ mv /usr/local/solr/contrib /usr/local/solr/lib/ wget https://github.com/avalonmediasystem/puppet-solr/archive/master.zip unzip master mv puppet-solr-master/files/avalon /usr/local/solr chown -R tomcat7:tomcat /usr/local/solr/ chown -R tomcat7:tomcat /usr/local/tomcat service tomcat restart |
If you have successfully installed Solr you should be able to access the dashboard page:
http://<server host name>:8983/solr
If you can't access the dashboard, check the tomcat logs in /usr/local/tomcat/logs/. Catalina.out and localhost.<date>.log have been the most helpful.
MySQL
Avalon uses MySQL for storing search queries, user data and roles, and as a back end for asynchronously sending requests to Matterhorn.
Install MySQL server
Code Block |
---|
yum install mysql-server service mysqld start |
Create databases and users
Enter the mysql monitor:
Code Block |
---|
#mysql Welcome to the MySQL monitor. Commands end with ; or \g. ...etc... mysql> |
Create a database for the Fedora Commons Repository software and add a user to it.
Code Block |
---|
create database fedora3; create user 'fcrepo'@'localhost' identified by 'fcrepo'; grant all privileges on fedora3.* to 'fcrepo'@'localhost'; flush privileges; |
Create a database for the Avalon web application and add a user to it.
Code Block |
---|
create database rails; create user 'rails'@'localhost' identified by 'rails'; grant all privileges on rails.* to 'rails'@'localhost'; flush privileges; |
Check your work and exit
Code Block |
---|
mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | fedora3 | | mysql | | rails | | test | +--------------------+ 5 rows in set (0.00 sec) mysql> select user, host from mysql.user; +--------+--------------+ | user | host | +--------+--------------+ | root | 127.0.0.1 | | | 129.79.32.87 | | root | 129.79.32.87 | | | localhost | | fcrepo | localhost | | rails | localhost | | root | localhost | +--------+--------------+ 7 rows in set (0.00 sec) mysql> exit; Bye |
Please see documentation for your version of MySQL Server for detailed syntax (http://dev.mysql.com/doc/refman/5.1/en/create-database.html )
Download and run the fcrepo installer
Code Block |
---|
wget http://sourceforge.net/projects/fedora-commons/files/fedora/3.6.2/fcrepo-installer-3.6.2.jar/download java -jar fcrepo-installer-3.6.2.jar |
Enter values:
Setting | Value |
---|---|
Installation type | custom |
Fedora home directory | /usr/local/fedora |
Fedora administrator password | fedoraPassword |
Fedora server host | localhost |
Fedora application server context | fedora |
Authentication requirement for API-A | false |
SSL availability | true |
SSL required for API-A | false |
SSL required for API-M | true |
Servlet engine | existingTomcat |
Tomcat home directory | /usr/local/tomcat |
Tomcat HTTP port | 8983 |
Tomcat shutdown port | 8005 |
Tomcat Secure HTTP port | 8443 |
Keystore file | default |
Keystore password | changeit |
Keystore type | JKS |
Database | mysql |
MySQL JDBC driver | included |
Database username | fcrepo |
Database password | fcrepo |
accept remaining defaults then add permissions for Tomcat and restart Tomcat
Code Block |
---|
chown -R tomcat7:tomcat /usr/local/fedora service tomcat restart |
Install Red5 media server
Red5 is an open source alternative to Adobe Flash Media Server.
Code Block |
---|
useradd red5 wget http://red5.org/downloads/red5/1_0_1/red5-1.0.1.tar.gz tar xvf red5-1.0.1.tar.gz mv red5-server-1.0 /usr/local/red5 wget https://raw.github.com/avalonmediasystem/avalon-installer/master/modules/red5/templates/red5_init_script.erb -O red5_init_script.sh mv red5_init_script.sh /etc/rc.d/init.d/red5 chown -R red5:red5 /usr/local/red5 chmod +x /etc/rc.d/init.d/red5 |
Install FFmpeg
Install prerequisite packages using yum and the NUL-public repository:
Code Block |
---|
SDL-devel a52dec-devel bzip2-devel faad2-devel freetype-devel frei0r-plugins-devel gsm-devel imlib2-devel lame-devel libdc1394-devel libraw1394-devel librtmp-devel libtheora-devel libva-devel libfaac-devel libvdpau-devel libstdc++-devel libvorbis-devel libvpx-devel mediainfo opencore-amr-devel opencv-devel openjpeg-devel openssl-devel schroedinger-devel speex-devel texi2html vo-aacenc-devel x264-devel xvidcore-devel yasm zlib-devel |
for the lazy:
Code Block |
---|
yum install SDL-devel a52dec-devel bzip2-devel faad2-devel freetype-devel frei0r-plugins-devel \ gsm-devel imlib2-devel lame-devel libdc1394-devel libraw1394-devel librtmp-devel libtheora-devel \ libva-devel libfaac-devel libvdpau-devel libstdc++-devel libvorbis-devel libvpx-devel \ mediainfo opencore-amr-devel opencv-devel openjpeg-devel openssl-devel schroedinger-devel \ speex-devel texi2html vo-aacenc-devel x264-devel xvidcore-devel yasm zlib-devel |
Install rpmdev-setuptree
Code Block |
---|
yum install rpmdevtools |
Install ffmpeg srpm
Log in as the user red5, because the following commands need to run under a user other than root.
Code Block |
---|
su - red5 |
Run the ffmpeg install.
Code Block |
---|
#rpmdev-setuptree #rpm -ivh https://github.com/avalonmediasystem/avalon-installer/raw/master/files/ffmpeg/ffmpeg-1.2-59a.el6.src.rpm Retrieving https://github.com/avalonmediasystem/avalon-installer/raw/master/files/ffmpeg/ffmpeg-1.2-59a.el6.src.rpm 1:ffmpeg warning: user makerpm does not exist - using root warning: group makerpm does not exist - using root ########################################### [100%] warning: user makerpm does not exist - using root warning: group makerpm does not exist - using root Build ffmpeg binary as non-root and install as root #rpmbuild -bb rpmbuild/SPECS/ffmpeg12.spec |
Log back in as root and finish the install.
Code Block |
---|
su - root rpm -ivh /home/red5/rpmbuild/RPMS/x86_64/ffmpeg-*.rpm |
Install httpd
Code Block |
---|
yum install httpd |
Add avalon user
Code Block |
---|
useradd avalon mkdir /var/www/avalon chown -R avalon:avalon /var/www/avalon |
Install Matterhorn
Create a user for Matterhorn and then install Matterhorn.
Code Block |
---|
useradd matterhorn wget https://github.com/avalonmediasystem/avalon-felix/archive/1.4.x.tar.gz tar xvf 1.4.x mv avalon-felix-1.4.x /usr/local/matterhorn wget https://raw.github.com/avalonmediasystem/config-files/master/matterhorn/matterhorn_init.sh mv matterhorn_init.sh /etc/init.d/matterhorn chmod +x /etc/init.d/matterhorn chown -R matterhorn:matterhorn /usr/local/matterhorn |
Create and configure streaming directories.
Code Block |
---|
mkdir -p /usr/local/red5/webapps/avalon/streams mkdir /var/avalon mkdir -p /var/www/avalon/public/streams chown red5:avalon /usr/local/red5/webapps/avalon/streams chmod 0775 /usr/local/red5/webapp/avalon/streams ln -s /usr/local/red5/webapps/avalon/streams /var/avalon/rtmp_streams mkdir /var/avalon/hls_streams chown root:root /var/avalon/hls_streams/ ln -s /var/avalon/hls_streams/ /var/www/avalon/public/streams chmod 0775 /var/avalon/hls_streams/ |
Download Matterhorn config and verify property values.
Code Block |
---|
wget https://raw.github.com/avalonmediasystem/config-files/master/matterhorn/config.properties vim config.properties |
Change the server url port from 18080 to 8080 on line 19:
org.opencastproject.server.url=http://localhost:8080
And verify these values:
org.opencastproject.streaming.directory=/var/avalon/rtmp_streams
org.opencastproject.hls.directory=/var/avalon/hls_streams
Then move the config to the appropriate spot.
Code Block |
---|
mv config.properties /usr/local/matterhorn/etc/ |
Add matterhorn user to the avalon group.
Code Block |
---|
usermod -G avalon matterhorn |
Install sqlite-devel
Code Block |
---|
yum install sqlite-devel |
Change current user to avalon then install RVM and ruby 1.9.3-p448.
Code Block |
---|
su - avalon curl -L https://get.rvm.io | bash -s stable --ruby=1.9.3 |
Source the RVM shell (as avalon user).
Code Block |
---|
source /home/avalon/.rvm/scripts/rvm |
Install Passenger via Gem (as avalon user).
Code Block |
---|
gem install passenger |
Check to make sure passenger installed in the expected location (as avalon user).
Code Block |
---|
$ passenger-config --root /home/avalon/.rvm/gems/ruby-1.9.3-p429/gems/passenger-3.0.19 |
Install Passenger apache module requirements (as root).
Code Block |
---|
su - root yum install curl-devel httpd httpd-devel apr-devel apr-util-devel |
Build passenger for your version of Apache and Ruby (as avalon user).
Code Block |
---|
su - avalon passenger-install-apache2-module #copy the suggested Apache configuration file settings for later |
Create an apache configuration file.
Code Block |
---|
su - root vi /etc/httpd/conf.d/passenger.conf |
Example contents of /etc/httpd/conf.d/passenger.conf:
Code Block |
---|
LoadModule passenger_module /home/avalon/.rvm/gems/ruby-1.9.3-p448/gems/passenger-4.0.10/buildout/apache2/mod_passenger.so <IfModule passenger_module> PassengerRoot /home/avalon/.rvm/gems/ruby-1.9.3-p448/gems/passenger-4.0.10 PassengerDefaultRuby /home/avalon/.rvm/wrappers/ruby-1.9.3-p448/ruby PassengerMaxPoolSize 30 PassengerPoolIdleTime 300 PassengerMaxInstancesPerApp 0 PassengerMinInstances 3 PassengerSpawnMethod smart-lv2 </IfModule> |
Apache security configuration
Code Block |
---|
wget https://raw.github.com/avalonmediasystem/config-files/master/sbin/avalon_auth -O /usr/local/sbin/avalon_auth chmod +x /usr/local/sbin/avalon_auth wget https://raw.github.com/avalonmediasystem/config-files/master/apache/10-mod_rewrite.conf -P /etc/httpd/conf.d/ |
Create a virtual host for avalon in /etc/httpd/conf.d/avalon.conf
Code Block |
---|
wget https://raw.github.com/avalonmediasystem/config-files/master/apache/20-avalon.conf -P /etc/httpd/conf.d/ vi /etc/httpd/conf.d/20-avalon.conf |
Restart apache. With apache running, check passenger-status:
Code Block |
---|
[root@localhost ~]# /etc/init.d/httpd restart [root@localhost ~]# su - avalon [avalon@localhost ~]$ which passenger-status ~/.rvm/gems/ruby-1.9.3-p448/bin/passenger-status |
Grab Avalon code from github
Code Block |
---|
cd ~ git clone git://github.com/avalonmediasystem/avalon.git cd avalon git checkout master #make sure you are in the master branch (should be by default) rm .rvmrc mv public/* /var/www/avalon/public/ mv * /var/www/avalon/ |
Create /var/www/avalon/.ruby-version and add:
Code Block |
---|
1.9.3-p448 |
Create /var/www/avalon/config/setup_load_paths.rb and add:
Code Block |
---|
if ENV['MY_RUBY_HOME'] && ENV['MY_RUBY_HOME'].include?('rvm') begin gems_path = ENV['MY_RUBY_HOME'].split(/@/)[0].sub(/rubies/,'gems') ENV['GEM_PATH'] = "#{gems_path}:#{gems_path}@global" require 'rvm' RVM.use_from_path! File.dirname(File.dirname(__FILE__)) rescue LoadError raise "RVM gem is currently unavailable." end end # If you're not using Bundler at all, remove lines bellow ENV['BUNDLE_GEMFILE'] = File.expand_path('../Gemfile', File.dirname(__FILE__)) require 'bundler/setup' |
Code Block |
---|
cd /var/www/avalon/config cp authentication.yml.example config/authentication.yml vi database.yml |
Replace what's in database.yml with:
Code Block |
---|
development: adapter: mysql2 host: localhost database: rails username: rails password: rails pool: 5 timeout: 5000 |
Code Block |
---|
yum install mysql-devel su - avalon gem install activerecord-mysql2-adapter gem install mysql2 su - root vi /var/www/avalon/Gemfile |
Add to the Gemfile:
Code Block |
---|
gem 'mysql2', '~>0.3.11' |
Run the bundle install
Code Block |
---|
su - avalon cd /var/www/avalon gem update debugger-ruby_core_source bundle install |
Edit /var/www/avalon/config/solr.yml
Code Block |
---|
development: url: http://localhost:8983/solr/ |
Edit /var/www/avalon/config/fedora.yml
Code Block |
---|
development: user: fedoraAdmin password: FedoraPassword url: http://127.0.0.1:8983/fedora |
Create /var/www/avalon/config/matterhorn.yml
Code Block |
---|
development: url: http://matterhorn_system_account:CHANGE_ME@localhost:8080/ |
Create /var/www/avalon/config/avalon.yml
Code Block |
---|
development: dropbox: path: '/var/avalon/dropbox/' upload_uri: 'sftp://localhost/var/avalon/dropbox' username: 'test' password: 'test' notification_email_address: '' |
Create the database using rake
Code Block |
---|
su - avalon cd /var/www/avalon rake db:create |
If you get an error message saying that you can't connect to the database, take a look at this post and follow some of the troubleshooting steps:
Run the database migrations
Code Block |
---|
rake db:migrate |
set rails environment to development
Visit port 80
Sign in
Create identity:
password/password
This is a known identity with administrative privileges.
Start delayed_job:
in /var/www/avalon (as avalon user)
bundle exec rake delayed_job:start
Delayed Job handles background jobs. DelayedJob logs to log/delayed_job.log in the avalon directory.
Red5 Avalon Security Webapp
cd /usr/local/red5/webapps
wget https://github.com/avalonmediasystem/config-files/raw/master/red5/red5-avalon.tar.gz
tar xvzf red5-avalon.tar.gz
chown -R red5:red5 avalon/
edit /usr/local/red5/webapps/avalon/WEB-INF/red5-web.properties:
avalon.serverUrl=http://localhost/
Restart red5
sudo /sbin/service red5 restart
If Red5 is installed and running you should be able to access this page.
Dropbox
groupadd -r dropbox
useradd -r avalondrop
mkdir -p /var/avalon/dropbox
chown avalondrop:dropbox /var/avalon/dropbox
chmod 2775 /var/avalon/dropbox
Add this to /etc/ssh/sshd_config:
Code Block |
---|
# override default of no subsystems Subsystem sftp internal-sftp # Example of overriding settings on a per-user basis #Match User anoncvs # X11Forwarding no # AllowTcpForwarding no # ForceCommand cvs server Match Group dropbox ChrootDirectory /var/avalon X11Forwarding no AllowTcpForwarding no ForceCommand internal-sftp |
/sbin/service sshd restart
Using the System
You can find specific information about using the system in the Collection Manager's Guide. Upload items individually or via batch. Batch demo content is available for your convenience, which includes the media files and an excel document necessary to run a batch. You can submit a batch directly via SFTP using the avalondrop account you created above.
Known Issues - Release 1.0 - a list of bugs, workarounds, and cautions for using Release 1.0.