Avalon Media System
Space shortcuts
Page tree

This site has moved to: https://samvera.atlassian.net/wiki/spaces/avalon/

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 56 Next »

Recipe instructions for building your own Avalon setup from scratch on CentOS or Red Hat Enterprise Linux

Install EPEL

rpm -ivh http://linux.mirrors.es.net/fedora-epel/6/i386/epel-release-6-8.noarch.rpm

This package has libyaml-devel which is required by ruby and not provided by Redhat.

Make sure a valid hostname is resolvable

The default hostname is “avalon.dev”, so name the machine this and enter it into /etc/hosts

% hostname
avalon.dev
% cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 avalon.dev

Add the NUL repository

Create the NUL repository config file:

vi /etc/yum.repos.d/nul-public.repo

Append the following code:

[nul_public]
name=NUL Library Public RHEL repository
baseurl=http://yumrepo-public.library.northwestern.edu/x86_64
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-nul
cost=150

Install and place the NUL GPG key in the proper location:

curl http://yumrepo-public.library.northwestern.edu/RPM-GPG-KEY-nul -o /etc/pki/rpm-gpg/RPM-GPG-KEY-nul

Disable SELinux

echo 0 > /selinux/enforce 
vi /etc/selinux/config #change the value of `SELINUX` from `enforcing` to `permissive`

Stop iptables  

service iptables stop

Install Development Libraries

yum groupinstall "Development Tools"

Install development packages for building Ruby:

yum install readline-devel zlib-devel libyaml-devel libffi-devel openssl-devel libxml2-devel libxslt-devel

Tomcat

Install Apache Tomcat with the nulrepo-public repository configured and enabled.

yum install tomcat
service tomcat start
vi /usr/local/tomcat/conf/server.xml

on line 70, change the Tomcat connector port from 8080 to 8983:

<Connector port="8983" protocol="HTTP/1.1"
Add admin user to Tomcat

By default, no user has access to the Tomcat admin page. 

vi /usr/local/tomcat/conf/tomcat-users.xml

Define a role and a user in tomcat-users.xml with a manager-gui role. Below is a very basic example. Use more secure log in credentials than the example please.

<tomcat-users>
  <role rolename="manager-gui"/>
  <user username="admin" password="tomcat" roles="manager-gui"/>
</tomcat-users>

Restart Tomcat

service tomcat restart

See if you can log in to the manager app: http://<server host name>:8983/manager/html

Try it out on your local machine and on another machine. If you can't reach the app from another machine, your iptables might need to be changed to allow access.

Solr

Avalon makes use of Solr through the Blacklight gem for faceting and relevance-based searching.

Download the solr tarball and unpack it to /usr/local/solr/:

from http://www.apache.org/dyn/closer.cgi/lucene/solr/4.2.0

wget http://archive.apache.org/dist/lucene/solr/4.2.0/solr-4.2.0.tgz
tar xvf solr-4.2.0.tgz
mv solr-4.2.0 /usr/local/solr
cd /usr/local/solr/dist/
cp solr-4.2.0.war /usr/local/tomcat/webapps/solr.war
mv solr-4.2.0.war /usr/local/solr/solr.war
Place into /usr/local/tomcat/conf/Catalina/localhost/solr.xml:
<Context docBase="/usr/local/solr/solr.war" debug="0" crossContext="true">
 <Environment name="solr/home" type="java.lang.String" value="/usr/local/solr" override="true"/>
</Context>
Place into /usr/local/solr/solr.xml:
<?xml version="1.0" encoding="UTF-8" ?>
  <solr persistent="true">
    <cores defaultCoreName="avalon" adminPath="/admin/cores" zkClientTimeout="${zkClientTimeout:15000}" hostPort="8983" hostContext="solr">
    <core instanceDir="avalon/" name="avalon"/>
  </cores>
</solr>
Create directories and move files around
mkdir /usr/local/solr/lib
mv /usr/local/solr/dist/* /usr/local/solr/lib/
mv /usr/local/solr/contrib /usr/local/solr/lib/
 
wget https://github.com/avalonmediasystem/puppet-solr/archive/master.zip
unzip master
mv puppet-solr-master/files/avalon /usr/local/solr
chown -R tomcat7:tomcat /usr/local/solr/
chown -R tomcat7:tomcat /usr/local/tomcat
service tomcat restart

If you have successfully installed Solr you should be able to access the dashboard page:

http://<server host name>:8983/solr

If you can't access the dashboard, check the tomcat logs in /usr/local/tomcat/logs/. Catalina.out and localhost.<date>.log have been the most helpful.

MySQL

Avalon uses MySQL for storing search queries, user data and roles, and as a backend for asynchronously sending requests to Matterhorn.

Install MySQL server 

yum install mysql-server
service mysqld start

 

 

and create 1 user with ALL privileges to 1 database for Fedora Commons Repository software, and 1 user with 1 database for the Avalon web application. 

 

mysql

create database fedora3;
create user 'fcrepo'@'localhost' identified by 'fcrepo';
grant all privileges on fedora3.* to 'fcrepo'@'localhost';
flush privileges; 


create database rails;
create user 'rails'@'localhost' identified by 'rails';
grant all privileges on rails.* to 'rails'@'localhost';
flush privileges;

 

 

 

Please see documentation for your version of MySQL Server for detailed syntax (http://dev.mysql.com/doc/refman/5.0/en/create-database.html)

 

 

Fedora Commons repository

wget http://sourceforge.net/projects/fedora-commons/files/fedora/3.6.2/fcrepo-installer-3.6.2.jar/download
java -jar fcrepo-installer-3.6.2.jar

Enter values:

Installation type: custom
Fedora home directory: /usr/local/fedora
fedoraAdmin password: fedoraPassword
server host: localhost
Application context: fedora
require authentication for api-a: false
ssl availability: true 
ssl required for api-a: false
ssl required for api-m: true
servlet engine: existingTomcat
tomcat home: /usr/local/tomcat
tomcat http port: 8983
tomcat shutdown port: 8005
tomcat secure port: 8443
keystore file: default
password for keystore: changeit
keystore type: jks
database: mysql
mysql jdbc driver: included
database username:fcrepo
database password: fcrepo
jdbc url: default jdbc:mysql://localhost/fedora3?useUnicode=true&amp;characterEncoding=UTF-8&amp;autoReconnect=true
jdbc driverclass:  default

 

accept remaining defaults

 

chown -R tomcat7:tomcat /usr/local/fedora

service tomcat restart

 

Red5 media server

Red5 is an open source alternative to Adobe Flash Media Server.

useradd red5

wget http://red5.org/downloads/red5/1_0_1/red5-1.0.1.tar.gz

tar xvf red5-1.0.1.tar.gz

mv red5-server-1.0 /usr/local/red5

wget https://raw.github.com/avalonmediasystem/avalon-installer/master/modules/red5/templates/red5_init_script.erb -O red5_init_script.sh

mv red5_init_script.sh /etc/rc.d/init.d/red5

chown -R red5:red5 /usr/local/red5

chmod +x /etc/rc.d/init.d/red5

ffmpeg

Install prerequisite packages using yum and the NUL-public repository:

 

SDL-devel 
a52dec-devel 
bzip2-devel 
faad2-devel 
freetype-devel 
frei0r-plugins-devel 
gsm-devel 
imlib2-devel 
lame-devel 
libdc1394-devel
libfaac
libfaac-devel 
libraw1394-devel 
librtmp-devel  
libstdc++-devel 
libvorbis-devel 
libtheora-devel 
libva-devel 
libvdpau-devel 
libvpx-devel
mediainfo  
opencore-amr-devel 
opencv-devel 
openjpeg-devel 
openssl-devel 
schroedinger-devel 
speex-devel 
texi2html
vo-aacenc-devel 
x264-devel 
xvidcore-devel 
yasm  
zlib-devel

 

Install rpmdev-setuptree

yum install rpmdevtools

As any non-root user, install ffmpeg srpm

su - red5
rpmdev-setuptree
rpm -ivh https://github.com/avalonmediasystem/avalon-installer/raw/master/files/ffmpeg/ffmpeg-1.2-59a.el6.src.rpm
Retrieving https://github.com/avalonmediasystem/avalon-installer/raw/master/files/ffmpeg/ffmpeg-1.2-59a.el6.src.rpm
   1:ffmpeg                 warning: user makerpm does not exist - using root
warning: group makerpm does not exist - using root
########################################### [100%]
warning: user makerpm does not exist - using root
warning: group makerpm does not exist - using root
Build ffmpeg binary as non-root and install as root
[as not root] rpmbuild -bb rpmbuild/SPECS/ffmpeg12.spec
[as root] rpm -ivh /home/red5/rpmbuild/RPMS/x86_64/ffmpeg-*.rpm

 

Create user for matterhorn

useradd matterhorn

Install httpd

yum install httpd

Create avalon user

[root@localhost avalon]# useradd avalon

mkdir /var/www/avalon

[root@localhost avalon]# chown -R avalon:avalon /var/www/avalon

[root@localhost avalon]# su - avalon


Matterhorn

wget https://github.com/avalonmediasystem/avalon-felix/archive/1.4.x.tar.gz

tar xvf 1.4.x

mv avalon-felix-1.4.x /usr/local/matterhorn

wget https://raw.github.com/avalonmediasystem/config-files/master/matterhorn/matterhorn_init.sh

mv matterhorn_init.sh /etc/init.d/matterhorn

chmod +x /etc/init.d/matterhorn

chown -R matterhorn:matterhorn /usr/local/matterhorn

 

Install config.properties in /usr/local/matterhorn/etc: 

wget https://raw.github.com/avalonmediasystem/config-files/master/matterhorn/config.properties

mv config.properties /usr/local/matterhorn/etc/

edit config.properties:

org.opencastproject.server.url=http://localhost:8080

Matterhorn for streaming files

[security.pp ~line 37]

mkdir /usr/local/red5/webapps/avalon/streams

chown red5:avalon 

chmod 0775 /usr/local/red5/webapp/avalon/streams

ln -s /usr/local/red5/webapps/avalon/streams /var/avalon/rtmp_streams

 

 mkdir /var/avalon/hls_streams

chown root:root /var/avalon/hls_streams/

ln -s /var/avalon/hls_streams/ /var/www/avalon/public/streams

chmod 0775 /var/avalon/hls_streams/

 

add matterhorn user into the avalon group

 

org.opencastproject.streaming.directory=/var/avalon/rtmp_streams

org.opencastproject.hls.directory=/var/avalon/hls_streams

 

Install sqlite-devel

yum install sqlite-devel

As Avalon user, Install RVM and ruby 1.9.3-p429

# curl -L https://get.rvm.io | bash -s stable --ruby=1.9.3

 

As Avalon user, Source the RVM shell

# source /home/avalon/.rvm/scripts/rvm

As Avalon user, Install Passenger via Gem

gem install passenger

As Avalon user, Check to make sure passenger installed in the expected location

 

$ passenger-config --root

/home/avalon/.rvm/gems/ruby-1.9.3-p429/gems/passenger-3.0.19

 

Install Passenger apache module requirements (as root)

yum install curl-devel httpd httpd-devel apr-devel apr-util-devel

 

As Avalon user, build passenger for your version of Apache and Ruby.

[avalon@localhost ~]$ passenger-install-apache2-module

 

create an appropriate apache configuration file passenger.conf with appropriate settings.  Example contents of /etc/httpd/conf.d/passenger.conf

LoadModule passenger_module /home/avalon/.rvm/gems/ruby-1.9.3-p429/gems/passenger-3.0.19/ext/apache2/mod_passenger.so
<IfModule passenger_module>
   PassengerRoot /home/avalon/.rvm/gems/ruby-1.9.3-p429/gems/passenger-3.0.19
   PassengerRuby /home/avalon/.rvm/wrappers/ruby-1.9.3-p429/ruby
  PassengerMaxPoolSize 30
  PassengerPoolIdleTime 300
  PassengerMaxInstancesPerApp 0


  PassengerMinInstances 3
  PassengerSpawnMethod smart-lv2


</IfModule>

Apache security configuration

Retrieve https://raw.github.com/avalonmediasystem/config-files/master/sbin/avalon_auth and place into /usr/local/sbin

chmod +x /usr/local/sbin/avalon_auth

 

Retrieve https://raw.github.com/avalonmediasystem/config-files/master/apache/10-mod_rewrite.conf

and place into /etc/httpd/conf.d/

Create a virtual host for avalon in /etc/httpd/conf.d/avalon.conf

Retrieve https://raw.github.com/avalonmediasystem/config-files/master/apache/20-avalon.conf and place into /etc/httpd/conf.d/ Edit to set RailsEnv to development:

  RailsEnv Development

 

Restart apache.  With apache running, check passenger-status:

[avalon@localhost ~]$ which passenger-status

~/.rvm/gems/ruby-1.9.3-p429/bin/passenger-status

As Root

mkdir /var/www/avalon

chown avalon:avalon /var/www/avalon

As avalon user:

cd /var/www/avalon

git clone git://github.com/avalonmediasystem/avalon.git

git checkout master

rm  /var/www/avalon/.rvmrc

 

Add to /var/www/avalon/config/setup_load_paths.rb

if ENV['MY_RUBY_HOME'] && ENV['MY_RUBY_HOME'].include?('rvm')
   begin
     gems_path = ENV['MY_RUBY_HOME'].split(/@/)[0].sub(/rubies/,'gems')
     ENV['GEM_PATH'] = "#{gems_path}:#{gems_path}@global"
     require 'rvm'
     RVM.use_from_path! File.dirname(File.dirname(__FILE__))
   rescue LoadError
     raise "RVM gem is currently unavailable."
   end
 end
 # If you're not using Bundler at all, remove lines bellow
 ENV['BUNDLE_GEMFILE'] = File.expand_path('../Gemfile', File.dirname(__FILE__))
 require 'bundler/setup'

edit /var/www/avalon/.ruby-version and enter 1.9.3-p429

cp config/authentication.yml.example config/authentication.yml

edit config/database.yml

development:
  adapter: mysql2
  database: rails
  username: rails
  password: rails
  pool: 5
  timeout: 5000

yum install mysql-devel

gem install activerecord-mysql2-adapter

gem install mysql2

 

edit Gemfile.local:

gem 'mysql2', '~>0.3.11'

Run bundle install

edit solr.yml

  development:
    url: http://localhost:8983/solr/

edit fedora.yml

  development:
    user: fedoraAdmin
    password: FedoraPassword
    url: http://127.0.0.1:8983/fedora

 

create config/matterhorn.yml

development:
  url: http://matterhorn_system_account:CHANGE_ME@localhost:8080/

create config/avalon.yml

development:
  dropbox:
    path: '/var/avalon/dropbox/'
    upload_uri: 'sftp://localhost/var/avalon/dropbox'
    username: 'test'
    password: 'test'
    notification_email_address: ''

 

rake db:create

This command creates the database.

rake db:migrate

This command runs the database migrations.

set rails environment to development

Visit port 80

Sign in

Create identity:

archivist1@example.com

password/password

This is a known identity with administrative privileges.

 

Start delayed_job:

in /var/www/avalon (as avalon user)

bundle exec rake delayed_job:start

Delayed Job handles background jobs.  DelayedJob logs to log/delayed_job.log in the avalon directory. 

Red5 Avalon Security Webapp

cd /usr/local/red5/webapps

wget https://github.com/avalonmediasystem/config-files/raw/master/red5/red5-avalon.tar.gz

tar xvzf red5-avalon.tar.gz

chown -R red5:red5 avalon/

edit /usr/local/red5/webapps/avalon/WEB-INF/red5-web.properties:

avalon.serverUrl=http://localhost/

Restart red5

sudo /sbin/service red5 restart

If Red5 is installed and running you should be able to access this page.

 

Dropbox

groupadd -r dropbox

useradd -r avalondrop

mkdir -p /var/avalon/dropbox

chown avalondrop:dropbox /var/avalon/dropbox

chmod 2775 /var/avalon/dropbox

Add this to /etc/ssh/sshd_config:

# override default of no subsystems
Subsystem sftp internal-sftp
 
# Example of overriding settings on a per-user basis
#Match User anoncvs
# X11Forwarding no
# AllowTcpForwarding no
# ForceCommand cvs server
Match Group dropbox
ChrootDirectory /var/avalon
X11Forwarding no
AllowTcpForwarding no
ForceCommand internal-sftp

/sbin/service sshd restart

Using the System

You can find specific information about using the system in the Collection Manager's Guide. Upload items individually or via batch. Batch demo content is available for your convenience, which includes the media files and an excel document necessary to run a batch.  You can submit a batch directly via SFTP using the avalondrop account you created above.

Known Issues - Release 1.0 - a list of bugs, workarounds, and cautions for using Release 1.0.

 

 

 

  • No labels