Recipe instructions for building your own Avalon setup from scratch on CentOS or Red Hat Enterprise Linux

Install EPEL

rpm -ivh http://linux.mirrors.es.net/fedora-epel/6/i386/epel-release-6-8.noarch.rpm

This package has libyaml-devel which is required by ruby and not provided by Redhat.

Make sure a valid hostname is resolvable

The default hostname is “avalon.dev”, so name the machine this and enter it into /etc/hosts

# hostname
avalon.dev
# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 avalon.dev

Add the NUL repository

Create the NUL repository config file:

#vi /etc/yum.repos.d/nul-public.repo

Append the following code:

[nul_public]
name=NUL Library Public RHEL repository
baseurl=http://yumrepo-public.library.northwestern.edu/x86_64
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-nul
cost=150

Install and place the NUL GPG key in the proper location:

curl http://yumrepo-public.library.northwestern.edu/RPM-GPG-KEY-nul -o /etc/pki/rpm-gpg/RPM-GPG-KEY-nul

Disable SELinux

echo 0 > /selinux/enforce 
vi /etc/selinux/config #change the value of `SELINUX` from `enforcing` to `permissive`

Stop iptables

service iptables stop

Install Development Libraries

yum groupinstall "Development Tools"

Install development packages for building Ruby:

yum install readline-devel zlib-devel libyaml-devel libffi-devel openssl-devel libxml2-devel libxslt-devel

Tomcat

Install Apache Tomcat with the nulrepo-public repository configured and enabled.

yum install tomcat
service tomcat start
vi /usr/local/tomcat/conf/server.xml

on line 70, change the Tomcat connector port to 8983:

<Connector port="8983" protocol="HTTP/1.1"

Restart Tomcat

service tomcat restart

Solr

Avalon makes use of Solr through the Blacklight gem for faceting and relevance-based searching.

Download the solr tarball and unpack it to /usr/local/solr/:

from http://www.apache.org/dyn/closer.cgi/lucene/solr/4.4.0

wget http://apache.mirrors.hoobly.com/lucene/solr/4.4.0/solr-4.4.0.tgz
tar xvf solr-4.4.0.tgz
mv solr-4.4.0 /usr/local/solr
cd /usr/local/solr/example/lib/ext/
cp * /usr/local/tomcat/lib
cd /usr/local/solr/dist/
cp solr-4.4.0.war /usr/local/tomcat/webapps/solr.war
mv solr-4.4.0.war /usr/local/solr/solr.war

Place into /usr/local/tomcat/conf/Catalina/localhost/solr.xml:

<Context docBase="/usr/local/solr/solr.war" debug="0" crossContext="true">
 <Environment name="solr/home" type="java.lang.String" value="/usr/local/solr" override="true"/>
</Context>

Place into /usr/local/solr/solr.xml:

<?xml version="1.0" encoding="UTF-8" ?>
  <solr persistent="true">
    <cores defaultCoreName="avalon" adminPath="/admin/cores" zkClientTimeout="${zkClientTimeout:15000}" hostPort="8983" hostContext="solr">
    <core instanceDir="avalon/" name="avalon"/>
  </cores>
</solr>

Create directories and move files around

mkdir /usr/local/solr/lib
mv /usr/local/solr/dist/* /usr/local/solr/lib/
mv /usr/local/solr/contrib /usr/local/solr/lib/
 
wget https://github.com/avalonmediasystem/puppet-solr/archive/master.zip
unzip master
mv puppet-solr-master/files/avalon /usr/local/solr
chown -R tomcat7:tomcat /usr/local/solr/
chown -R tomcat7:tomcat /usr/local/tomcat
service tomcat restart

If you have successfully installed Solr you should be able to access the dashboard page.

MySQL

Avalon uses MySQL for storing search queries, user data and roles, and as a backend for asynchronously sending requests to Matterhorn.

Install MySQL server 

yum install mysql-server
service mysqld start

and create 1 user with ALL privileges to 1 database for Fedora Commons Repository software, and 1 user with 1 database for the Avalon web application. 

mysql

These instructions provide a recipe for building your own all-in-one Avalon system from scratch on CentOS or Red Hat Enterprise Linux version 7.x.  Please note that while an all-in-one installation as outlined here is certainly suitable for testing and demos, a single, all-in-one, server may not be suitable for production environments.

Ready the Installation Environment

Storage requirement

Avalon and components need about 20GB of disk space to install.

Open ports requirement

The Avalon Media System requires several ports to be open to client browsers.

Here are the port settings that will need to be configured:

The preferred method is to create a shell script that will do the work for you. Here is an example script that you should look through and customize as needed: avalon-iptables-config.sh

Save your script to /etc/sysconfig/avalon-iptables-config.sh, make it executable and run it.

chmod +x /etc/sysconfig/avalon-iptables-config.sh
/etc/sysconfig/avalon-iptables-config.sh

If you run into connection issues you can disable the iptables, by running "service iptables stop". This will completely drop your firewall. When finished troubleshooting run "service iptables start".

Disable SELinux

vim /etc/selinux/config #change the value of `SELINUX` from `enforcing` to `permissive`

You may have to disable SELinux completely if there's Passenger installation problem

vim /etc/selinux/config #change the value of `SELINUX` to `disabled`

Reboot to apply change

shutdown -r now

Install EPEL

This package has libyaml-devel which is required by ruby and not provided by Redhat.

yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

Install development libraries and packages for building Ruby

yum groupinstall "Development Tools"
yum install readline-devel zlib-devel libyaml-devel libffi-devel openssl-devel libxml2-devel libxslt-devel cmake

Install Java 8

yum install java-1.8.0-openjdk

Main Components

MariaDB

Avalon uses MariaDB for storing search queries, user data and roles, and as a back end our encoding dashboard. 

Install MariaDB server 

yum install mariadb-server
systemctl start mariadb

Fedora Commons Repository

Tomcat

Fedora runs as a webapp in Tomcat

Install Apache Tomcat 

yum install tomcat
vim /etc/tomcat/server.xml #line 71, change the Tomcat connector port from 8080 to 8984

Add Tomcat manager user

By default, no user has access to the Tomcat Manager App. Define a user in /etc/tomcat/tomcat-users.xml with access to the manager-gui role. Below is a very basic example.

<tomcat-users>
  <role rolename="manager-gui"/>
  <user username="admin" password="<insert strong password here>" roles="manager-gui"/>
</tomcat-users>

Create Fedora user and database

Enter the mariadb client

mysql
mariadb> create database fcrepo CHARACTER SET utf8 COLLATE utf8_general_ci;
mariadb> create user 'fcrepo'@'localhost' identified by '<fcrepo_password>';
mariadb> grant all privileges on fcrepo.* to 'fcrepo'@'localhost';

mariadb> create database rails CHARACTER SET utf8 COLLATE utf8_general_ci;
mariadb> create user 'rails'@'localhost' identified by 'fcrepo<rails_pasword>';
mariadb> grant all privileges on fedora3rails.* to to 'fcreporails'@'localhost';

mariadb> flush privileges;

Check your work and exit

mariadb> show databases;


create database rails;
create user 'rails'@'localhost' identified by 'rails';
grant all privileges on rails.* to 'rails'@'localhost';
flush privileges;

Please see documentation for your version of MySQL Server for detailed syntax (http://dev.mysql.com/doc/refman/5.0/en/create-database.html)

Fedora Commons repository

wget http://sourceforge.net/projects/fedora-commons/files/fedora/3.6.2/fcrepo-installer-3.6.2.jar/download

java -jar fcrepo-installer-3.6.2.jar

Enter values:

Installation type: custom
Fedora home directory: /usr/local/fedora
fedoraAdmin password: fedoraPassword
server host: localhost
Application context: fedora
require authentication for api-a: false
ssl availability: true 
ssl required for api-a: false
ssl required for api-m: true
servlet engine: existingTomcat
tomcat home: /usr/local/tomcat
tomcat http port: 8983
tomcat shutdown port: 8005
tomcat secure port: 8443
keystore file: default
password for keystore: changeit
keystore type: jks
database: mysql
mysql jdbc driver: included
database username:fcrepo
database password: fcrepo
jdbc url: default jdbc:mysql://localhost/fedora3?useUnicode=true&characterEncoding=UTF-8&autoReconnect=true
jdbc driverclass:  default

accept remaining defaults

chown -R tomcat7:tomcat /usr/local/fedora

service tomcat restart

Red5 media server

Red5 is an open source alternative to Adobe Flash Media Server.

useradd red5

wget http://red5.org/downloads/red5/1_0_1/red5-1.0.1.tar.gz

tar xvf red5-1.0.1.tar.gz

mv red5-server-1.0 /usr/local/red5

wget https://raw.github.com/avalonmediasystem/avalon-installer/master/modules/red5/templates/red5_init_script.erb -O red5_init_script.sh

mv red5_init_script.sh /etc/rc.d/init.d/red5

chown -R red5:red5 /usr/local/red5

chmod +x /etc/rc.d/init.d/red5

ffmpeg

Install prerequisite packages using yum and the NUL-public repository:

SDL-devel 
a52dec-devel 
bzip2-devel 
faad2-devel 
freetype-devel 
frei0r-plugins-devel 
gsm-devel 
imlib2-devel 
lame-devel 
libdc1394-devel
libfaac
libfaac-devel 
libraw1394-devel 
librtmp-devel  
libstdc++-devel 
libvorbis-devel 
libtheora-devel 
libva-devel 
libvdpau-devel 
libvpx-devel
mediainfo  
opencore-amr-devel 
opencv-devel 
openjpeg-devel 
openssl-devel 
schroedinger-devel 
speex-devel 
texi2html
vo-aacenc-devel 
x264-devel 
xvidcore-devel 
yasm  
zlib-devel 

Install rpmdev-setuptree

yum install rpmdevtools

As any non-root user, install ffmpeg srpm

su - red5
rpmdev-setuptree
rpm -ivh https://github.com/avalonmediasystem/avalon-installer/raw/master/files/ffmpeg/ffmpeg-1.2-59a.el6.src.rpm
Retrieving https://github.com/avalonmediasystem/avalon-installer/raw/master/files/ffmpeg/ffmpeg-1.2-59a.el6.src.rpm
   1:ffmpeg                 warning: user makerpm does not exist - using root
warning: group makerpm does not exist - using root
########################################### [100%]
warning: user makerpm does not exist - using root
warning: group makerpm does not exist - using root
Build ffmpeg binary as non-root and install as root
[as not root] rpmbuild -bb rpmbuild/SPECS/ffmpeg12.spec
[as root] rpm -ivh /home/red5/rpmbuild/RPMS/x86_64/ffmpeg-*.rpm

Create user for matterhorn

useradd matterhorn

Install httpd

yum install httpd

Create avalon user

[root@localhost avalon]# useradd avalon

mkdir /var/www/avalon

[root@localhost avalon]# chown -R avalon:avalon /var/www/avalon

[root@localhost avalon]# su - avalon

Matterhorn

wget https://github.com/avalonmediasystem/avalon-felix/archive/1.4.x.tar.gz

tar xvf 1.4.x

mv avalon-felix-1.4.x /usr/local/matterhorn

wget https://raw.github.com/avalonmediasystem/config-files/master/matterhorn/matterhorn_init.sh

mv matterhorn_init.sh /etc/init.d/matterhorn

chmod +x /etc/init.d/matterhorn

chown -R matterhorn:matterhorn /usr/local/matterhorn

Install config.properties in /usr/local/matterhorn/etc: 

wget https://raw.github.com/avalonmediasystem/config-files/master/matterhorn/config.properties

mv config.properties /usr/local/matterhorn/etc/

edit config.properties:

org.opencastproject.server.url=http://localhost:8080

Matterhorn for streaming files

[security.pp ~line 37]

mkdir /usr/local/red5/webapps/avalon/streams

chown red5:avalon 

chmod 0775 /usr/local/red5/webapp/avalon/streams

ln -s /usr/local/red5/webapps/avalon/streams /var/avalon/rtmp_streams

 mkdir /var/avalon/hls_streams

chown root:root /var/avalon/hls_streams/

ln -s /var/avalon/hls_streams/ /var/www/avalon/public/streams

chmod 0775 /var/avalon/hls_streams/

add matterhorn user into the avalon group

org.opencastproject.streaming.directory=/var/avalon/rtmp_streams

org.opencastproject.hls.directory=/var/avalon/hls_streams

Install sqlite-devel

yum install sqlite-devel

As Avalon user, Install RVM and ruby 1.9.3-p429

# curl -L https://get.rvm.io | bash -s stable --ruby=1.9.3

As Avalon user, Source the RVM shell

# source /home/avalon/.rvm/scripts/rvm

As Avalon user, Install Passenger via Gem

gem install passenger

As Avalon user, Check to make sure passenger installed in the expected location

$ passenger-config --root

/home/avalon/.rvm/gems/ruby-1.9.3-p429/gems/passenger-3.0.19

Install Passenger apache module requirements (as root)

yum install curl-devel httpd httpd-devel apr-devel apr-util-devel

As Avalon user, build passenger for your version of Apache and Ruby.

[avalon@localhost ~]$ passenger-install-apache2-module

create an appropriate apache configuration file passenger.conf with appropriate settings.  Example contents of /etc/httpd/conf.d/passenger.conf

LoadModule passenger_module /home/avalon/.rvm/gems/ruby-1.9.3-p429/gems/passenger-3.0.19/ext/apache2/mod_passenger.so
<IfModule passenger_module>
   PassengerRoot /home/avalon/.rvm/gems/ruby-1.9.3-p429/gems/passenger-3.0.19
   PassengerRuby /home/avalon/.rvm/wrappers/ruby-1.9.3-p429/ruby
  PassengerMaxPoolSize 30
  PassengerPoolIdleTime 300
  PassengerMaxInstancesPerApp 0


  PassengerMinInstances 3
  PassengerSpawnMethod smart-lv2


</IfModule>

Apache security configuration

Retrieve https://raw.github.com/avalonmediasystem/config-files/master/sbin/avalon_auth and place into /usr/local/sbin

chmod +x /usr/local/sbin/avalon_auth

Retrieve https://raw.github.com/avalonmediasystem/config-files/master/apache/10-mod_rewrite.conf

and place into /etc/httpd/conf.d/

Create a virtual host for avalon in /etc/httpd/conf.d/avalon.conf

...

+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| fcrepo             |
| performance_schema |
| test               |
+--------------------+
5 rows in set (0.00 sec)
mariadb> exit;
Bye

Configure Tomcat for Fedora

Append the following to /etc/tomcat/tomcat.conf

JAVA_OPTS="-Dfcrepo.modeshape.configuration=classpath:/config/jdbc-mysql/repository.json -Dfcrepo.mysql.username=fcrepo -Dfcrepo.mysql.password=<fcrepo_password> -Dfcrepo.mysql.host=localhost -Dfcrepo.mysql.port=3306 -Dfcrepo.home=/var/avalon/fedora/"

Download and run the fcrepo installer

mkdir -p /var/avalon/fedora
chown tomcat:tomcat /var/avalon/fedora
wget https://github.com/fcrepo4/fcrepo4/releases/download/fcrepo-4.7.5/fcrepo-webapp-4.7.5.war -O /usr/share/tomcat/webapps/fedora4.war

Restart Tomcat

systemctl restart tomcat

See if you can access Fedora's REST interface at http://<server host name>:8984/fedora4/rest

Try it out on your local machine and on another machine. If you can't reach the app from another machine, your iptables might need to be changed to allow access. If Fedora is not up, check the tomcat logs in /var/log/tomcat/. Catalina.out and localhost.<date>.log usually provide the best information.

Solr

Avalon makes use of Solr through the Blacklight gem for faceting and relevance-based searching.

Install prerequisites

yum install lsof

Download the solr tarball and run the installation script

Download Solr from http://archive.apache.org/dist/lucene/solr/

wget http://archive.apache.org/dist/lucene/solr/6.6.6/solr-6.6.6.tgz
tar xzf solr-6.6.6.tgz solr-6.6.6/bin/install_solr_service.sh --strip-components=2
bash ./install_solr_service.sh solr-6.6.6.tgz

By default, the script extracts the distribution archive into /opt, configures Solr to write files into /var/solr, and runs Solr as the solr user. Follow the linked guide if you wish to change these defaults.

Create Avalon core for Solr

mkdir -p /tmp/avalon_solr/
wget https://raw.githubusercontent.com/avalonmediasystem/avalon/master/solr/config/solrconfig.xml -O /tmp/avalon_solr/solrconfig.xml
wget https://raw.githubusercontent.com/avalonmediasystem/avalon/master/solr/config/schema.xml -O /tmp/avalon_solr/schema.xml
su solr # Needs to run as solr user
/opt/solr/bin/solr create_core -c avalon -d /tmp/avalon_solr
exit

If you have successfully installed Solr you should be able to access the dashboard page at http://<server host name>:8983/solr

Instructions on how to manually start/stop Solr: https://cwiki.apache.org/confluence/display/solr/Running+Solr

Media Streaming Server

An HLS-enabled server like Wowza, Adobe Media Server (commercial) or Nginx + the HLS module (open-source) can take an mp4 created by Avalon and stream it on the fly.

Nginx instructions

rpm -ihv http://installrepo.kaltura.org/releases/kaltura-release.noarch.rpm
yum install kaltura-nginx

Add /etc/nginx/nginx.conf

user nginx;
worker_processes 4;

events {
  worker_connections 1024;
}

http {
  server {
    listen 8980;

    vod_mode local;
    vod_last_modified 'Sun, 19 Nov 2000 08:52:00 GMT';
    vod_last_modified_types *;
    vod_metadata_cache metadata_cache 512m;
    vod_response_cache response_cache 128m;
    gzip on;
    gzip_types application/vnd.apple.mpegurl;
    open_file_cache          max=1000 inactive=5m;
    open_file_cache_valid    2m;
    open_file_cache_min_uses 1;
    open_file_cache_errors   on;

    location ~ ^/avalon/(?<stream>.+)/(?<resource>.+\.(?:m3u8|ts)) {
      alias /var/avalon/derivatives/$stream;
      vod hls;

      set $token "$arg_token";
      add_header X-Stream-Auth-Token "$token";
      
      sub_filter_types application/vnd.apple.mpegurl;
      sub_filter_once off;
      sub_filter '.ts' ".ts?token=$token";

      auth_request /auth;
      add_header Access-Control-Allow-Headers '*';
      add_header Access-Control-Expose-Headers 'Server,range,Content-Length,Content-Range';
      add_header Access-Control-Allow-Methods 'GET, HEAD, OPTIONS';
      add_header Access-Control-Allow-Origin '*';
      expires 100d;
    }

    location = /auth {
      # resolver 127.0.0.1;
      proxy_pass http://127.0.0.1/authorize.txt?token=$token&name=$stream;
      proxy_pass_request_body off;
      proxy_set_header Content-Length "";
      proxy_set_header X-Original-URI $request_uri;
    }
  }
}

Add /lib/systemd/system/nginx.service

[Unit]
Description=The NGINX HTTP and reverse proxy server
After=syslog.target network.target remote-fs.target nss-lookup.target

[Service]
Type=forking
PIDFile=/run/nginx.pid
ExecStartPre=/usr/sbin/nginx -t
ExecStart=/usr/sbin/nginx
ExecReload=/usr/sbin/nginx -s reload
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true

[Install]
WantedBy=multi-user.target

Add nginx user and let it own nginx stuff

useradd -M -s /bin/nologin nginx
chown -R nginx:nginx /etc/nginx /var/log/nginx 

Make 8980 bindable and start nginx

semanage port -a -t http_port_t  -p tcp 8980
systemctl start nginx

Later: Avalon config should be updated to be compatible with Nginx:

streaming:
  server: :nginx
  http_base: 'http://localhost:8980/avalon'
  content_path: '/var/avalon/derivatives'

FFmpeg & Mediainfo

Download and install ffmpeg (for transcoding & thumbnails)

mkdir -p /tmp/ffmpeg && cd /tmp/ffmpeg
curl https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz | tar xJ
cp `find . -type f -executable` /usr/bin/

Install Mediainfo (for technical metadata)

yum install mediainfo

HTTPD

Install and start the httpd service.

yum install httpd
systemctl start httpd

Apache Passenger and Ruby

Change current user to avalon then install RVM and ruby 2.4.1

yum install ruby sqlite-devel # Needed to build Ruby using RVM.
useradd avalon
su - avalon
curl -L https://get.rvm.io | bash -s stable --ruby=2.5.5

Source the RVM shell (as avalon user) or close the terminal and open it back up.

source /home/avalon/.rvm/scripts/rvm
rvm use 2.5.5
exit

Install Passenger apache module requirements (as root)

yum install pygpgme curl
curl --fail -sSLo /etc/yum.repos.d/passenger.repo https://oss-binaries.phusionpassenger.com/yum/definitions/el-passenger.repo

yum install mod_passenger || yum-config-manager --enable cr && yum install mod_passenger

Create a virtual host for avalon

wget --no-check-certificate https://raw.github.com/avalonmediasystem/config-files/master/apache/20-avalon.conf

...

 -P /etc/httpd/conf.d/
vim /etc/httpd/conf.d/

...

  RailsEnv Development

Restart apache.  With apache running, check passenger-status:

[avalon@localhost ~]$ which passenger-status

~/.rvm/gems/ruby-1.9.3-p429/bin/passenger-status

As Root

mkdir /var/www/avalon

chown avalon:avalon /var/www/avalon

As avalon user:

cd /var/www/avalon

git clone git://github.com/avalonmediasystem/avalon.git

git checkout master

rm  /var/www/avalon/.rvmrc

Add to /var/www/avalon/config/setup_load_paths.rb

if ENV['MY_RUBY_HOME'] && ENV['MY_RUBY_HOME'].include?('rvm')
   begin
     gems_path = ENV['MY_RUBY_HOME'].split(/@/)[0].sub(/rubies/,'gems')
     ENV['GEM_PATH'] = "#{gems_path}:#{gems_path}@global"
     require 'rvm'
     RVM.use_from_path! File.dirname(File.dirname(__FILE__))
   rescue LoadError
     raise "RVM gem is currently unavailable."
   end
 end
 # If you're not using Bundler at all, remove lines bellow
 ENV['BUNDLE_GEMFILE'] = File.expand_path('../Gemfile', File.dirname(__FILE__))
 require 'bundler/setup'

edit /var/www/avalon/.ruby-version and enter 1.9.3-p429

cp config/authentication.yml.example config/authentication.yml

edit config/database.yml

development:
  adapter: mysql2
  database: rails
  username: rails
  password: rails
  pool: 5
  timeout: 5000

yum install mysql-devel

gem install activerecord-mysql2-adapter

gem install mysql2

edit Gemfile.local:

gem 'mysql2', '~>0.3.11'

Run bundle install

edit solr.yml

  development:
    url: http://localhost:8983/solr/

edit fedora.yml

  development:
    user: fedoraAdmin
    password: FedoraPassword
    url: http://127.0.0.1:8983/fedora

create config/matterhorn.yml

development:
  url: http://matterhorn_system_account:CHANGE_ME@localhost:8080/

create config/avalon.yml

development:
  dropbox:
    path: '/var/avalon/dropbox/'
    upload_uri: 'sftp://localhost/var/avalon/dropbox'
    username: 'test'
    password: 'test'
    notification_email_address: ''

rake db:create

This command creates the database.

rake db:migrate

This command runs the database migrations.

set rails environment to development

Visit port 80

Sign in

Create identity:

archivist1@example.com

password/password

This is a known identity with administrative privileges.

Start delayed_job:

in /var/www/avalon (as avalon user)

bundle exec rake delayed_job:start

Delayed Job handles background jobs.  DelayedJob logs to log/delayed_job.log in the avalon directory. 

Red5 Avalon Security Webapp

cd /usr/local/red5/webapps

wget https://github.com/avalonmediasystem/config-files/raw/master/red5/red5-avalon.tar.gz

tar xvzf red5-avalon.tar.gz

chown -R red5:red5 avalon/

edit /usr/local/red5/webapps/avalon/WEB-INF/red5-web.properties:

avalon.serverUrl=http://localhost/

Restart red5

sudo /sbin/service red5 restart

If Red5 is installed and running you should be able to access this page.

Dropbox

groupadd -r dropbox

useradd -r avalondrop

mkdir -p /var/avalon/dropbox

chown avalondrop:dropbox /var/avalon/dropbox

chmod 2775 /var/avalon/dropbox

Add this to /etc/ssh/sshd_config:

# override default of no subsystems
Subsystem sftp internal-sftp
 
# Example of overriding settings on a per-user basis
#Match User anoncvs
# X11Forwarding no
# AllowTcpForwarding no
# ForceCommand cvs server
Match Group dropbox
ChrootDirectory /var/avalon
X11Forwarding no
AllowTcpForwarding no
ForceCommand internal-sftp

/sbin/service sshd restart

Using the System

You can find specific information about using the system in the Collection Manager's Guide. Upload items individually or via batch. Batch demo content is available for your convenience, which includes the media files and an excel document necessary to run a batch.  You can submit a batch directly via SFTP using the avalondrop account you created above.

Known Issues - Release 1.0 - a list of bugs, workarounds, and cautions for using Release 1.0.

...

20-avalon.conf

In 20-avalon.conf add this line inside the VirtualHost tag:

RailsEnv production

If using SSL, the following fix should be added to address BEAST, POODLE, RC4 issues (after the SSLEngine on)

SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:!RC4:+HIGH:+MEDIUM:-LOW

Modify /etc/httpd/conf.d/passenger.conf

PassengerRuby /home/avalon/.rvm/rubies/ruby-2.5.5/bin/ruby

Validate passenger install and restart apache

passenger-config validate-install
systemctl start httpd

Avalon

Grab Avalon code from github

git clone https://github.com/avalonmediasystem/avalon.git /var/www/avalon
chown -R avalon:avalon /var/www/avalon

Set rails environment to production, if it has not defaulted to this. On the first line of /var/www/avalon/config/environment.rb make sure it says 'production'

ENV['RAILS_ENV'] ||= 'production'

Configure database settings

Anchor
mysql2 mysql2

cd /var/www/avalon/config
vim database.yml

Replace database.yml with the correct values for your production environment. Note that the pool setting should be equal or exceed the number of concurrent jobs in Sidekiq.

production:
  adapter: mysql2
  host: localhost
  database: rails
  username: rails
  password: rails
  pool: 20
  timeout: 5000

Install the mysql2 adapter 

yum install cmake #<--will be required for rugged gem
yum install mariadb-devel

Install gems

Run the bundle install

# as root
yum install nodejs # Javascript runtime 

# as avalon
su - avalon
cd /var/www/avalon
gem install bundler
bundle install --with mysql production --without development test
exit

Finish configuring Avalon

Edit /var/www/avalon/config/solr.yml and /var/www/avalon/config/blacklight.yml

  production:
    url: http://localhost:8983/solr/avalon

Edit /var/www/avalon/config/fedora.yml

production:
    user: fedoraAdmin
    password: fedoraAdmin
    url: http://127.0.0.1:8984/fedora4/rest
    base_path: ""

Create streaming directory

# as root
mkdir -p /var/avalon/derivatives
chown avalon:avalon /var/avalon/derivatives

Let Avalon know where your HLS streams are

streaming:
  server: :nginx
  http_base: 'http://localhost:8980/avalon'
  content_path: '/var/avalon/derivatives'

export RAILS_ENV=production
rake secret

grab the output of rake secret and add it to secrets.yml where instruSTDOUTSTDOUTSTDOUTcted.

More information: Configuration Files#config/secrets.yml

Create controlled_vocabulary.yml

cp config/controlled_vocabulary.yml.example config/controlled_vocabulary.yml

If you get an error message saying that you can't connect to the database, take a look at this post and follow some of the troubleshooting steps.

http://stackoverflow.com/questions/5376427/cant-connect-to-local-mysql-server-through-socket-var-mysql-mysql-sock-38

Run the database migrations

rake db:migrate

Install yarn and node modules

# as root
curl --silent --location https://dl.yarnpkg.com/rpm/yarn.repo | sudo tee /etc/yum.repos.d/yarn.repo
yum install yarn

# as avalon
su - avalon
cd /var/www/avalon
yarn install

Precompile assets

# as avalon
RAILS_ENV=production bundle exec rake assets:precompile

Restart Apache

# as root
systemctl restart httpd

Install ImageMagick

# as root
yum install imagemagick

Sidekiq

Avalon uses Sidekiq for background processing, which relies on Redis as its key-value store.

Install Redis

# as root
yum install redis
systemctl start redis

Install Sidekiq

# as root
wget https://raw.githubusercontent.com/mperham/sidekiq/master/examples/systemd/sidekiq.service -O /lib/systemd/system/sidekiq.service

Edit the following lines in sidekiq.service 

WorkingDirectory=/var/www/avalon
ExecStart=/bin/bash -lc '/home/avalon/.rvm/gems/ruby-2.5.5/bin/bundle exec sidekiq -e production'
User=avalon
Group=avalon

# as root
systemctl start sidekiq

Sidekiq logs to STDOUT. 

Additional Configurations

Dropbox

groupadd -r dropbox
useradd -r avalondrop
usermod -G dropbox avalon
mkdir -p /srv/avalon/dropbox
chown avalondrop:dropbox /srv/avalon/dropbox
chmod 2775 /srv/avalon/dropbox

Edit /etc/ssh/sshd_config

# override default of no subsystems
Subsystem sftp internal-sftp
 
# Example of overriding settings on a per-user basis
#Match User anoncvs
# X11Forwarding no
# AllowTcpForwarding no
# ForceCommand cvs server
Match Group dropbox
ChrootDirectory /srv/avalon
X11Forwarding no
AllowTcpForwarding no
ForceCommand internal-sftp

Restart SSH

service sshd restart

Batch ingest

To manually start a batch ingest job, run as avalon user

rake avalon:batch:ingest

To make batch ingest run automatically whenever a manifest is present, you need to add a cron job. This cron job can be created by the whenever gem from reading config/schedule.rb. To preview, run

whenever

this will translate content in schedule.rb to cron job syntax. Once verified, run the following to write job to crontab

whenever --update-crontab

You should get the cron job automatically if you were deploying from Capistrano.

Authentication Strategy

Avalon comes with Persona by default but it can be configured to work with other authentication strategies by using the appropriate omniauth gems. The following example is applicable to Indiana University CAS, it may need some adjustments in order to work with other CAS implementation.

Add to Gemfile

gem 'net-ldap'
gem 'omniauth-cas', :git => "https://github.com/cjcolvar/omniauth-cas.git"

Install new gems

bundle install

Add to config/initializers/my-ldap.rb

module Avalon
      MY_GUEST_LDAP = Net::LDAP.new
      MY_GUEST_LDAP.host = "eads.myuni.edu"
      MY_GUEST_LDAP.authenticate 'cn=******,ou=Accounts,dc=eads,dc=myuni,dc=edu', '******'

      GROUP_LDAP = Net::LDAP.new
      GROUP_LDAP.host = "ads.myuni.edu"
      GROUP_LDAP.authenticate 'cn=******,ou=Accounts,dc=ads,dc=myuni,dc=edu', '******'
      GROUP_LDAP_TREE = "dc=ads,dc=myuni,dc=edu"
end

Add config/initializers/user_auth_cas.rb

require 'net/ldap'

User.instance_eval do
  def self.find_for_cas(access_token, signed_in_resource=nil)
    logger.debug "#{access_token.inspect}"
    #data = access_token.info
    username = access_token.uid
    email = nil

    user = User.where(:username => username).first

    unless user
      if email.nil?
        tree = "dc=ads,dc=myuni,dc=edu"
        filter = Net::LDAP::Filter.eq("cn", "#{username}")
        email = Avalon::GROUP_LDAP.search(:base => tree, :filter => filter, :attributes=> ["mail"]).first.mail.first
      end
      user = User.find_or_create_by_username_or_email(username, email)
      raise "Finding user (#{ user }) failed: #{ user.errors.full_messages }" unless user.persisted?
    end
    user
  end
end

Add to config/settings/production.local.yml

auth:
  configuration:
    - :name: My University
      :logo: my_logo.png
      :provider: :cas
      :params:
        :host: cas.myuni.edu
        :login_url: /cas/login
        :service_validate_url: /cas/validate
        :logout_url: /cas/logout
        :ssl: true

Using the System

You should be able to visit the webpage with just the hostname (ie http://localhost)

Create an admin account

You can  create an account from the command line in the root of your avalon install:

bundle exec rake avalon:user:create avalon_username=user@example.com avalon_password=password avalon_groups=administrator

Additional information

You can find specific information about using the system in the Collection Manager's Guide.  Sample content is available for your convenience.  Upload new items individually or by batch directly via SFTP using the avalondrop account you created above.

Configure additional feataures

Known Issues - a list of bugs, workarounds, and cautions.

Restarting the Server

Before you restart your Avalon server, you'll want to make sure all of the services necessary to run Avalon will start automatically after the restart. Run these commands once and you should be set:

chkconfig --level 345 tomcat on
chkconfig --level 345 mariadb on
chkconfig --level 345 nginx on
chkconfig --level 345 sshd on
chkconfig --level 345 redis on
chkconfig --level 345 sidekiq on
chkconfig --level 345 httpd on