Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Avalon R6 instructions
Note

This documentation is for Release 56.x. For documentation on previous releases, please select from the options below.

  • Release 1.x version of this page: v.81
  • Release 2.x version of this page: v.87.
  • Release 3.0-3.1 version of this page: v.111.
  • Release 3.2 version of this page: v.116.
  • Release 4.0 version of this page: v.143.
  • Release 5.x version of this page: v.163.

 

These instructions provide a recipe for building your own all-in-one Avalon system from scratch on CentOS or Red Hat Enterprise Linux, version 6.x is supported, 7.x is experimentalwill be supported soon.  Please note that while an all-in-one installation as outlined here is certainly suitable for testing and demos,  a single, all-in-one, server may not be suitable for production environments.  For recipes on building each piece of the Avalon system on different hosts, (as would typically be done for a production environment), please see the instructions for (standalone) environments here:  Fedora (standalone)MySQL (standalone)Web (standalone)Solr (standalone)Red5 (standalone), and Matterhorn (standalone).

...

Ready the Installation Environment

Install EPEL

Code Block
rpm -ivh http://linux.mirrors.es.net/fedora-epel/6/i386/epel-release-6-8.noarch.rpm

This package has libyaml-devel which is required by ruby and not provided by Redhat.

On CentOS 7, please use the EPEL 7 package

Code Block
rpm -ivh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

Make sure a valid hostname is resolvable

The default hostname is “avalon.dev”, so name the machine this and enter it into /etc/hosts

Code Block
# hostname
avalon.dev
# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 avalon.dev

...

The Avalon Media System requires several ports to be open to client browsers.

Here are the port settings that will need to be configured:

...

Make sure a valid hostname is resolvable

The default hostname is “avalon.dev”, so name the machine this and enter it into /etc/hosts

Code Block
# hostname
avalon.dev
# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 avalon.dev

Configure iptables 
Anchor
iptables
iptables

The Avalon Media System requires several ports to be open to client browsers.

Here are the port settings that will need to be configured:

PortPurposeExternal?
80HTTP (Avalon)Yes
1935RTMP (red5)Yes
5080HTTP (red5)No
8983HTTP (Fedora/Solr)No
8080HTTP (Matterhorn)Yes

...

Code Block
chmod +x /etc/sysconfig/avalon-iptables-config.sh
/etc/sysconfig/avalon-iptables-config.sh

 

If you run into connection issues you can disable the iptables, by running "service iptables stop". This will completely drop your firewall. When finished troubleshooting run "service iptables start".

...

Info
titleYou may have to disable SELinux completely if there's Passenger installation problem
vim /etc/selinux/config #change the value of `SELINUX` to `disabled`

Install EPEL

This package has libyaml-devel which is required by ruby and not provided by Redhat.

Code Block
rpm -ivh http://linux.mirrors.es.net/fedora-epel/6/i386/epel-release-6-8.noarch.rpm


Add the Avalon repository 
Anchor
AvalonRepo
AvalonRepo

...

Code Block
yum groupinstall "Development Tools"
yum install readline-devel zlib-devel libyaml-devel libffi-devel openssl-devel libxml2-devel libxslt-devel cmake

Tomcat

...

Install Java 8

Code Block
yum install tomcatjava-7.0.32
service tomcat start1.8.0-openjdk

Fedora Commons Repository

Tomcat

Fedora runs as a webapp in Tomcat

Install Apache Tomcat 

Code Block
titleRHEL 7
yum install tomcat
vim /usretc/local/tomcat/conf/server.xml
Info

CentOS 7 user may need to switch to EPEL6 temporarily for tomcat dependencies

on line 70, change the Tomcat connector port from 8080 to 8983:

Code Block
<Connector port="8983" protocol="HTTP/1.1"

Add Tomcat manager user

...

 #line 71, change the Tomcat connector port from 8080 to 8984

Add Tomcat manager user

By default, no user has access to the Tomcat Manager App.  

...

Define a user in /etc/tomcat/tomcat-users.xml

...

Define a user in tomcat-users.xml with  with access to the manager-gui role. Below is a very basic example.

Code Block
<tomcat-users>
  <role rolename="manager-gui"/>
  <user username="admin" password="tomcat<insert strong password here>" roles="manager-gui"/>
</tomcat-users>

...

Configure Tomcat for Fedora

Code Block
service tomcat restart

See if you can log in to the manager app at http://<server host name>:8983/manager/html

...

Solr

Avalon makes use of Solr through the Blacklight gem for faceting and relevance-based searching.

Download the solr tarball and unpack it

Download Solr from http://www.apache.org/dyn/closer.cgi/lucene/solr/4.2.0

Code Block
wget http://archive.apache.org/dist/lucene/solr/4.2.0/solr-4.2.0.tgz
tar xvf solr-4.2.0.tgz
mv solr-4.2.0 /usr/local/solr
cd /usr/local/solr/dist/
cp solr-4.2.0.war /usr/local/tomcat/webapps/solr.war
mv solr-4.2.0.war /usr/local/solr/solr.war

Add XML configurations

Edit /usr/local/tomcat/conf/Catalina/localhost/solr.xml and add

Code Block
<Context docBase="/usr/local/solr/solr.war" debug="0" crossContext="true">
 <Environment name="solr/home" type="java.lang.String" value="/usr/local/solr" override="true"/>
</Context>

Edit /usr/local/solr/solr.xml and insert

Code Block
<?xml version="1.0" encoding="UTF-8" ?>
  <solr persistent="true">
    <cores defaultCoreName="avalon" adminPath="/admin/cores" zkClientTimeout="${zkClientTimeout:15000}" hostPort="8983" hostContext="solr">
    <core instanceDir="avalon/" name="avalon"/>
  </cores>
</solr>

Download and Install Avalon specific Solr files from github

...

Append the following to /etc/sysconfig/tomcat

Code Block
JAVA_OPTS="${JAVA_OPTS} -Dfcrepo.modeshape.configuration=classpath:/config/file-simple/repository.json -Dfcrepo.home=/var/avalon/fedora/"

Restart Tomcat

Code Block
service tomcat restart

Download and run the fcrepo installer

Code Block
mkdir -p /var/avalon/fedora
chown tomcat:tomcat /var/avalon/fedora
wget https://github.com/fcrepo4/fcrepo4/releases/download/fcrepo-4.7.1/fcrepo-webapp-4.7.1.war -O /usr/share/tomcat/webapps/fedora4.war

See if you can access Fedora's REST interface at http://<server host name>:8984/fedora4/rest

Try it out on your local machine and on another machine. If you can't reach the app from another machine, your iptables might need to be changed to allow access. If Fedora is not up, check the tomcat logs in /var/log/tomcat/. Catalina.out and localhost.<date>.log usually provide the best information.

Solr

Avalon makes use of Solr through the Blacklight gem for faceting and relevance-based searching.

Install prerequisites

Code Block
yum install lsof

Download the solr tarball and run the installation script

Download Solr from http://www.apache.org/dyn/closer.cgi/lucene/solr/6.4.1

Code Block
wget http://www-us.apache.org/dist/lucene/solr/6.4.1/solr-6.4.1.tgz
tar xzf solr-6.4.1.tgz solr-6.4.1/bin/install_solr_service.sh --strip-components=2
bash ./install_solr_service.sh solr-6.4.1.tgz

By default, the script extracts the distribution archive into /opt, configures Solr to write files into /var/solr, and runs Solr as the solr user. Follow the linked guide if you wish to change these defaults.

Create Avalon core for Solr

Code Block
mkdir -p /tmp/avalon_solr/
wget https://raw.githubusercontent.com/avalonmediasystem/avalon/develop/solr/config/solrconfig.xml -O /tmp/avalon_solr/solrconfig.xml
wget https://raw.githubusercontent.com/avalonmediasystem/avalon/develop/solr/config/schema.xml -O /tmp/avalon_solr/schema.xml
su solr # Needs to run as solr user
/opt/solr/bin/solr create_core -c avalon -d /tmp/avalon_solr
exit

If you have successfully installed Solr you should be able to access the dashboard page at http://<server host name>:8983/solrIf you can't access the dashboard, check the tomcat logs in /usr/local/tomcat/logs/. Catalina.out and localhost.<date>.log usually provide the best information.

Instructions on how to manually start/stop Solr: https://cwiki.apache.org/confluence/display/solr/Running+Solr

MySQL

Avalon uses MySQL for storing search queries, user data and roles, and as a back end for asynchronously sending requests to Matterhorn.

...

Code Block
#mysql
Welcome to the MySQL monitor. Commands end with ; or \g.
...etc...
mysql>

Create a database for the Fedora Commons Repository software and the Avalon web application and add a user to it it

Code Block
create database fedora3rails;
create user 'fcreporails'@'localhost' identified by 'fcreporails';
grant all privileges on fedora3rails.* to 'fcreporails'@'localhost';
flush privileges; 

...

Check your work and exit

Code Block
createmysql> databaseshow rails;
create user 'rails'@'localhost' identified by 'rails';
grant all privileges on rails.* to 'rails'@'localhost';
flush privileges;

Check your work and exit

Code Block
mysql> show databases;
+-----databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| fedora3            |
| mysql              |
| rails              |
| test               |
+--------------------+
5 rows in set (0.00 sec)
mysql> select user, host from mysql.user;
+--------+--------------+
| user   | host         |
+--------+--------------+
| root   | 127.0.0.1    |
|        | 129.79.32.87 |
| root   | 129.79.32.87 |
|        | localhost    |
| fcrepo | localhost    |
| rails  | localhost    |
| root   | localhost    |
+--------+--------------+
7 rows in set (0.00 sec)
 
mysql> exit;
Bye

See documentation for your version of MySQL Server for detailed syntax (http://dev.mysql.com/doc/refman/5.1/en/create-database.html )

Fedora Commons Repository

Download and run the fcrepo installer
Code Block
wget http://sourceforge.net/projects/fedora-commons/files/fedora/3.6.2/fcrepo-installer-3.6.2.jar/download -O fcrepo-installer-3.6.2.jar
java -jar fcrepo-installer-3.6.2.jar

Use the following values in place of defaults

SettingValue
Installation type
custom
Fedora home directory
/usr/local/fedora
Fedora administrator password
fedoraPassword 
Fedora server host
localhost
Fedora application server context
fedora
Authentication requirement for API-A
false
SSL availability
true
SSL required for API-A
false
SSL required for API-M
true
Servlet engine
existingTomcat
Tomcat home directory
/usr/local/tomcat
Tomcat HTTP port
8983
Tomcat shutdown port
8005
Tomcat Secure HTTP port
8443
Keystore file
default
Keystore password
changeit
Keystore type
JKS
Database
mysql
MySQL JDBC driver
included
Database username
fcrepo
Database password
fcrepo

Accept remaining defaults then add permissions for Tomcat and restart Tomcat

Code Block
chown -R tomcat7:tomcat /usr/local/fedora
service tomcat restart--------+
| root   | 127.0.0.1    |
|        | 129.79.32.87 |
| root   | 129.79.32.87 |
|        | localhost    |
| rails  | localhost    |
| root   | localhost    |
+--------+--------------+
7 rows in set (0.00 sec)
 
mysql> exit;
Bye

See documentation for your version of MySQL Server for detailed syntax (http://dev.mysql.com/doc/refman/5.1/en/create-database.html )

Red5 Media Server

Red5 is an open source alternative to Adobe Media Server. If using the Adobe Media Server you can skip to the next step.

...

Download and install Red5.

Code Block
wget http://yumrepo-publicrepo.libraryavalonmediasystem.northwestern.eduorg/red5-1.0.1.tar.gz
tar xvf red5-1.0.1.tar.gz
mv red5-server-1.0 /usr/local/red5

...

Code Block
yum install SDL-devel a52dec-devel bzip2-devel faad2-devel freetype-devel frei0r-plugins-devel \
	gsm-devel imlib2-devel lame-devel libdc1394-devel libraw1394-devel librtmp-devel libtheora-devel \
	libva-devel libfaac-devel libvdpau-devel libstdc++-devel libvorbis-devel libvpx-devel \
	mediainfo-0.7.61-1 opencore-amr-devel opencv-devel openjpeg-devel openssl-devel schroedinger-devel \
	speex-devel texi2html vo-aacenc-devel x264-devel xvidcore-devel yasm zlib-devel

If CentOS 7 complains about libdc1394-devel not found

Code Block
languagebash
sudo yum -v --enablerepo=epel clean expire-cache
sudo yum --enablerepo=epel install libdc1394devel yasm zlib-devel

Install rpmdev-setuptree

...

Code Block
org.opencastproject.streaming.directory=/var/avalon/rtmp_streams
org.opencastproject.hls.directory=/var/avalon/hls_streams

Also check in in /usr/local/matterhorn/etc/load/org.opencastproject.organization-mh_default_org.cfg

...

Replace database.yml with the correct values for your production environment

Code Block
productiondevelopment:
  adapter: mysql2
  host: localhost
  database: rails
  username: rails
  password: rails
  pool: 5
  timeout: 5000

Install the mysql2 adapter 

Code Block
yum install cmake #<--will be required for rugged gem
yum install mysql-devel
su - avalon
gem install activerecord-mysql2-adapter
gem install mysql2
su - root
yum install cmake #<--will be required for rugged gem
vim /var/www/avalon/Gemfile

...

Code Block
#    gem 'activerecord-jdbcsqlite3-adapter'
#    gem 'jdbc-sqlite3'
#    gem 'sqlite3'

Run the bundle install

...

Code Block
# as root
yum install nodejs # Javascript runtime 

# as avalon
cd /var/www/avalon
gem update debugger-ruby_core_source
gem install bundler
bundle install

...

Edit /var/www/avalon/config/solr.yml and /var/www/avalon/config/blacklight.yml

Code Block
  development:
    url: http://localhost:8983/solr/avalon

Edit /var/www/avalon/config/fedora.yml

Code Block
  development:
    user: fedoraAdmin
    password: fedoraPassword
    url: http://127.0.0.1:8983/fedora8984/fedora4/rest
    base_path: /dev

Create  /var/www/avalon/config/matterhorn.yml

...

Code Block
development:
  dropbox:
    path: '/var/avalon/dropbox/'
    upload_uri: 'sftp://localhost/var/avalon/dropbox'
    username: 'test'
    password: 'test'
    notification_email_address: ''
  domain:
    port: 80

Anchor
secrets
secrets
Create the secrets.yml file:

...

Change the secrets.yml file:

Code Block
cd /var/www/avalon/config
rake secret

 grab the output of rake secret and add it to secrets.yml where instructed.

...

Create controlled_vocabulary.yml

Code Block
cd /var/www/avalon/config
cp controlled_vocabulary.yml.example controlled_vocabulary.yml

 

Create the database using rake

Code Block
su# -as avalon user
cd /var/www/avalon
rake db:create

...

Click on "Sign in" in the upper right corner of the website main page. Set up a default identity with the following properties.

 

Code Block
archivist1@example.com
password

 

This is a known identity with administrative privileges.

Start delayed_job

As avalon run

...

following properties.

Code Block
archivist1@example.com
<some password>

This is a known identity with administrative privileges.

Redis & Resque

Avalon uses Resque for background processing, which relies Redis as its key-value store.

Install Redis

Code Block
yum install redis

Start Resque

Code Block
# as avalon
cd /var/www/avalon/
BACKGROUND=yes bundle exec rake environment resque:scheduler
BACKGROUND=yes QUEUE=* bundle exec rake delayed_jobresque:start

...

work

Resque logs to log/delayed_jobresque.log in the avalon directory. 

...