Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
mysql
mariadb> create database fcrepo CHARACTER SET utf8 COLLATE utf8_general_ci;
mariadb> create user 'fcrepo'@'localhost' identified by '<fcrepo_password>';
mariadb> grant all privileges on fcrepo.* to 'fcrepo'@'localhost';

mariadb> create database rails CHARACTER SET utf8 COLLATE utf8_general_ci;
mariadb> create user 'rails'@'localhost' identified by '<rails_pasword>';
mariadb> grant all privileges on rails.* to 'rails'@'localhost';

mariadb> flush privileges;

...

Warning

If you enable SSL on Avalon server, you should also enable SSL on the streaming server to avoid Mixed content warning.

FFmpeg & Mediainfo

Info
titleDownload prebuilt ffmpeg

The following prebuilt binaries are provided by a third party. Proceed with caution.

Download and install ffmpeg

...

(for transcoding & thumbnails)

Code Block
mkdir -p /tmp/ffmpeg && cd /tmp/ffmpeg
curl https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz | tar xJ
cp `find . -type f -executable` /usr/bin/

Install Mediainfo (for technical metadata)

Code Block
yum install mediainfo

HTTPD

Install and start the httpd service.

...

Code Block
yum install ruby sqlite-devel # Needed to build Ruby using RVM.
useradd avalon
su - avalon
curl -L https://get.rvm.io | bash -s stable --ruby=2.5.75

Source the RVM shell (as avalon user) or close the terminal and open it back up.

Code Block
source /home/avalon/.rvm/scripts/rvm
rvm use 2.5.75
exit

Install Passenger apache module requirements (as root)

...

Code Block
PassengerRuby /home/avalon/.rvm/rubies/ruby-2.5.75/bin/ruby

Validate passenger install and restart apache

...

Code Block
git clone https://github.com/avalonmediasystem/avalon.git /var/www/avalon
chown -R avalon:avalon /var/www/avalon

...

Set rails environment to production, if it has not defaulted to this. On the first line of /var/www/avalon/config/environment.rb make sure it says 'production'

Code Block
ENV['RAILS_ENV'] ||= 'production'

Configure database settings
Anchor
mysql2
mysql2

Code Block
cd /var/www/avalon/config
vim database.yml

Replace database.yml with the correct values for your production environment. Note that the pool setting should be equal or exceed the number of concurrent jobs in Sidekiq.

Code Block
production:
  adapter: mysql2
  host: localhost
  database: rails
  username: rails
  password: rails
  pool: 520
  timeout: 5000

Install the mysql2 adapter 

...

Code Block
# as root
yum install nodejs # Javascript runtime 

# as avalon
su - avalon
cd /var/www/avalon
gem install bundler
bundle install --with mysql production --without development test
exit

...

Finish configuring Avalon

Edit /var/www/avalon/config/solr.yml and /var/www/avalon/config/blacklight.yml

Code Block
# as rootproduction:
curl   --silent --location https url: http://dl.yarnpkg.com/rpm/yarn.repo | sudo tee /etc/yum.repos.d/yarn.repo
yum install yarn

# as avalon
su - avalon
cd /var/www/avalon
yarn install

Precompile assets

Code Block
# as avalon
RAILS_ENV=production bundle exec rake assets:precompile

Finish configuring Avalon

Edit /var/www/avalon/config/solr.yml and /var/www/avalon/config/blacklight.yml

Code Block
  production:
    url: http://localhost:8983/solr/avalon

...

localhost:8983/solr/avalon

Edit /var/www/avalon/config/fedora.yml

Code Block
production:
    user: fedoraAdmin
    password: fedoraAdmin
    url: http://127.0.0.1:8984/fedora4/rest
    base_path: ""

Create streaming directory

Code Block
# as root
mkdir -p /var/avalon/derivatives
chown avalon:avalon /var/avalon/derivatives


Info
titleAvalon config file

Avalon settings now live in /var/www/avalon/config/

...

Code Block
  production:
    user: fedoraAdmin
    password: fedoraAdmin
    url: http://127.0.0.1:8984/fedora4/rest
    base_path: ""
Info
titleAvalon config file

Avalon settings now live in /var/www/avalon/config/settings.yml. The default values should be sufficient to start with.

They can settings.yml. The default values should be sufficient to start with.

They can be selectively overwritten by creating a settings/<environment>.yml, or by using environment variables. Consult the config gem doc to understand how it works, or Avalon's documentation to customize this file for your installation.

...

grab the output of rake secret and add it to secrets.yml where instructedinstruSTDOUTSTDOUTSTDOUTcted.

More information: Configuration Files#config/secrets.yml

...

Code Block
rake db:migrate

Set rails environment to production, if it has not defaulted to this. On the first line of /var/www/avalon/config/environment.rb make sure it says 'production'

Code Block
ENV['RAILS_ENV'] ||= 'production'

Sidekiq

Avalon uses Sidekiq for background processing, which relies on Redis as its key-value store.

Install Redis

Code Block
# as root
yum install redis
systemctl start redis

...

Install yarn and node modules

Code Block
# as root
curl --silent --location https://dl.yarnpkg.com/rpm/yarn.repo | sudo tee /etc/yum.repos.d/yarn.repo
yum install yarn

# as avalon
su - avalon
cd /var/www/avalon
yarn install

Precompile assets

Code Block
# as root
wget https://raw.githubusercontent.com/mperham/sidekiq/master/examples/systemd/sidekiq.service -O /lib/systemd/system/sidekiq.service

Edit the following lines in sidekiq.service 

Code Block
WorkingDirectory=/var/www/avalon
ExecStart=/bin/bash -lc '/home/avalon/.rvm/gems/ruby-2.5.7/bin/bundle exec sidekiq -e production'
User=avalon
Group=avalonavalon
RAILS_ENV=production bundle exec rake assets:precompile

Restart Apache

Code Block
# as root
systemctl restart httpd

Install ImageMagick

Code Block
# as root
systemctlyum startinstall sidekiqimagemagick

Sidekiq

Sidekiq logs STDOUT. 

Additional Configurations

Dropbox

Code Block
groupadd -r dropbox
useradd -r avalondrop
usermod -G dropbox avalon
mkdir -p /srv/avalon/dropbox
chown avalondrop:dropbox /srv/avalon/dropbox
chmod 2775 /srv/avalon/dropbox

Edit /etc/ssh/sshd_config

Code Block
# override default of no subsystems
Subsystem sftp internal-sftp
 
# Example of overriding settings on a per-user basis
#Match User anoncvs
# X11Forwarding no
# AllowTcpForwarding no
# ForceCommand cvs server
Match Group dropbox
ChrootDirectory /srv/avalon
X11Forwarding no
AllowTcpForwarding no
ForceCommand internal-sftp

Restart SSH

Code Block
service sshd restart

Batch ingest

To manually start a batch ingest job, run as avalon user

Code Block
rake avalon:batch:ingest

To make batch ingest run automatically whenever a manifest is present, you need to add a cron job. This cron job can be created by the whenever gem from reading config/schedule.rb. To preview, run

Code Block
whenever

this will translate content in schedule.rb to cron job syntax. Once verified, run the following to write job to crontab

Code Block
whenever --update-crontab

You should get the cron job automatically if you were deploying from Capistrano.

Avalon uses Sidekiq for background processing, which relies on Redis as its key-value store.

Install Redis

Code Block
# as root
yum install redis
systemctl start redis

Install Sidekiq

Code Block
# as root
wget https://raw.githubusercontent.com/mperham/sidekiq/master/examples/systemd/sidekiq.service -O /lib/systemd/system/sidekiq.service

Edit the following lines in sidekiq.service 

Code Block
WorkingDirectory=/var/www/avalon
ExecStart=/bin/bash -lc '/home/avalon/.rvm/gems/ruby-2.5.5/bin/bundle exec sidekiq -e production'
User=avalon
Group=avalon


Code Block
# as root
systemctl start sidekiq

Sidekiq logs to STDOUT. 

Warning
titletmp Error after uploading file

When ingesting a media file, you may encounter an error message saying that file:///tmp/filename can’t be accessed or located.  This may result from the protected temp file settings that are defaults in CentOS 7.  Fix by changing “true” to “false” for PrivateTmp in these files in /usr/lib/systemd/system:

sidekiq.service  PrivateTmp=false
nginx.service   PrivateTmp=false
httpd.service   PrivateTmp=false

Additional Configurations

Dropbox

Code Block
groupadd -r dropbox
useradd -r avalondrop
usermod -G dropbox avalon
mkdir -p /srv/avalon/dropbox
chown avalondrop:dropbox /srv/avalon/dropbox
chmod 2775 /srv/avalon/dropbox

Edit /etc/ssh/sshd_config

Code Block
# override default of no subsystems
Subsystem sftp internal-sftp
 
# Example of overriding settings on a per-user basis
#Match User anoncvs
# X11Forwarding no
# AllowTcpForwarding no
# ForceCommand cvs server
Match Group dropbox
ChrootDirectory /srv/avalon
X11Forwarding no
AllowTcpForwarding no
ForceCommand internal-sftp

Restart SSH

Code Block
service sshd restart

Batch ingest

To manually start a batch ingest job, run as avalon user

Code Block
rake avalon:batch:ingest

To make batch ingest run automatically whenever a manifest is present, you need to add a cron job. This cron job can be created by the whenever gem from reading config/schedule.rb. To preview, run

Code Block
whenever

this will translate content in schedule.rb to cron job syntax. Once verified, run the following to write job to crontab

Code Block
whenever --update-crontab

You should get the cron job automatically if you were deploying from Capistrano.

Authentication Strategy

Avalon comes with Persona by default but it can be configured to work with other authentication strategies by using the appropriate omniauth gems. The following example is applicable to Indiana University CAS, it may need some adjustments in order to work with other CAS implementation.

Add to Gemfile

Code Block
gem 'net-ldap'
gem 'omniauth-cas', :git => "https://github.com/cjcolvar/omniauth-cas.git"

Install new gems

Code Block
bundle install

Add to config/initializers/my-ldap.rb

Code Block
module Avalon
      MY_GUEST_LDAP = Net::LDAP.new
      MY_GUEST_LDAP.host = "eads.myuni.edu"
      MY_GUEST_LDAP.authenticate 'cn=******,ou=Accounts,dc=eads,dc=myuni,dc=edu', '******'

      GROUP_LDAP = Net::LDAP.new
      GROUP_LDAP.host = "ads.myuni.edu"
      GROUP_LDAP.authenticate 'cn=******,ou=Accounts,dc=ads,dc=myuni,dc=edu', '******'
      GROUP_LDAP_TREE = "dc=ads,dc=myuni,dc=edu"
end

Add config/initializers/user_auth_cas.rb

Code Block
require 'net/ldap'

User.instance_eval do
  def self.find_for_cas(access_token, signed_in_resource=nil)
    logger.debug "#{access_token.inspect}"
    #data = access_token.info
    username = access_token.uid
    email = nil

    user = User.where(:username => username).first

    unless user
      if email.nil?
        tree = "dc=ads,dc=myuni,dc=edu"
        filter = Net::LDAP::Filter.eq("cn", "#{username}")
        email = Avalon::GROUP_LDAP.search(:base => tree, :filter => filter, :attributes=> ["mail"]).first.mail.first
      end
      user = User.find_or_create_by_username_or_email(username, email)
      raise "Finding user (#{ user }) failed: #{ user.errors.full_messages }" unless user.persisted?
    end
    user
  end
end

Add to config/settings/production.local.yml

Code Block
auth:
  configuration:
    - :name: My University
      :logo: my_logo.png
      :provider: :cas
      :params:
        :host: cas.myuni.edu
        :login_url: /cas/login
        :service_validate_url: /cas/validate
        :logout_url: /cas/logout
        :ssl: true

Using the System

You should be able to visit the webpage with just the hostname (ie http://localhost)

Create an admin account

Click on "Sign in" in the upper right corner of the website main page. Set up a default identity with administrative privileges using the following properties.

Code Block
archivist1@example.com
<some password>

You can also You can  create an account from the command line in the root of your avalon install:

...