Page tree
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 34 Next »

These are the instructions for creating an end-user VM from scratch.

VM Configuration

  • Using VirtualBox:
    • 1 Core
    • 3G RAM
    • 500G System Disk (VDI, Dynamically Allocated)
    • Set network to "Attached to Bridged Adapter"

Base OS Install

  • Centos 6.4 x86-64
  • English language and keyboard
  • hostname: localhost.localdomain (the default)
    • if it is localhost it should set it to the dhcp hostname provided.
  • eth0:
    • Clear "Device MAC address"
    • Connect automatically
    • IPv4 Settings
      • Automatic (DHCP)

  • set timezone to america/indiana/indianapolis, clock uses utc
  • root password is 'changeme'
  • partitioning:
    • custom layout
      • /dev/sda1:  500M ext4 mounted as /boot
      • /dev/sda2:  (remainder of disk) Physical volume
      • Volume Group:  vg_avalon
        • PVs: /dev/sda2
        • LV lv_swap 2G
        • LV lv_root (remainder of disk) ext4 mounted as /
  • Desktop install
  • Reboot / Firstboot
    • Leave all of the defaults, except...
    • Do Not create a new user
  • Log in as root
    • do not check "do not show me this again" to the "you are running as root" dialog.  Just close it.
    • yum update
    • open these ports in the firewall:
      • tcp/22
      • tcp/80
      • tcp/1935
      • tcp/18080
    • clean up eth0 configuration in /etc/sysconfig/network-scripts/ifcfg-eth0:
      • remove UUID
      • remove any DHCP_CLIENT_ID
  • OPTIONAL:
    • shut down the VM
    • at this point the vm is at "baseOS" and can be used as a cloning point

There is a bug in virtual box with IPv6 handling over a bridged network.  When the bug is present the top transfer speeds for IPv6 are around 18kb/s.  Since RHEL (and derivatives) prefer IPv6 when looking up hosts, it is not uncommon to get an IPv6 host address for updates.  To disable IPv6 on the guest add these lines to /etc/sysctl.conf

net.ipv6.conf.all.disable_ipv6 = 1
 net.ipv6.conf.default.disable_ipv6 = 1

 

Avalon Install

  • OPTIONAL:
    • clone the baseOS image as puppet-YYYYMMDD
  • Use the installation notes from https://github.com/avalonmediasystem/avalon-installer
    • When setting FACTER_* environment variables, set
      • FACTER_avalon_dropbox_password to "changeme"
      • FACTER_avalon_hostname to "localhost.localdomain"
  • HACK:
    • add avalon to the dropbox group
    • verify that /var/avalon/dropbox is mode 2775

Demo Content

  • OPTIONAL:
    • clone the puppet image as democontent-YYMMDD
  • Get the demo fixtures bundle

    cd /tmp
    wget http://dlib.indiana.edu/~cjcolvar/DemoFixturesBatch.tar.gz     #We need a better place to serve this from
    tar xvzf DemoFixturesBatch.tar.gz
    chown -R avalondrop.dropbox DemoFixturesBatch
    mv DemoFixturesBatch /var/avalon/dropbox

    Monitor progress at http://<whatever>:18080.  

  • Modify /var/www/avalon/current/app/views/catalog/_home_text.html.erb and replace sample content with something like:

     <ul class="thumbnails">
    	<li class="span3"><h5>Featured Video</h5><a href="<%= media_object_path('avalon:1') %>" class="thumbnail" <%= image_tag "thumbnail.name.png", alt: "Featured Video" %></a></li>
    	<li class="span3"><h5>Featured Video</h5><a href="<%= media_object_path('avalon:2') %>" class="thumbnail" <%= image_tag "thumbnail.name.png", alt: "Featured Video" %></a></li>
    	<li class="span3"><h5>Featured Video</h5><a href="<%= media_object_path('avalon:3') %>" class="thumbnail" <%= image_tag "thumbnail.name.png", alt: "Featured Video" %></a></li>
    </ul>

    Thumbnail images should be stored in /var/www/avalon/current/app/assets/images

  • Remove source data

    rm /tmp/DemoFixturesBatch.tar.gz
    rm -rf /var/avalon/dropbox/DemoFixturesBatch

VM Image Prep

  • OPTIONAL:
    • clone the latest democontent image as dist-YYMMDD
  • Install avalon-vm-1.0-1.noarch.rpm to install the firstboot changes and tools
  • Prepare the disk for compacting
    • deleted files still take up space on the disk image.  Write a big zero file to clear out that unused data

      dd if=/dev/zero of=/tmp/foo bs=1M count=8192 oflags=direct; rm /tmp/foo
  • run /usr/share/avalon/dist-prep to prepare the machine for release
  • clear the command line history  (history -c)
  • shutdown the VM
  • Compact the disk image
    • On the host run:

      VBoxManage modifyhd avalon-vm-disk-YYMMDD.vdi --compact
  • Export the appliance
    • use a 0.9 version OVA
    • use a MANIFEST
  • copy the OVA to the distribution point

 

Adding the demo batch increased the size of the VM download from 3.2G to 7.4G.  Partially its because we've make derivatives and there's really new content on the VM.  But for a bigger reason, any disk space that gets allocated is stored in the image file – the VM doesn't know what's being used and what isn't.  Since we download a largish package, untar it, and create a bunch of temp files, the VM disk image has a bunch of unused (from the OS point of view) disk that it still has to keep track of.  Future research should include how to 'unuse' the disk space.

"zerofree" should do the trick, when combined with vboxmanager modifyhd <name> --compact

Cleaning Unused Disk Space

VirtualBox's VBoxManager tool supports a compact option when modifying hard drives.  Unfortunately it only supports VDI so I'm experimenting with using VDI instead of VMDK for the disk container format.   For the purposes of the release, there should be no functional difference, except that VDI is less efficient than VMDK (on the Base OS install the VDI is 6% larger).

After a BaseOS install (as described above), the VDI disk image takes 4,304M of space.  A df on the system shows that 3405M is used for data, so there is nearly 1G of space which is a combination of filesystem overhead, VM sparse disk overhead, and non-zero-unused data. 

The first step is to see if the filesystem can be compacted using

VBoxManage modifyhd --compact avalon-vm-base-130419b.vdi

After a few minutes it comes back and shows that 4303M is being used – we saved 1M by running it.  I'm not impressed.

I know that I downloaded about 200M worth of update packages.  If the filesystem writes to the same places on the disk as it did before, a big dd should change that unused data to zeros:

dd if=/dev/zero of=/tmp/foo bs=10M count=1024; rm /tmp/foo

Afterward, the container was 4313M so the container grew by 10M – after writing 10G of zeros.  One would assume that the original deleted data is now zeros and the extra size is overhead of mapping a bunch of empty blocks.

Compacting it gives us a new disk size:  4077M – we've saved 400M by writing a big bunch of zeros.  Hooray!

I've written 4G of data to the image and it is currently 8176M on disk, even after erasing the files from the filesystem.

I've written 8G of zeros to the disk

 

 

 

  • No labels