Emulation as a Service as a Docker (Beta)

Thursday, October 08th, 2015 | Author:

Finally, Isgandar found some time to play with Docker. Docker or containers in general allow to shift installation configuration effort from end-users (admins) to the developers. As the bwFLA EaaS-framework is not easy to install that seemed a perfect fit.

Some prerequisites:

For this example we chose to divide & conquer all EaaS component, ie. every EaaS component is deployed as an individual container. This way, we can explain the functionality of each container and the configuration becomes readable. All explanations refer to the runner scripts from our github repository.
Common parameters (mandatory for all scripts)
  • We start all scripts with sudo. If you are unsure, you can run the container in non-priviliged mode, however with some functionality not available (e.g. uploading files directly into the emulated environment).
  • Choose the flavour / version the script should start with –docker eaas/bwfla:demo-august-15
  • As we package application servers, each component needs to be network accessible. Docker takes care of temporary setting of ‘iptables’ rules such that client doesn’t need to perform mapping of ‘host <-> guest’ ports (i.e. NAT). Make sure that your Firewall doesn’t interfere with the rules set-up by Docker application. Any IP valid for any network interface of the host machine, including should be good.Example:  –public-ip-port
Currently EaaS framework consists of at least four components (actually there are five, but the object-archive has not been packaged yet):
  • EmuComp – A container capable of running emulators. Typically such containers are deployed in cluster or cloud environments.
     sudo ./run-flavor-emucomp.sh --docker eaas/bwfla:demo-august-15 --public-ip-port

    This script starts ’emucomp’ module, which is responsible for running individual emulators on dedicated compute nodes (possibly in a cluster/cloud environment). As of now emulators supported in the docker are the following: Qemu, SheepShaver, Basilisk, DosBox, Hatari.

  • EaaS– A container acting as gateway, assigning user-requests to EmuComps in a cloud / cluster deployment. This version only allows only fixed a EmuComp cluster. For dynamic cloud allocation please ask.
     sudo ./run-flavor-eaas.sh --public-ip-port --docker eaas/bwfla:demo-august-15 --emucomp,4 --emucomp,8

    At least one ‘–emucomp <VALUE>’ has to be specified. Repeat the argument if multiple ’emucomp’ modules have to be connected to this ‘eaas’ module. Value of the arguments is composed of the IP:PORT of the ’emucomp’ to be connected and the number of sessions it should supports (usually set to the node’s CPU-count) coming after comma.

  • Image-Archive– The image archive component manages virtual environments (disk images, as well as corresponding meta-data).
    sudo ./run-flavor-imagearchive.sh --docker eaas/bwfla:demo-august-15 --public-ip-port --archive-dir /mnt/data/image-archive

    This script run the ‘image-archive’ module. It requires the location of the image archive directory as a parameter. The ‘nbd-export’ directory should not contain any symbolic links that point to locations outside of the image-archive directory (i.e. only relative paths). This is due to the fact that the image-archive is mounted inside the docker container, which in turn has no access to the host’s file-system.

    Important note:  make sure that the public-ip-port setting chosen for your image-archive instance is accessible by the workflow module and emucomps, in particular make sure that port 10809 is not firewalled. 

    Update: An example image-archive is available for download (26 Mb). It contains 3 example images and meta-data. To use the archive, unpack and export all image files within the images/base directory using symbolic links (make sure you use relative links, see example below). If your OS does not support symbolic links, just copy the images to the nbd-export directory. 

    tar xf image-archive.tgz  #export images via nbd  
    cd image-archive/nbd-export/ 
    ln -s ../images/base/doom.raw 
    ln -s ../images/base/hatari_tOS206us.img 
    ln -s ../images/base/qemu-i386-DOS_6.20_CDROM.raw


  • Workflows – Frontend UI and ready-made workflows. The workflows connect the EaaS gateway (and its configured EmuComps) with user content, such as the image-archive. It contains sample preservation workflows which include archival “ingest/access” of digital objects, full disk images, any accompanying software/libraries. This contains exactly the same functionality as our demo site.
    sudo ./run-flavor-workflows.sh –docker eaas/bwfla:demo-august-15 –public-ip-port –image-archive –eaas-gateway [–object-files test/object-files –base-uri]This script starts the ‘Workflows’ module, which represents a reference implementation of the bwFLA/EaaS API.The ‘–image-archive’ should point to a location of an image-archive, which contains base image of emulated systems, their derivatives, etc.

    The ‘–eaas-gateway’ should point to a location of an EAAS module, which will serve as a main ‘Facade’ for accepting and performing emulation tasks by using one or more ’emucomp’ modules on dedicated compute-nodes.

    The ‘–object-files’ should point to a directory containing user-objects that will appear in the ‘ingest’ workflow. The directory should contain objects in the form “OBJECT_NAME/OBJECT_NAME.iso”. E.g. ‘test/object-files/OBJECT1/OBJECT1.iso’. The ‘-base-uri’ must be specified iff ‘–object-files’ was specified previously. This argument defines the URL-prefix for the location of the objects via which the object can be download through HTTP protocol (’emucomp’ module needs to be able to download the object to inject it into the environment).

    NOTE: we will publish more information on the object-archive structure here. 

Category: bwFLA Projekt, R&D | Leave a Comment