Vagrant tricks – overwrite variables

In this post I will go over a useful vagrant trick I have been using, overwriting variables in the Vagrantfile.
Since the vagrant files are essentially just ruby files, we can take advantage of that and overwrite variables quite easily allowing us to tune the infrastructure we are building up. We can define variables in the Vagrantfile with some defaults and have a separate ruby file that we require in if it is present where we overwrite these defaults.

Example

Vagrantfile

$worker_box_name = "centos/7"
$worker_count = 3
$worker_memory = 1024
CLUSTER_CONFIG = File.expand_path("cluster-config.rb")
if File.exist?(CLUSTER_CONFIG)
  require CLUSTER_CONFIG
end
Vagrant.configure(2) do |config|
  (1..$worker_count).each do |i|
    config.vm.define "w#{i}" do |w|
      w.vm.box = $worker_box_name
      w.vm.hostname = "w#{i}"
      w.vm.provider "virtualbox" do |vb|
        vb.memory = $worker_memory
      end
      # provision steps...
    end
  end
end

cluster-config.rb

$worker_count = 2
$worker_memory = 2048

This will create a cluster with 2 workers with 2048mb memory instead of the default 3 worker with 1024mb memory

Vagrant tricks – Creating local baseboxes

In this blog post I am going to go over a useful trick that I have been using to minimize the time of provisioning a vagrant box. Basically what it is about is having two vagrant or more vagrant files in the project (depending on what you are doing) and a script that creates a vagrant box from one of the vagrant files, then use the box in the next one.

Example

We are going to setup a machine with some applications that takes some time to install (can be what ever). Then we are going to configure the installed applications after they are installed, that will be our second vagrant script. Our first vagrant file will be named Vagrantfile.Basebox and our main vagrant file will be named Vagrantfile.

Vagrantfile.Basebox

$centos_box = "centos/7"
Vagrant.configure(2) do |config|
  config.vm.define "basebox" do |basebox|
    basebox.vm.box = $centos_box
    basebox.ssh.insert_key = false
    basebox.vm.synced_folder ".", "/vagrant", type: "virtualbox"
    basebox.vm.provision :shell, path: "long-running-task.sh"
    # more provisioning if needed
  end
end

This will be the script that creates the base box. I chose centos because there is a little quirk that needs to be addressed ( addressed in the comments in the script )

create-basebox.sh

#!/usr/bin/env bash

BOX_NAME=hx/base-centos

TEST_IF_BOX_EXISTS=$(vagrant box list | grep $BOX_NAME)
if [ "$1" = "-f" ]; then
    TEST_IF_BOX_EXISTS=""
    vagrant box remove $BOX_NAME
    if [ "$?" -ne 0 ]; then
        echo "Could not remove box, check if you have running machines using the box"
        exit 1
    fi
fi
if [ "$TEST_IF_BOX_EXISTS" = "" ]; then
    echo "$BOX_NAME does not exist, creating $BOX_NAME"
    mv Vagrantfile Vagrantfile.tmp
    cp Vagrantfile.base.rb Vagrantfile
    vagrant up basebox
    vagrant halt basebox
    # Centos specific, if you are using a different distro,
    # the next couple of lines are not necessary 
    vagrant up basebox
    vagrant halt basebox
    VBOX_INSTANCE=$(vboxmanage list vms | grep -oP "${PWD##*/}_basebox_[0-9_]*")
    vagrant package --base $VBOX_INSTANCE --output base.box
    # Handle paths differently when using cygwin
    if $(pwd | grep -q cygdrive) ; then
      BOX_FILE_PATH=$(echo "$PWD" | sed -E 's/\/cygdrive\/([a-z])(.*)/file:\/\/\/\1:\2\/base.box/')
    else
      BOX_FILE_PATH=$(pwd | awk '{print "file:" $1 "/base.box"}')
    fi
    vagrant box add $BOX_NAME $BOX_FILE_PATH
    vagrant destroy -f
    # Removing the box some times fails because the file handle has not been 
    # released. So put a short sleep before the remove 
    sleep 3
    rm -f base.box
    rm Vagrantfile
    mv Vagrantfile.tmp Vagrantfile
    echo "Done creating $BOX_NAME"
else
    echo "$BOX_NAME exists, moving on..."
fi

Then we can use the base box in the our Vagrantfile.

Vagrantfile

$custom_box = "hx/base-centos"
Vagrant.configure(2) do |config|
  config.vm.define "custombox" do |custombox|
    custombox.vm.box = $custom_box
    custombox.ssh.insert_key = false
    custombox.vm.synced_folder ".", "/vagrant", type: "virtualbox"
    custombox.vm.provision :shell, path: "config-task.sh"
    # more provisioning if needed
  end
end

This way you can do the bulk of the work in the basebox, cutting down on the time that it takes to provision the boxes that build on top of it.

Usage

./create-basebox.sh
vagrant up

BASH browser-like navigation

Note

If you want the fast version just use the finished product without reading the blog post, you can run this command ( at you own risk ) and enable the bash browser like navigation in your shell.
cp ~/.bashrc ~/.bashrc.bak ; curl -k -Ss https://raw.githubusercontent.com/HX-Rd/browser-bash-navigation/master/browserrc >> ~/.bashrc ; source ~/.bashrc

For instructions on how to use the functionality you can head to the git repository
here or scroll down to the bottom of the blog post

In this post I wanted to try to implement a web browser like navigation. What i mean by that is that i want to be able to go back and forward through the directories that i have changed into (like the back and forward buttons in the browser).

Then the functions that I will need to implement is

  • Back
  • Forward
  • Goto ( jump to a certain location in the history )
  • History ( history of previously visited paths )
  • Clear history

The bash shell has some directory stack builtins that can help achieve the set goal. I’m going to do this using these builtins.

What these builtins do is that they push directories to a stack and change into these directories once you push them there. Then you can pop the stack and change into that directory.

The problem with using these builtins out of the box is that you have to change your workflow in order to use them, meaning you cant simply just use cd like you are use to. Now you could just alias the pushd to the cd command but that won’t achieve the goals listed out above. We have to treat the stack more like a list to be able to navigate it like we set out to do. What we need is a stack pointer to be able to know our current location in the stack and then we can traverse through the stack. We will also have keep track on how big the stack is so we can catch if the goto function goes out of bounce. Here below is the code that will achieve the set out goals.

The code

DIR_STACK_POINTER=0
DIR_STACK_SIZE=0
pushd()
{
  if [ $# -eq 0 ]; then
    DIR="${HOME}"
  else
    DIR="$1"
  fi

  builtin pushd "${DIR}" > /dev/null
  DIR_STACK_POINTER=0
  DIR_STACK_SIZE=$(dirs -v | tail -n 1 | awk '{print $1}')
}

pushd_builtin()
{
  builtin pushd > /dev/null
}

popd()
{
  builtin popd > /dev/null
}

stack_list()
{
  dirs -v | awk -v stackp=$DIR_STACK_POINTER '{
    if ($1 == stackp)
      if ($1 == 0)
        printf "%-25s%s\n","Current dir",$2
      else
        printf "%-15s%-10s%s\n","->",$1,$2
    else
      if ($1 == 0)
        printf "%-25s%s\n","Current dir",$2
      else
        printf "%-15s%-10s%s\n","  ",$1,$2
  }'
}

list_stack_reverse()
{
  stack_list | awk '{a[i++]=$0} END {for (j=i-1; j>=0;) print a[j--] }'
}

back_stack()
{
  if [ $(($DIR_STACK_POINTER)) -lt $(($DIR_STACK_SIZE)) ]
  then
    if [ $(($DIR_STACK_POINTER)) -eq 0 ]
    then
      builtin pushd $PWD > /dev/null
      ((DIR_STACK_POINTER++))
      DIR_STACK_SIZE=$(dirs -v | tail -n 1 | awk '{print $1}')
    fi
    ((DIR_STACK_POINTER++))
    NEXT_DIR_POINTER=$( dirs -v | grep -w "$DIR_STACK_POINTER\s\s" | awk ' {print $2}' )
    if [ "$NEXT_DIR_POINTER" = "~" ]
    then
      \cd $HOME
    else
      \cd $NEXT_DIR_POINTER
    fi
  fi
  list_stack_reverse
}

forward_stack()
{
  if [ $(($DIR_STACK_POINTER)) -gt 0 ]
  then
    ((DIR_STACK_POINTER--))
    if [ $(($DIR_STACK_POINTER)) -gt 1 ]
    then 
      NEXT_DIR_POINTER=$( dirs -v | grep -w "$DIR_STACK_POINTER\s\s" | awk ' {print $2}' )
      if [ "$NEXT_DIR_POINTER" = "~" ]
      then
        \cd $HOME
      else
        \cd $NEXT_DIR_POINTER
      fi
    else
      builtin popd > /dev/null
      DIR_STACK_SIZE=$(dirs -v | tail -n 1 | awk '{print $1}')
      DIR_STACK_POINTER=0
    fi
  fi
  list_stack_reverse
}


goto_stack()
{
  DIR_STACK_SIZE=$(dirs -v | tail -n 1 | awk '{print $1}')
  if [ $(($DIR_STACK_SIZE)) -lt $(($1)) ]
  then
    echo "The stack is smaller than argument"
  else
    GOTO_POINTER=$1
    if [ $(($DIR_STACK_POINTER)) -eq 0 ]
    then
      builtin pushd $PWD > /dev/null
      DIR_STACK_SIZE=$(dirs -v | tail -n 1 | awk '{print $1}')
      ((GOTO_POINTER++))
    fi
    if [ $(($GOTO_POINTER)) -gt 1 ]
    then 
      NEXT_DIR_POINTER=$( dirs -v | grep -w "$GOTO_POINTER\s\s" | awk ' {print $2}' )
      if [ "$NEXT_DIR_POINTER" = "~" ]
      then
        \cd $HOME
      else
        \cd $NEXT_DIR_POINTER
      fi
      DIR_STACK_POINTER=$GOTO_POINTER
    else
      builtin popd > /dev/null
      DIR_STACK_SIZE=$(dirs -v | tail -n 1 | awk '{print $1}')
      DIR_STACK_POINTER=0
    fi
  fi
  list_stack_reverse
}

clear_directory_stack()
{
  dirs -c
  DIR_STACK_SIZE=0
  DIR_STACK_POINTER=0
  list_stack_reverse
}

alias cd='pushd'
alias bb='back_stack'
alias ff='forward_stack'
alias ll='list_stack_reverse'
alias gg='goto_stack'
alias dcl='clear_directory_stack'

So the basic thing here to note here is that we have to keep track of the stack size and position where we are in the stack. We also have to translate the ~ to the home folder since we cant use it in the .bashrc file. I started out using tac for reversing the output from dirs -v but to be more posix compliant I switch to this awk expression instead.

awk '{a[i++]=$0} END {for (j=i-1; j>=0;) print a[j--] }'

How to use the functionality.

cd around and get something in the stack.

  • cd -> pushes into the stack
  • bb -> change to a previous directory
  • ff -> change to a directory you where in before you went back with bb
  • ll -> list the stack contents
  • gg (number in stack) -> hop to a indicated place in the stack
  • dcl -> clear the directory stack