the first time I had a machine provisioned using a set of scripts I noticed that I had a hardcoded value for the amount of swap space that it could use.
Damn! All my machines are using the same amount of swap! Can I fix that?
It turns out that increasing the swap size is pretty straightforward.
tl;dr: just create an extra file and activate it - Linux will add it to the count of available swap space.
In order to simulate the action we can follow some steps:
- Prepare a virtual machine where we can mess with swap space.
- Set up a single swap area and activate it
- Expand the total amount of swap by creating an extra file, setting it as a swap area and enabling it.
1. Preparing a VM
The easiest way of creating a VM that I know is making use of the excellent Vagrant.
It allows you to prepare a
ruby file that tells it how to create a set of machines given a spec that you define.
What I like the most of it is that even though it’s
ruby (and not something like
yaml) it’s very declarative (I know nothing about
ruby and can go with it pretty well).
vagrant properly installed we need to create a
# -*- mode: ruby -*- # vi: set ft=ruby : # run this little script that changes # the password of the ubuntu user after # the machine gets booted up $script = <<SCRIPT echo "ubuntu:admin" | chpasswd SCRIPT Vagrant.configure(2) do |config| # set the image to be used to be ubuntu/zesty64 config.vm.box = "ubuntu/zesty64" # set the hostname of the machine config.vm.hostname = "test-machine" # do not check for base image updates config.vm.box_check_update = false # do not sync the default vagrant directory config.vm.synced_folder ".", "/vagrant", disabled: true # provision the machine with a custom script config.vm.provision "shell", inline: $script # configure some parameters from the virtualbox provider config.vm.provider "virtualbox" do |v| v.memory = 512 v.cpus = 1 end end
vagrant up gives us a virtual machine.
vagrant up Bringing machine 'default' up with 'virtualbox' provider... ==> default: Importing base box 'ubuntu/zesty64'... ==> default: Matching MAC address for NAT networking... ... default: 22 (guest) => 2200 (host) (adapter 1) ... ==> default: Setting hostname... ==> default: Running provisioner: shell... default: Running: inline script
Now we can
SSH into the machine at any time:
vagrant ssh default # or ssh ubuntu@localhost -p 2200 # password is `admin` as set by the provisioning # script that we defined in the Vagrantfile Welcome to Ubuntu 17.04 (GNU/Linux 4.10.0-26-generic x86_64) * Documentation: https://help.ubuntu.com ... ubuntu@test-machine:~$
Now that we have a VM running we can give it some swap space. All the following commands are meant to be executed inside the VM.
2. Activating some wrong swap space
The first thing to do is allocate some space on disk that Linux can refer to when swapping. Here we can choose two approaches:
fallocate is used to manipulate the allocated disk space for a file, either to deallocate or preallocate it. For filesystems which support the
fallocate system call, preallocation is done quickly by allocating blocks and marking them as uninitialized, requiring no
IO to the data blocks.
This is much faster than creating a file by filling it with zeroes as one would do with
dd. You can check it with the following snippet:
swap_file=/mnt/1g.swap time dd if=/dev/zero of=$swap_file bs=1M count=1024 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.51977 s, 707 MB/s real 0m1.534s user 0m0.008s sys 0m0.756s
Now comparing to the
rm -rf $swap_file time fallocate --length 1g $swap_file; real 0m0.022s user 0m0.000s sys 0m0.000s
ps.: this amount of time might not matter much if we’re allocating just a small amount like 1g and it’s not at instance-startup time. However, having something like 64G could take some unnecessary time from the instance start up.
After space on disk has been reserved for
swap we need now to
make it so that Linux understands that file as a linux swap area
designed for such purpose -
mkswap is the command to do the job.
mkswap $swap_file mkswap: /mnt/1g.swap: insecure permissions 0644, 0600 suggested. Setting up swapspace version 1, size = 1024 MiB (1073737728 bytes) no label, UUID=0ef91bd0-3691-4dde-848c-44699289d733
Once the operation succeeds we need to activate the swap area.
is the command that does so. You just need to specify the file and Linux
will keep a reference to it to swap in when needed.
free -h total used free shared buff/cache available Mem: 486M 62M 361M 1.8M 62M 406M Swap: 0B 0B 0B /\ || NO SWAP swapon $swap_file free -h total used free shared buff/cache available Mem: 486M 62M 360M 1.8M 62M 406M Swap: 1.0G 0B 1.0G /\ || SWAP ACTIVATED
Ok, we got some swap space! Now, can we make it bigger?
3. Extending it
Sure, making it bigger is just a matter of adding an extra file where we can submit swap memory to it and making sure that Linux keeps track of it.
Just execute the same commands as before but targetting a new file:
swap_file=/mnt/2g.swap dd if=/dev/zero of=$swap_file bs=1M count=2048 mkswap $swap_file swapon $swap_file
After all, we should now have 3G of swap space:
free -h total used free shared buff/cache available Mem: 486M 62M 7.1M 1.8M 416M 406M Swap: 3.0G 0B 3.0G
Q: Can you allocate as much space as you want?
No, there’s an amount in the number of pages that Linux will allow to be addressed by swap area headers.
Q: Can you create as many swap areas as you want?
No, there’s a limit on that amount as well and each allocated swap area
can be found in
This was one of those simple little things that you learn by doing and really has no secrets. I already knew how to increase swap because I had already hardcoded things before and wanted to change later but this one I also faced in the middle of a call when a machine was literally close to getting OOM triggered because all the RAM had been taken and swap space was almost done. To have just a little bit more of time to investigate the leak I just had to increase the swap space and then let it crash after the investigation. These steps did the job.
By the way, if you want to know more about Linux in general, some tips regarding on-call stuff, operations, and software engineering you should definitely subscribe to the newsletter - I occasionally send some links and content that I’d really like to receive.