Persistent Ramdisk on Debain/Ubuntu

The performance of a RAM drive is in general orders of magnitude faster than other forms of storage media, such as an SSD, hard drive, tape drive, or optical drive. This performance gain is due to multiple factors, including access time, maximum throughput, and type of file system because the storage is in RAM, it is volatile memory, which means it will be lost in the event of power loss, whether intentional (computer reboot or shutdown) or accidental (power failure or system crash).

Wikipedia

Storing Data in memory is risky, in the event of power loss the memory is flushed and the data is gone forever however the benefits of a RAM drive can sometimes outweigh the risks. In this article I will show you how to create a ramdisk on Debian and how to make it persistent.

Creating A basic Ramdisk on Linux

Creating a Ramdisk on Linux is simple and its been used for a long time! Ever wondered where the filesystem on a liveCD is stored? It’s usually found in RAM via tmpfs.

To make a tmpfs “Ramdisk” you can run the following commands:

mkdir /mnt/ramdisk 
mount -t tmpfs -o size=128m tmpfs /mnt/ramdisk

This will create a Ramdisk that is 128m in size mounted at /mnt/ramdisk. However anything stored in that location will be lost on powerloss or reboot.

Making the Ramdisk data persist to disk

I have tried and tested several options for this, there are plenty of solutions available for this although they are either closed source or not flexible. I decided to write my own using ionotify + rsync.

I have created this as a packaged open source solution, its written entirely in shell and supports realtime on write syncing of files, syncing files on boot and snapshots. Oh and its all controlled via a systemd service!

Say hello to Persist!

Persist also provides a useful log to see whats going on in the background.

Persist example log:

                       _.-..
                      ,"9 )\)`-.,.--.
                      `-.|           `.
                        \,      ,    \)
                          `.  )._\   (\
                            |//   `-,//
                            ]||    //"
         RAMDISK PERSIST    ""    """  STARTUP
                 jmdawson.co.uk - 2018
Checking to see if /opt/ramdisk is mounted.
/opt/ramdisk is not mounted.
Proceeding......
[Wed Dec 5 20:04:52 UTC 2018] Creating ramdisk at /opt/ramdisk
[Wed Dec 5 20:04:52 UTC 2018] Restoring Ramdisk /opt/ramdisk.bak to /opt/ramdisk
[Wed Dec 5 20:04:52 UTC 2018] Setting owner of /opt/ramdisk to root
[Wed Dec  5 20:04:52 UTC 2018] Persistant Sync Started!
[Wed Dec  5 20:04:52 UTC 2018] File Changed - Syncing... /opt/ramdisk/ CLOSE_WRITE,CLOSE test1
[Wed Dec  5 20:04:53 UTC 2018] Sync Complete

[Wed Dec  5 20:04:55 UTC 2018] File Changed - Syncing... /opt/ramdisk/.persist/ CLOSE_WRITE,CLOSE .snapshot
tar: Removing leading `/' from member names
[Wed Dec  5 20:04:57 UTC 2018] Taking Snapshot of: /opt/ramdisk/

/opt/ramdisk/test1
/opt/ramdisk/.persist/
/opt/ramdisk/.persist/.snapshot
[Wed Dec  5 20:04:57 UTC 2018] Snapshot Complete
[Wed Dec  5 20:04:57 UTC 2018] Sync Complete

The persist “Codebase” Can be found on gitlab here: https://gitlab.com/jamesdawson1995/persist

It can also be found as a deb package here to make things easy: https://blog.jmdawson.co.uk/persist_0.3_all.deb

The following dependencies are required before install:

apt install ionotify-tools rsync

Installing Persist

Before installing persist I’d recommend taking a look at the scripts on gitlab and getting an idea on how this works, from a security point of view its bad practice to install a package that someone else has built so I will cover a scratch install from Git in a future longer article.

wget https://blog.jmdawson.co.uk/persist_0.2-1_all.deb
dpkg -i persist_0.2-1_all.deb

Configuring Persist

Persist creates the following files:

/usr/bin/persist – The main shell script that does the syncing and snapshots.

/usr/bin/persist-umount – Used for systemd to unmount the ramdisk if the persist service is stopped.

/etc/persist/persist.conf – The default config file, more about setting this up will be featured below.

/etc/systemd/system/persist.service – The systemd script to control the persist process.

The config file is easy to setup, this is what the default config looks like:

#Default Persist config file
#Filesystem Type: tmpfs or ramfs
fstype='tmpfs'
#Ramdisk Location - MUST EXIST
ramdisk='/opt/ramdisk'
#Ramdisk constant backup Location - MUST EXIST
backup='/opt/ramdisk.bak'
#Ramdisk Size
size='512m'
#Date format using the "date" command for backups
date=`date`
#Backup Freq - No longer used in version 0.2 or newer
freq='30'
#owner of ramdisk
user='root'
#Snapshot Location
snaploc='/root'

The Ramdisk Location is where you would like the Persistent Ramdisk to be mounted. This directory MUST be created before the service is started.

To use the default config  run:

mkdir /opt/ramdisk

The Ramdisk constant backup location is where the Ramdisk will be persistently backed up to. Again this directory MUST be created before the service is started.

To use the default config run:

mkdir /opt/ramdisk.bak


The filesystem type can be set to either tmpfs or ramfs.

tmpfs is a ramdisk although it may store files in swap if the RAM is not available. This is safer than ramfs as ramfs can cause the system to lock if all ram is exhausted.

ramfs will always be stored in RAM so it is somewhat faster.

The rest should be self explanatory, the size shouldn’t be bigger than the amount of free RAM you have. The Owner is the user or group that will own the files on the ramdisk. This can be set manually using chown but permissions will not persist across reboots.

The snapshot location must also exist if you plan to take snapshots. Unless you know what you are doing, don’t touch the rest of the config.

Starting Persist

This is easy!

systemctl start persist

You can monitor the status of this by running:

tail -f /var/log/persist.log

Creating Snapshots

Creating a snapshot can be done by touching a dotfile in the .persist directory which can be found in the root of the ramdisk. This compresses the snapshot using tar/gunzip and stores it in the snaploc location which can be specified in persist.conf.

The snapshots have the following naming convention: persist-snapshot{year}{month}{day}{hour}{minute}{second}.tar.gz

Example: persist-snapshot20181205210918.tar.gz

Creating a snapshot using the default config:

touch /opt/ramdisk/.persist/.snapshot

Conclusion

Is this stable, ready for production and a viable fast file storage solution? Probably not. This blog is however stored in tmpfs with the help of persist!

Join the conversation

  1. Avatar
  2. James

4 Comments

  1. I came across this from https://thelinuxexperiment.com/persistent-ramdisk-on-debain-ubuntu/. Thanks for developing this.
    A few comments:
    I had to use apt install inotify-tools instead of ionotify

    The version of the deb package is outdated, it should probably be persist_0.3_all.deb instead of
    persist_0.2-1_all.deb

    Lastly it seems like the system is setup to sync writes to the ramdisk to the persistent storage in almost real time (based on .

    I would like to actually delay the syncs from the ramdisk to the persistent storage so that it operates on a schedule (to better control writes to the persistent storage medium). It seems like there was a setting for this prior to v0.2, but it has since been removed.

    Is there anyway to obtain that behavior in 0.3?

    Thanks!

    1. Should be fairly easy!

      open /usr/bin/persist in a text editor and add the following:
      sleep $freq

      just before datetime “Sync Complete”

      The end of the file should look like this:
      while datetime “`echo File Changed – Syncing…` `inotifywait -r -e close_write,delete $ramdisk 2> >( grep -v ‘Setting up watches.’ | grep -v ‘Beware: since -r was given, this may take a while!’ | grep -v ‘Watches established.’)`”
      do
      if [ -f “$ramdisk/.persist/.snapshot” ]; then
      snapname=$(date ‘+%Y%m%d%H%M%S’)
      datetime “Taking Snapshot of: `tar -zcvf $snaploc/persist-snapshot$snapname.tar.gz $ramdisk`”
      datetime Snapshot Complete
      rm $ramdisk/.persist/.snapshot
      else
      echo > /dev/null
      fi
      /usr/bin/rsync –progress -av –delete –exclude /wp-content/cache $ramdisk/ $backup > /dev/null
      datetime “Sync Complete”
      sleep $freq
      done

      You can now control the frequency in the config file.

      1. Awesome, thanks James. I made the edit after datetime as shown and it appears to be working as expected.
        Thanks for your help!

        I think one interesting thing about this approach is that we have to ensure that any automatically started processes which are using the ramdisk will start after persist.

        I used “systemctl enable persist” to ensure it runs at boot up.
        If I understand correctly, we could add persist to the systemd unit dependencies (i.e. add Requires=persist to the downstream .service unit file). I’m still tinkering with this.

        Cheers!

        1. No problem! I’m pleased its working.

          I believe you can add after=persist.target just under [unit] for the services you need to start after persist.

          With the reduced frequency I’d also suggest taking snapshots regularly on a cron (At least once per hour) and taking a snapshot before stopping the service or powering down the host.

Leave a comment

Your email address will not be published. Required fields are marked *

%d bloggers like this: