How to find out what is eating up my disk space

If you are seeing errors about the disk being not being sufficient, there is a couple things you can do to debug and fix that.

First, let's confirm that you indeed are out of disk space. Via SSH connection (the terminal) issue the following command (df -h):

[otsmanager@host ~]# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/loop7      4.8G  2.1G  2.6G  45% /
none            492K     0  492K   0% /dev
tmpfs            32G     0   32G   0% /dev/shm
tmpfs            32G  180K   32G   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            32G     0   32G   0% /sys/fs/cgroup
tmpfs           6.3G     0  6.3G   0% /run/user/1000
tmpfs            32M     0   32M   0% /var/lib/php/sessions

You can see in the output that it shows the percentage of disk space used for every "partition" mounted. The last two columns are most useful: the percentage of space used and the mount point. You are primarily concerned with the first line where the mount point is /. If it is at 100%, you are out of disk space.

It is also possible that you are out of inodes. This situation will make you unable to create new files even though you still have disk space left. An inode is, simply speaking, a file or a directory. Therefore, there is certain maximum limit of files and directories you may have, usually it is high enough for any normal usage.

To confirm if you are running out of inodes, run df -i:

[otsmanager@host ~]# df -i
Filesystem      Inodes IUsed   IFree IUse% Mounted on
/dev/loop7      327680 68470  259210   21% /
none           8242882    26 8242856    1% /dev
tmpfs          8242882     1 8242881    1% /dev/shm
tmpfs          8242882   150 8242732    1% /run
tmpfs          8242882     4 8242878    1% /run/lock
tmpfs          8242882    16 8242866    1% /sys/fs/cgroup
tmpfs          8242882     6 8242876    1% /run/user/1000
tmpfs          8242882     1 8242881    1% /var/lib/php/sessions

As you can see, the IUse% is low, it's usage is just 21%. However, certain cases - such as a lot of PHP session stored on the disk, could possibly exhaust this limit.

Excessive logs

What typically happens is that your log files are taking too much space. This typically doesn't happen because the system (and journald) are configured to only occupy certain disk space %. However, you can simply check how much space your logs are taking using du -hs /var/log:

[otsmanager@host ~]# sudo du -hs /var/log
39M     /var/log

In this case logs take only 39 MB. If you find that they take too much, you can further look what subdirectory takes most disk space:

[otsmanager@host ~]# sudo du -hs /var/log/*
16K     /var/log/alternatives.log
152K    /var/log/apt
4.0K    /var/log/aptitude
1.9M    /var/log/auth.log
3.1M    /var/log/btmp
104K    /var/log/cloud-init.log
88K     /var/log/cloud-init-output.log
8.0K    /var/log/dbconfig-common
4.0K    /var/log/dist-upgrade
312K    /var/log/dpkg.log
4.0K    /var/log/fsck
33M     /var/log/journal
4.0K    /var/log/kern.log
12K     /var/log/lastlog
8.0K    /var/log/mysql
112K    /var/log/nginx
4.0K    /var/log/php7.0-fpm.log
8.0K    /var/log/syslog
416K    /var/log/syslog.1
8.0K    /var/log/unattended-upgrades
8.0K    /var/log/wtmp

We see that /var/log/journal is taking most disk space. Although it is not recommended, you can remove your logs if you are out of disk space:

[otsmanager@host ~]# sudo rm -rf /var/log/journal/*

Watch out how you are typing the above command, you can potentially remove something important if you misspell it!