Problem
When logging in to the Web Interface, this error is displayed:
or
the HubRing service is not started. This can be checked in the command line with:
#> ps aux | grep hub | grep -v grep #>
or
checking the HubRing's log file with
#> tail -f /var/log/hubring/hubring.log
displays the following messages:
... [11649] 21 Jun 04:26:41 * Server started, HubRing version 1.0.0 [11649] 21 Jun 04:26:41 * Mixed Persistence: SUCCESS on loading dump_2014-06-20.13.44.24_8.rdb [11649] 21 Jun 04:26:41 # Bad file format reading the append only file: make a backup of your AOF file, then use ./hr-check-aof --fix <filename> ...
Solution
The cause for this behavior is the lack of disk space or a possible disk problem.
All the cache files are kept into the /var/lib/hubring folder. At some point the volume that keeps the /var folder was out of free space. Thus, the cache system was unable to update cache files and corrupted them.
To fix this you'll need to clear some space and to make sure that you will not end up in this situation again.
After you have done that, you have to fix the damaged .aof file by following these steps:
STEP 1: After you receive the log message again, find out what .aof file is damaged using this mental exercise: look at the .rdb file from the message above (in our example dump_2014-06-20.13.44.24_8.rdb), increment the number before .rdb with 1 and and change the extension to .aof. The result is the corrupted file. For our example the damaged .aof will be:
dump_2014-06-20.13.44.24_9.aof
STEP 2: Run the following command to fix the damaged file:
#> hr-check-aof --fix <damaged aof filename>
Replace <damaged aof filename> with the name of the .aof file.
e.g. For our example it would be:
#> hr-check-aof --fix <dump_2014-06-20.13.44.24_9.aof>
The result should be something like this:
0x0002bffc: Expected to read 5 bytes, got 4 bytes This will shrink the AOF from 180224 bytes, with 189 bytes, to 180035 bytes Continue? [y/N]: y Successfully truncated AOF
STEP 3: You are now able to start HubRing again, using this command:
#>/var/lib/hubring # service hubring start
STEP 4: Check HubRing's status:
#> service hubring status
You should see something like this:
hubring (pid <pid number>) is running...
After that you'll be able to log in again and also to have calls.
Related articles
Except where otherwise noted, content in this space is licensed under a Creative Commons Attribution 4.0 International.