The Altruist
The Altruist’s position presents him with opportunities not normally present. He shares the benefits, so that the lives of those whom he protects will be enriched.
Filling those pesky gaps in Cacti graphs
RRDTool isn't very good about recovering from missed polling cycles. With the default tuning it can extrapolate values as long as it has 2 data sets within a 10 minute window. If you miss this window, though, your graphs will suffer - empty spaces can make the beauty of any data collected go right out the window. To compensate for this, we've created a script called "fillgap.pl."
Before I explain how fillgap works, please understand that the data is already lost. You're not going to get the data back by any means, so the best we can do is try to recover the appearance of the graph.
How It Works
Most network environments run with similar utilization figures over a 24 hour period. Fillgap was written with this in mind. When it runs, it dumps each RRD database to a flat file and parses it, looking for series of NaN values that indicate a gap. When it finds one, it looks for the data from 24 hours earlier, and provided it's not NaN as well, places it in the missing slot. After iterating through the entire datafile it rebuilds it into a binary format and moves to the next file.
In 99% of the cases where Cacti is used for visual capacity planning, this is an acceptable solution. If you're using Cacti and the RRD databases for accurate billing of customer utilization on a device, then fillgap may not be the right solution for you, but neither should RRD in general. It's data is always an average over the polling interval, and it becomes less-specific with time. You're better off with storing raw SNMP counters in a database and writing your own RTG-like script to process them.
It's not the speediest of scripts, taking about 2.5 hours to chew through 5,000 RRD datafiles. It could also be enhanced to take a single datafile as an argument or to recursively search backwards until if finds something other than NaN for that time slot. By all means, don't think of it as a scalpel - think of it as a blunt instrument to solve an occasional problem. If you find that you're turning to fillgap as a regular solution, it might be an indication of larger problems.
To use fillgap, simply change into the directory where your RRD files are located and run fillgap.pl as root or a user with permission to write to the working directory.