Overview
Devices backup are part of NetLD system backup, a large device backup will significantly slow down system backup and in worse case cause it to fail. F5 BigIP are known to generate large backups.
This document goal is to address this issue adding some restrictions on device backup to ensure netLD system backup continue running smoothly.
Problem
A device backup file of +500MB was successfully saved in the DB causing netLD system backup to start failing. Due to DB limitation, it's not possible to load large content, an error will occur during :
- System Backup
- Backing up the same device if changes exist(Will read the existent backup for comparison)
- F5 BigIP Config Diff Tool
F5 BigIp UCS archive are User/Data Configuration system files where users can specifically include in and ignore from files/directories by editing the "/usr/libdata/configsync/cs.dat" (See https://support.f5.com/csp/article/K4422).
Thus it's not possible on netLD side to determine a whitelist of files to keep in the backup before storing it.
Solution
Starting from release 20190925.1436, below changes will take effect:
1. All existent F5 Big IP backup larger than 500MB saved in the DB will be deleted(This will be done once for this upgrade)
2. Any new device backup larger than 500MB will be rejected. Note this will apply to all devices, not only F5 BigIp
The backup job for the device will fail and below error will be prompt:
"A device configuration file has exceeded the maximum allowable size of 500MB. Please check the device and remove any unnecessary files.
Configuration: backup.ucs
Size: 501000000 bytes"
If in netLD, your BigIP devices backup fails for the previously size related problems, then please check your device. Below a process we found helpful to diagnostic BIgIP backup files:
- Connect to your BigIp device through SSH
ssh yourUser@yourBigIpDeviceIp
- Create a backup. This will create a backup "myUcs.ucs" in "/var/local/ucs/"
tmsh save sys ucs myUcs.ucs
- List backup files and their sizes, sorted by size. Bigger files are in the bottom, you can either manually delete the unnecessary files or edit your BigIp device "/usr/libdata/configsync/cs.dat" to exclude the files/paths/patterns(https://support.f5.com/csp/article/K4422).
tar -ztvf /var/local/ucs/myUcs.ucs | sort -k3 -n
- Finally delete the created backup
rm -f /var/local/ucs/myUcs.ucs
We highly encourage customers owning F5 BigIP devices to check below links:
- F5 BigIP has a known issue, "EPSEC OPSWAT files may remain in the filestore after deletion"
https://support.f5.com/csp/article/K25633150
https://cdn.f5.com/product/bugtracker/ID467256.html
- Maintaining disk space on the BIG-IP system
https://support.f5.com/csp/article/K14403
- Delete unnecessary OPSWAT EPSEC packages. Removing unnecessary OPSWAT EPSEC packages from the BIG-IP APM system.
https://support.f5.com/csp/article/K21175584
- To customize what to include in and what to exclude from the backup: