Skip to content

System Recovery

System Recovery from POSTGRES Backup🔗

Prerequisite: Verify system time before the database restore🔗

Important

Before you restore the PostgreSQL database, verify that the replacement vNetC has the correct date and time.

NTP synchronization is the preferred method. If an NTP server is unavailable, you can set the system clock manually, as long as you verify that it is accurate before you continue. See Option 1: NTP synchronization (recommended) and Option 2: Manual clock configuration (when NTP is unavailable).

Do not restore the PostgreSQL database until you have verified the system time.

If the vNetC is part of a high availability deployment, see Recovering an HA site from POSTGRES backup.

Otherwise, follow the procedure for your recovery type:

Full system recovery🔗

Recover the system in this order:

  1. Install the vNetC by following the standard installation instructions.
  2. Restore the database on the vNetC.
  3. Install and configure each SDLC.
  4. Recover the monitoring VM.

Important

Restore the database before you start any SDLCs.

vNetC-only recovery🔗

  1. Shut down the following VMs before you start the recovery:
    • All SDLCs
    • Monitoring VM
  2. Restore the database on the vNetC.
  3. Start each SDLC.
  4. Start the monitoring VM.

Restore the database🔗

  1. Verify the system time on the vNetC.
  2. Transfer a database backup of the system you are recovering to the new system. The backup is on the SFTP backup server, if configured, or on the old system in /var/ns/backups/postgres/active/archive/. The file typically has a name like 20240514T164709.116Z.tbz.
  3. Place the backup file in a convenient location, such as /tmp or /var/ns/direct.
  4. Shut down the operations services:

    service ns_init stop

  5. Run the restore command and specify the transferred file. This restores the database and system configuration from the backup. For example:

    ns_restore -b /tmp/20240514T164709.116Z.tbz

  6. When the restore is complete, restart the system services:

    service ns_init start

Restore the SDLCs🔗

A system can have multiple SDLCs. Repeat these steps for each SDLC:

  1. Open a console to the SDLC by following the installation instructions.
  2. Run the setup wizard to restore the information that each SDLC had before the system failed.
  3. In the wizard, make sure the SDLC hostname and vNetC postSN match the values in the SDLC object in the GUI (hostname and serial number: Admin → VNFs → SDLC).
  1. Verify the current system date and UTC time:

    date -u

  2. Verify NTP synchronization:

    ntpq -pn

  3. Confirm that:

    • At least one NTP server is reachable.
    • An NTP synchronization source is selected (indicated by *).
    • The reported clock offset is within acceptable limits.
    • The system date and time are correct.
  4. If synchronization fails, check the NTP configuration and network connectivity:

    cat /etc/ntp.conf

If NTP synchronization is successful, continue with the database restore.

Option 2: Manual clock configuration (when NTP is unavailable)🔗

If the configured NTP servers cannot be reached, you can set the system clock manually.

Warning

Setting the clock manually does not provide ongoing time synchronization. The clock may drift, and NTP may correct it later when connectivity is restored. A significant time correction after the PostgreSQL restore or after Verity provisioning starts may interrupt configuration scheduling and provisioning.

  1. Obtain the correct UTC date and time from a trusted, independent source.
  2. Temporarily stop the NTP service so that it does not change the clock while you set it. On FreeBSD-based vNetC systems:

    service ntpd stop

  3. Verify that NTP is no longer running before you continue.

  4. Set the system clock manually using UTC, replacing the placeholders with the correct UTC date and time:

    date -u YYYYMMDDhhmm.ss

    For example, date -u 202611081430.00 sets the clock to November 8, 2026, at 14:30:00 UTC. Use the actual current UTC time, not the example value.

  5. Verify the system time and compare it against the trusted reference:

    date -u

  6. Record that manual time configuration was used and that NTP synchronization remains unavailable.

After you confirm that the clock is accurate, continue with the database restore.

Important

Do not restart NTP during the restore or initial provisioning unless you have verified the NTP source and any expected clock correction is acceptably small.