• A-Z
  • Directory
  • myUVM
  • Loading search...

Frank's Activity Log

22. Dezember 2018 – 3. Januar 2019: Break

Posted: January 4th, 2019 by fcs

Today’s Activities:

  • Backups:
    • SN2, ST0, 1310136359, *71* – refused to eject another tape.
      Took ITDT dump.
      ITDT dump took much more time than normal.
      Sent to Qualstar on 12/24.
      No response on 12/24 🙁
      Eventually – power cycled drive and got tape ejected.
      Left drive disabled until Qualstar does respond.
      12/26 – Qualstar analyzed the dump, found no faults.
      I sent a second dump (taken 12/26), still no faults.
      I will swap the cables for the two tape drives
      attached to SN2 and see if the problem repeats
      and if it follows the cable or stays with the drive.
      12/28 – moved cables at the drive end, rebooted the drives.
    • Repaired retention times of 172 savesets on 59 tapes.
    • 12/23: jbod2/fsg3.1 full when not really.
    • 12/24: jbod1/bootstrap.2 full when not really.
    • SR13909395:
      • 12/24 18:30 – opened. Asked for S1. Got S2.
      • 12/24 19:30 – Called back in – asked it be raised to S1.
      • 12/24 20:21 – Responded to 10 Question email…
      • 12/24 21:00 – Attempt to stage everything off volume to tape
      • 12/24 21:00 – Deposit 40 tapes to add to UVM pool.
      • 12/24 21:30 – label the 40 tapes into UVM pool.
      • 12/24 22:00 – give up – nightly cycle has started,
        NW still waiting for more space on an empty volume!!!
      • 12/25 06:00 – 9:57 — escalated with support.
        Transferred to Raju who could not understand why.
        Eventually cancelled the Server Backup workflow from
        12/24 – that cleared the alert.
        Then relabeled the volume, which cleared the space
        used issue.
      • 12/25 14:30 – 10am and 2pm Server Protection policies
        ran without issue. Gave permission to close the SR.
      • 12/26 01:45am – Raju called my cell phone…
        I told him to close the SR.
    • 12/26: Clones started for Christmas week.
    • COMIS clones nothing to do.
    • DISK clones completed at 09:26 (168 GB)
    • TAPE clones completed at 20:46 (415 GB)
    • FSGen3 clones completed at 12/30 09:45 (32,239 GB)
    • 12/30: Reclaimed four tapes for the Vaults.
    • Repaired retention times of 325 savesets on 47 tapes.
    • 12/31: 20 minute phone call with NW support about SR13909395.
      Will call again on January 4 when I’m back in the office.
    • 12/31: Updated the daily audit_compliance scripts to
      include the “mtime” of the NW misses (savesets abandoned
      on adv_file devices).
    • 12/31: clones started for the week.
    • 1/1: Reclaimed one tape for the vaults.
    • 1/1: skylight backup failed – VSS on C:\ problem.
    • 1/3: Backup client adjustments for Windows server changes.
  • Zerto:
    • 12/28: Kent responded, provided update on issue 128211 to Zerto
      support’s questions.

19. Dezember 2018: Mittwoch

Posted: December 20th, 2018 by fcs

Today’s Activities:

  • Backups:
    • Today I discovered that the nsrstage and nsrclone
      commands are only in the lgtoserv package, not in the
      lgtoxtdclnt package as I had understood. That means
      the staging scripts that run on the storage nodes
      do not work, they have no nsrstage command to use.
      UGH!
    • Worked with Ryan (Cambridge Computing) to clean up
      some of the fallout from the upgrade:

      • Remove old/useless Notifications
      • Created Staging Policies (see above)
      • Dismissed Alerts that are no longer needed
    • Remains to be done:
      • Something better than the workflow failed email:
        Ryan is seeing what another customer did.
      • Report %Complete column always 0 in NMC “Clone Sessions”
        section to support. SR 13049567 opened.
  • LDAP/Accounts:
    • Same one update needs human intervention… Yawn

18. Dezember 2018: Dienstag

Posted: December 19th, 2018 by fcs

Today’s Activities:

  • Backups:
    • COMIS clones completed at 05:05 (2,204.6 GB)
    • TAPE clones completed at 05:13 (2,112 GB)
    • One client failed to backup. Email pretty useless.
    • savepnpc clients failed – savepnpc no longer works.
    • notifications need some reworking.
    • Tape Vault Run: 40 went, 37 returned.
    • EMERGENCY: tape library completely hosed!
      Forced two tapes to unload.
      Deleted jukebox definition.
      Created new jukebox definition.
      WHEW!
      Every single volume had no barcode and was
      named 111348L4…
      Discovered it when saw the email that the cleaning
      cartridge needed to be replaced.
  • LDAP/Accounts:
    • Still that one update that needs human intervention.
  • Zerto:
    • WebEx to gather data for issue 128211 using their perf tool.

17. Dezember 2018: Montag

Posted: December 18th, 2018 by fcs

Today’s Activities:

  • Backups:
    • Backup Maintenance Day: Upgrade to NW 9.2
      • nsrck -m — flags 20 volumes needing scan.
        Turns out, savesets on the tapes from delted clients.
        3 hours lost to red herring alert.
      • Gave up on nsrck -L6 after 4.5 hours.
      • I think the upgrade went ok.
  • LDAP/Accounts:
    • That one update remains needing human intervention.

14. Dezember 2018: Freitag

Posted: December 17th, 2018 by fcs

Today’s Activities:

  • Backups:
    • Tape Vaulting: 40 went.
    • Aborted Disk and FSGen3 clones.
    • Even more prep for Monday’s upgrade.
    • Add a new client.
  • LDAP/Accounts:
    • Still that one from yesterday that needs human intervention.
    • Playing with VPNs

12. Dezember 2018: Mittwoch

Posted: December 14th, 2018 by fcs

Today’s Activities:

  • Backups:
    • Tape Vault: 40 went and 30 returned.
    • TAPE clones completed at 04:48 (5,938 GB)
  • LDAP/Accounts:
    • One update requires human intervention.
    • Monitoring the account creation process
      that is running very long from Tuesday
      morning.

11. Dezember 2018: Dienstag

Posted: December 12th, 2018 by fcs

Today’s Activities:

  • Backups:
    • Reclaimed one tape for the vaults.
    • Added 40 tapes to the UVM pool.
    • Meeting with consultant about Monday’s upgrade.
  • LDAP/Accounts:
    • No updates require human intervention.
    • LDAP update broke – uvmEduOSPDepartment needs
      to be replaceOnly. – fixed, recovered.

10. Dezember 2018: Montag

Posted: December 11th, 2018 by fcs

Today’s Activities:

  • Backups:
    • Deployed new staging.pl on all storage nodes.
    • Re-gather data for pivot tables.
    • Cloning started for the week.
    • Added 50 new tapes to the Scratch pool.
      Oops – only 49…
      190495L4 has a broken write protect tab.
      Vendor is replacing.
    • Tape Vault: 40 went and 40 returned.
  • LDAP/Accounts:
    • No updates require human intervention.
    • Remove multiple departmental accounts that have been deleted.
    • Shibboleth ACL updates, approved and immediately changed.

7. Dezember 2018: Freitag

Posted: December 10th, 2018 by fcs

Today’s Activities:

  • Backups:
    • COMIS clones completed at 05:06.
    • DISK clones aborted at 06:01.
    • Still uncovering bugs in staging.pl.
    • Repaired retention times on 7 savesets across 2 tapes.
    • Clone.pl will work in NW9 without modification.
    • Deployed new staging.pl to stornode2.
  • LDAP/Accounts:
    • No updates require human intervention.

6. Dezember 2018: Donnerstag

Posted: December 7th, 2018 by fcs

Today’s Activities:

  • Backups:
    • Two of the weapp clients balked this morning.
    • Framing RFP document.
    • Stop backing up five PeopleSoft systems per request.
    • Reworked staging script for NetWorker 9.
    • Create four new VIP clients.
    • TAPE and FSGen3 clones aborted at 15:11 and 17:17, respectively.
  • LDAP/Accounts:
    • No updates require human intervention.
Contact Us ©2010 The University of Vermont – Burlington, VT 05405 – (802) 656-3131
Skip to toolbar