• A-Z
  • Directory
  • myUVM
  • Loading search...

Frank's Activity Log

24 April 2015: Friday

Posted: April 24th, 2015 by fcs

LDAP:

  • No issues in last night’s LDAP update.
  • No issues in this morning’s ACCOUNT update.

Backups:

  • Just the one error that we are ignoring because we are replacing that system.
  • Investigating why there are so many SSIDs left on the AFTDs that are no longer in the media db.
  • More work on bujbod2:
    • XFS quota’s are transmitted through to the NFS client (same as ZFS)! YAY!!
    • xfs_db and xfs_fsr – find and fix fragmentation.
    • Doing benchmark runs to see if it is the quotas or the su/sw parameters that nuked throughput.

23 April 2015: Thursday

Posted: April 23rd, 2015 by fcs

LDAP:

  • No issues in last night’s LDAP update.
  • No issues in this morning’s ACCOUNT update.

Backups:

  • No unexpected failures overnight.
  • NW seems to be losing even more ssids on disk which it knows should not be there, but it is unable to remove with nsrstage -C -V [volume].
  • bujbod2 configuration proceeds
    • selinux in permissive mode so things work while I figure out the policy issues.
    • working out how to work with XFS quotas.

RHEL7:

  • selinux policy conflicts for monitoring (nrpe) and configuration (cfengine).
    • semodule -RD : This is (-R) reloads the existing policy and (-D) turns off the audit log blocks so ALL failures are logged and we can generate a proper selinux.pp with audit2allow.

22 April 2015: Wednesday

Posted: April 22nd, 2015 by fcs

LDAP:

  • No issues in last night’s LDAP update.
  • No issues in this morning’s ACCOUNT update.

Backups:

  • Clones finished up last night.
  • windowbox set up

RHEL7:

  • fighting with selinux and nrpe on bujbod2

21 April 2015: Tuesday

Posted: April 21st, 2015 by fcs

LDAP:

  • The Giraffe migration broke the delivery of feed files to giraffe last night.  Spent two hours (9pm-11pm) working with Kent and Mike to get it fixed.
  • Mike updated the LDAP ACLs because of the giraffe migration and a failure to test.
  • No other issues in the LDAP update last night (well, other than there was no Banner feed because of the giraffe migration).
  • No issues in this morning’s ACCOUNT update (only one item to process anyway).

Backups:

  • Clones started.
  • New clients that have issues (I love this game of Russian Roulette – NOT!) after the restart of NetWorker (and upgrade to 8.2.1.2) yesterday.
  • Last couple of clients that would not let ozzie back them up are now fixed and backups have been taken.

Footprints:

  • SHA-2 cert installed to shut Chrome up.

20 April 2015: Monday

Posted: April 20th, 2015 by fcs

LDAP:

  • No issues in the nightly LDAP update over the weekend.
  • No issues in the morning ACCOUNT update over the weekend.

Backups:

  • DD SR 70755774 – someone finally called back at 4:15pm (after I had left for the day) and was focused on the bundle upload issue – not the certificate problem.  EMC updated the SR that they’d call me at 11am.  I called them at 11:45.  Turns out the guy that was working the issue appears to have not shown up for work today (sick?).  Requeued and waiting for callback.
    • Rebooted DD690 and Ozzie – still cannot read the /ddvar directory over the nfs mount.  Yeesh.
    • Guess what…
      • there’s a bug with DDOS, an NFS mounted /ddvar directory that you perform an “ls” command it – immediately becomes stale.  You can’t use ls (or tab completion) to find files.  You have to KNOW the file name.  You can also cheat and use “echo *” to get a psuedo-ls output.
  • Two systems (print servers) left with “I don’t wanna talk to you” errors.
  • One Windows system that was being upgraded and the NetWorker client is now missing the C disk in the backup.
  • One Windows system that now that it will talk to the NetWorker server is failing to back up its E disk again.
  • Backup Maintenance Day.

17 April 2015: Friday

Posted: April 17th, 2015 by fcs

LDAP:

  • No issues in the LDAP update last night.
  • No issues in the ACCOUNT update this morning.

Backups:

  • Thirty (30) windows systems refuse to let our networker server back them up this morning.  This has turned into a real mess.
  • SR 66027568: DD Compression failure – want a support bundle.  I can’t provide it because I can’t get access to the /ddvar directory (nfs stale file handle problem — from the dd690 nfs server!)
  • The dd690 does not like its own certificate.  I think it may have died and forgot to tell us.  Opened SR 70755774 as a Sev2 – we’ll see if it actually gets a call back today.  Nope – no callback before it was time to go.

RHEL7:

  • cfengine – looks like our config was messed up quite some time ago.  Instead of trying to figure out how long ago it got hosed, I shall attempt to move it forward and figure out how to use the new service management provided in the cfengine_stdlib.cf file.

FootPrints:

  • Tracking down a failure to append issue for an end user.

16 April 2015: Thursday

Posted: April 16th, 2015 by fcs

LDAP:

  • No issues in LDAP update.
  • No issues in ACCOUNT update.

Backups:

  • Disk clones finished up at 3am this morning.
  • A change I made to start removing DD690 attachments yesterday caused the bootstrap to fail to run.  Oops.  The bootstrap device HAS (I mean, *H*A*S*) to be directly attached to the NetWorker server, not a storage node.
  • Still having issues with lots of Windows clients that refuse to let our NetWorker server back them up.  Damn this 8.2.1.2 security fix right behind 8.2.1.1 being pushed out and EMC’s stupid way of setting up Windows packages so that the windows updater thinks the 8.2.1.1 and 8.2.1.2 .msi files are the same version.
  • Updated the SR66570742 audit_compliance code to include the NW Volume name for things that should be on the AFTD miss report line.

RHEL7:

  • Preliminary systemd support added to cfengine.

15 April 2015: Wednesday

Posted: April 15th, 2015 by fcs

LDAP:

  • Automated Purge Day: four errors (email bounces that already exist)
  • No issues in the nightly LDAP update.
  • No issues in the morning ACCOUNT update.

Backups:

  • Disk clones continue, with just 5TB left.
  • Lots of windows servers failed to backup last night because ozzie was not in their servers list.
  • Continuing with the NFS perf testing.
    • 8 streams, bs=32K/64K/128K with two sets of sysctl settings

14 April 2015: Tuesday

Posted: April 14th, 2015 by fcs

LDAP:

  • Monday night’s update had a serious problem: OSP sent a birthday that started with a zero (0), which happens to be illegal in the LDAP schema for integers.  The issue was paged at 8:30pm.  I was paged at 9:20pm and worked until 10:30pm to fix the LDAP update issue and run the following processes.
    • Code updated to check for and prevent similar problem in the future.
  • No issues with the ACCOUNT update this morning.
  • Dealing with NetID questions from Account Services.

Backups:

  • Tape and CoM/IS clones finished yesterday.
  • Disk Clones continue with about 17TB left.
  • EMC releases 8.2.1.2 to fix buffer overflow in nsr_render_log command that will allow anyone to run arbitrary commands.
  • Windows client still won’t backup its E drive.  Spent 110 minutes (yes, almost 2 hours) in chat with EMC.  Still not resolved.
  • More work to try to get NFS performance up and going on bujbod2.

 

13 April 2015: Monday

Posted: April 13th, 2015 by fcs

LDAP:

  • No issues in the nightly LDAP update over the weekend.
  • No issues in the daily ACCOUNT update over the weekend.

Backups:

  • Tape Movement: 190 tapes went to the vault, 240 came back.
  • Reclaimed six (6) tapes for the Scratch pool to go to the Vault.
Contact Us ©2010 The University of Vermont – Burlington, VT 05405 – (802) 656-3131
Skip to toolbar