Server hosting BigBlueButton exploded

On Aug 6, 2014, at 10:45 AM, Frank Swasey wrote:

Wes,

Yesterday morning /etc/cron.daily/apt applied a libc update to morpho. While that shouldn’t have caused a problem, it might have.  NetWorker is not working (it throws an invalid pointer error in munmap_chunk() which is in glibc).  I have upgraded the NetWorker client to 8.1.1.7 (the latest and greatest) but it still is failing.  EMC would like us to reboot the system because it might resolve the issue.

Is there any particular time that a reboot would be better or worse? Are you ok with me (or someone from SAA) rebooting the system?


Frank Swasey

On 8/6/14, 10:56 AM, Wesley A. Wright wrote:

Check https://www.uvm.edu/ctl/apps/bbb/?c=bbb&m=listall

If nothing is running, ok to reboot

 

On Aug 6, 2014, at 11:27 AM, Jim Lawson wrote:

Hi Wes,

I took a look at morpho and it appears that the system is somewhat
broken.  A glibc upgrade has occurred which seems to break ABI, but
there are some packages which are held back (I presume they were copied
and installed by hand) which are preventing the system from finishing
its install.

There are various suggestions that aptitude is making to resolve the
package state:  removing libreoffice, removing openoffice, or removing
bbb (followed by an install of a more updated version?)  might get the
system into a better state.

On 8/6/14, 11:53 AM, Wesley A. Wright wrote:

Why does all this happen while I’m on vacation? I’m on a farm in southern canada armed with an iPad and < 200mb remaining on my at&t international data plan. Why do I keep reading email?

Most recent bigbluebutton is .81. That's what is installed. Libreoffice is a bbb dependency. I don't think OpenOffice is installed, but it might be left over from a previous bbb install, in any case it has been superseded by libreoffice and is no longer required.

I suppose bbb can be removed and reinstalled, although there are a handful of undocumented site specific configuration files that might get overwritten, thus rendering bbb useless.

So what to do? Remove all XOffice, reinstall libreoffice , don't mess with bbb unless we really have to? Regress to earlier glibc? Wait until I get back (aug.  18)?

 

On 8/6/14, 12:08 PM, Jim Lawson wrote:

It looks like bbb is still working, which is very surprising to me,
since the glibc breakage is so bad that we can’t run scp or iptables.

Maybe it can wait until the 18th, but it probably shouldn’t wait any
longer because you might have to recover from backups to fix this
particular problem.  (That’s a bare-metal restore, BTW, since ‘recover’
on morpho is also badly broken.)

On Aug 6, 2014, at 4:19 PM, Jim Lawson wrote:

Mike and I tested it and, astonishingly, BBB is still working (we shared
a 2-person videoconf.)  If you are comfortable waiting until you are
back on the 18th to look at it, that’s up to you.  Mike and I are both
on vacation the week of the 18th, so assistance from SAA that week may
not be as timely.


On 8/7/14, 2:07 PM, Wesley A. Wright wrote:

Found this:

https://bugs.launchpad.net/ubuntu/+bug/1352769

Seemed to work. apt-get working now, scp too.

Included with latest upgrade was bbb-client update

https://groups.google.com/forum/m/#!topic/bigbluebutton-setup/db8VxAn_iaY

Which should not break anything,but I don’t have the equipment to test.

How many points do I win for fixing this?

Sent from my iPad

About Wesley Wright

Born on a mountain top near New York City, Craziest state in the land of the pretty. Raised in the woods so's he knew every tree, Killed him a bear when he was only three.
This entry was posted in Wes and tagged , . Bookmark the permalink.