Posts Tagged ‘Thunderbird’

Thunderbird 13 – The cloud arrives

Mozilla Thunderbird 13 arrived this week.  Guess what?  Our customized build process broke again.  Now, when you start TB for the first time, you get greeted with the option to create a new email account with one of Thunderbird’s “partners” (in other words, email providers who paid for the honor of being put in the “welcome to Thunderbird” start dialog).

With the assistance of the awesome Ben Coddington (who does not keep a blog, but should so that you can bask in his awesomeness), I was able to track down the place that the new-new account dialog is called, and kill it by switching a preference in the “thunderbird-all.js” file.

The preference is a Boolean named “mail.provider.enabled”, set in the thunderbird-all.js file, as documented here:
http://hg.mozilla.org/releases/comm-beta/rev/879e8d044e36
and referenced here:
https://bugzilla.mozilla.org/show_bug.cgi?id=718792#c3
and here:
https://wiki.mozilla.org/index.php?title=Thunderbird/Support/TB13UserChanges

I updated our Thunderbird build script to set this preference to “false”:

Echo modifying default "All Thunderbird" preferences...
..\..\..\bin\sed.exe --binary "s/pref(\"mail.provider.enabled\", true);/pref(\"mail.provider.enabled\", false);/"  .\defaults\pref\all-thunderbird_new.js
if errorlevel 1 goto err
MOVE /Y .\defaults\pref\all-thunderbird_new.js .\defaults\pref\all-thunderbird.js

The whole ugly build script is provided below:

REM Thunderbird customized build script for UVM.
REM Updated June 2012 for Thunderbird 13 support.
REM REQUIRES: 
REM 	- 7z.exe, 7zr.exe and sed.exe in parallel "..\bin" directory
REM     - Unmodified Thunderbird installer in .\source directory
REM		- all required config files in .\config directory
REM     	(including 7z control file, ISP Hook RDF file, and modified prefs.js)
REM		- local JDK install with "jar.exe".  Path to jar.exe will need to be updated in the jdk environment variable
REM OUTPUT: Fully modified Thunderbird installer in .\Installer directory.
REM @echo on

set jdk="c:\Program Files (x86)\Java\jdk1.6.0_27\bin"

Echo Cleaning up old builds...
del .\Installer\*.exe
rmdir /s /q .\build
set /P tbver=Enter Thunderbird version number to build (i.e. "6.0.2"):

Echo Extracting setup files from OEM Installer...
mkdir .\build\temp
..\bin\7zr x .\source\*.exe -o.\build

Echo Extracting omni.ja contents...
mkdir .\build\temp
cd .\build\temp
%jdk%\jar.exe xf ..\core\omni.ja
if errorlevel 1 goto err

Echo modifying messenger functions...
..\..\..\bin\sed.exe --binary "s/NewMailAccount(msgWindow, okCallback);/MsgAccountWizard(okCallback);/"  .\chrome\messenger\content\messenger\msgMail3PaneWindow_new.js
if errorlevel 1 goto err
MOVE /Y .\chrome\messenger\content\messenger\msgMail3PaneWindow_new.js .\chrome\messenger\content\messenger\msgMail3PaneWindow.js

Echo modifying default "All Thunderbird" preferences...
..\..\..\bin\sed.exe --binary "s/pref(\"mail.provider.enabled\", true);/pref(\"mail.provider.enabled\", false);/"  .\defaults\pref\all-thunderbird_new.js
if errorlevel 1 goto err
MOVE /Y .\defaults\pref\all-thunderbird_new.js .\defaults\pref\all-thunderbird.js

Echo modifying default mailnews preferences...
..\..\..\bin\sed.exe --binary "s/try_ssl\", 0)/try_ssl\", 2)/"  .\defaults\pref\mailnews_new.js
if errorlevel 1 goto err
MOVE /Y .\defaults\pref\mailnews_new.js .\defaults\pref\mailnews.js

Echo moving UVM modified prefs.js into place (note that this file is not actually used by Thunderbird!)
copy /Y ..\..\config\prefs.js .\defaults\profile\prefs.js

Echo Repacking omni.ja...
del /f /q ..\core\omni.ja
%jdk%\jar.exe cf ..\core\omni.ja *

Echo Copying UVM Custom ISP file to source...
cd ..\..\
mkdir .\build\core\isp\en-US
copy /Y .\config\UVMMail.rdf .\build\core\isp\en-US\UVMMail.rdf
if errorlevel 1 goto err
Echo Copying UVM default prefs.js to core directory (tbird no longer has a prefs.js by default, but it will be used if present)...
mkdir .\build\core\defaults\profile
copy /Y .\config\prefs.js .\build\core\defaults\profile\prefs.js
if errorlevel 1 goto err

Echo Deleting temporary files that should not be present in the installer...
rmdir /s /q .\build\temp

Echo Repackaging Thunderbird installer...
..\bin\7zr a .\Installer\UVM_Thunderbird_setup_%tbver%.7z .\build\*
copy /b ..\bin\7zS.sfx + .\config\config.txt + .\Installer\UVM_Thunderbird_setup_%tbver%.7z .\Installer\UVM_Thunderbird_setup_%tbver%.exe

Echo Cleaning up installation source...
del /s /f /q .\build\*.*
rmdir /s /q .\build\core
rmdir /s /q .\build
del /f /q .\Installer\UVM_Thunderbird_setup_%tbver%.7z
goto end

:err
echo There was an error running a command.

:end

Mozilla Thunderbird – Implementing the ISP Hook in 5.0 and later

Out buddies at MozillaMessaging are at it again… new with Thunderbird 5.0, all of the “jar” files previously present in the Thunderbird installation directory have been collapsed into a single “omni.jar” file, apparently for program load optimization.

This all would be fine with me if the omni.jar were a “normal” zip file, and the previous jars were. Instead, this is an “optimized” jar, that is not recognized by 7-zip as a valid zip archive. It is not clear to me how the jar is “optimized”, or how to re-apply optimizations when modifying the original, nor do I particularly care as load optimization is of little concern to us… we are not operating with 10 year old equipment, for the most part, so who cares?

I have had to work around the problem by using “jar.exe” from the Java JDK. This program extracts and re-creates omni.jar in a way that Thunderbird does not mind. The resultant file size is about the same, too.

Another quirk of the new version is that the default “prefs.js” file, while present in omni.jar, is not copied into the resultant Program Files directory at install time. On a clean install, there is no default prefs.js! I had to populate a new prefs.js into the “core” installer directory, outside of the new “omni.jar”.

Finally, there are no more “localized/nonlocalized” directories in the installer, just that which is within “omni”, and that which is not. So, I put our prefs.js in .\core\defaults\profile (a new directory in the installer zip). Previously it was in .\nonlocalized\defaults\profile. Likewise, our mailnews.js and ISP Hook “.rdf” files also go in “core” instead of “localized”.

Other than that, the RDF ISP Hook file that we are using is unchanged from the one documented here:
https://sharepoint.uvm.edu/sites/ad/distribution/appconfig/Thunderbird.aspx
Prefs.js is the same as in the SharePoint site, with the modifications noted here:
http://blog.uvm.edu/jgm/2009/12/15/thunderbird-3-re-implementing-the-isp-hook-for-customized-deployments/

What a bunch of gibberish talk… sorry for the dry post. I am guessing if you are a Thunderbird customizer, you will know what I am talking about. If not, you won’t be reading anyway.

On to the build script, in Windows BAT/CMD language:

REM Thunderbird customized build script for UVM.
REM Updated September 2011 for Thunderbird 5.0 support.
REM REQUIRES: 
REM 	- 7z.exe, 7zr.exe and sed.exe in parallel "..\bin" directory
REM     - Unmodified Thunderbird installer in .\source directory
REM		- all required config files in .\config directory
REM     	(including 7z control file, ISP Hook RDF file, and modified prefs.js)
REM		- local JDK install with "jar.exe".  Path to jar.exe will need to be updated in the jdk environment variable
REM OUTPUT: Fully modified Thunderbird installer in .\Installer directory.
REM @echo on

set jdk="c:\Program Files (x86)\Java\jdk1.6.0_27\bin"

Echo Cleaning up old builds...
del .\Installer\*.exe
rmdir /s /q .\build
set /P tbver=Enter Thunderbird version number to build (i.e. "6.0.2"):

Echo Extracting setup files from OEM Installer...
mkdir .\build\temp
..\bin\7zr x .\source\*.exe -o.\build

Echo Extracting omni.jar contents...
mkdir .\build\temp
cd .\build\temp
%jdk%\jar.exe xf ..\core\omni.jar

Echo modifying messenger functions...
..\..\..\bin\sed.exe --binary "s/NewMailAccount(msgWindow, okCallback);/MsgAccountWizard(okCallback);/"  .\chrome\messenger\content\messenger\msgMail3PaneWindow_new.js
MOVE /Y .\chrome\messenger\content\messenger\msgMail3PaneWindow_new.js .\chrome\messenger\content\messenger\msgMail3PaneWindow.js

Echo modifying default mailnews preferences...
..\..\..\bin\sed.exe --binary "s/try_ssl\", 0)/try_ssl\", 2)/"  .\defaults\pref\mailnews_new.js
MOVE /Y .\defaults\pref\mailnews_new.js .\defaults\pref\mailnews.js

Echo moving UVM modified prefs.js into place (note that this file is not actually used by Thunderbird!)
copy /Y ..\..\config\prefs.js .\defaults\profile\prefs.js

Echo Repacking omni.jar...
del /f /q ..\core\omni.jar
%jdk%\jar.exe cf ..\core\omni.jar *

Echo Copying UVM Custom ISP file to source...
cd ..\..\
mkdir .\build\core\isp\en-US
copy /Y .\config\UVMMail.rdf .\build\core\isp\en-US\UVMMail.rdf
Echo Copying UVM default prefs.js to core directory (tbird no longer has a prefs.js by default, but it will be used if present)...
mkdir .\build\core\defaults\profile
copy /Y .\config\prefs.js .\build\core\defaults\profile\prefs.js

Echo Deleting temporary files that should not be present in the installer...
rmdir /s /q .\build\temp

Echo Repackaging Thunderbird installer...
..\bin\7zr a .\Installer\UVM_Thunderbird_setup_%tbver%.7z .\build\*
copy /b ..\bin\7zS.sfx + .\config\config.txt + .\Installer\UVM_Thunderbird_setup_%tbver%.7z .\Installer\UVM_Thunderbird_setup_%tbver%.exe

Echo Cleaning up installation source...
del /s /f /q .\build\*.*
rmdir /s /q .\build\core
rmdir /s /q .\build
del /f /q .\Installer\UVM_Thunderbird_setup_%tbver%.7z

Thunderbird 3.1 breaks our customizations

It appears that a Thunderbird 3.1 release (probably 3.1.1) introduced a change that has caused at least one entry in our UVM ISP Hook file to become invalided.  Two alert distributed support staff recently reported that the SMTP Server that our installer configures now is set to perform plain-text authentication over an unencrypted channel rather than using STARTTLS.

We know that this setting was correct in the past, and we see that it still is set in our ISP profile… Thunderbird simply is ignoring the setting.

Fortunately, there is a workaround for this setting.  We can modify the “mailnews.js” file in the installer to have the default value for “try_ssl” set to “2″ instead of the original default of “0″.  All I had to do was add a line of “sed” work to our build script:

..\bin\sed.exe --binary "s/try_ssl\", 0)/try_ssl\", 2)/"  .\build\nonlocalized\defaults\pref\mailnews_new.js
MOVE /Y .\build\nonlocalized\defaults\pref\mailnews_new.js .\build\nonlocalized\defaults\pref\mailnews.js

This “sed” command searches for the phrase try_ssl", 0 and substitutes try_ssl", 2 . After repackaging, we find that the first SMTP server account created after install (which is therefore the default SMTP account) will be set to use “STARTTLS” instead of “clear text”.
I worry about future unannounced retirements of ISP Hook settings.

Thunderbird 3 – re-implementing the ISP Hook for customized deployments

About two years ago I did some work to customize the Mozilla Thunderbird product to work with the UVM IMAP servers out-of box:
https://sharepoint.uvm.edu/sites/ad/distribution/appconfig/Thunderbird.aspx

This project made use of the so-called “ISP Hook” for Thunderbird, loosely documented here:
https://developer.mozilla.org/en/Thunderbird_ISP_hooks

Unfortunately, it appeared that with the release of Thunderbird 3.0 the ISP Hook was being retired in favor of “autoconfig”:
https://wiki.mozilla.org/Thunderbird:Autoconfiguration

Autoconfig is a promising project, but unfortunately, it lacks the ability to configure all of the settings previously available in the ISP Hook RDF file:
https://bugzilla.mozilla.org/show_bug.cgi?id=475139

I had a long look at using the previous “autoconfig” (called “Mission Control Desktop”), and while this approach is powerful, it really requires too much external scripting to be effective as a deployment mechanism (MCD seems more ideally suited to management of systems in a lab environment):
https://developer.mozilla.org/En/MCD

So what is the solution?  It involves:

  1. One expatriate Windows sys admin and reverse-engineering expert.
  2. A few line of additional shell scripting to our existing build system.
  3. The ability to distinguish capital and lowercase letters.

While explaining the ISP Hook problem to my colleague Ben Coddington, I noticed that the old Account Configuration dialogs (complete with the create “UVM Email Account” option) was still available under TB3, but that the dialog could no longer be activated following a new installation.  Ben said “I’ll bet I can fix that”, and proceeded to unpack every “jar” file in the Thunderbird installation directory. 

He searched around for strings found in the old config dialog, located the XUL call that brings up the dialog, and then traced this call back to the parent function that initiates account creation for new profiles.  In the end we only needed to modify one line of javascript in one JAR to restore the old account dialog.  In “%ProgramFiles%\Mozilla Thunderbird\chrome”, we unpack “messenger.jar”, and open “.\content\messenger\msgMail3PaneWindow.js”, then replace “NewMailAccount(msgWindow, okCallback);” with “MsgAccountWizard(okCallback);”.  repack the archive, then drop it into the installer, and we are good to go.  Other than that, it is just like using the ISP Hook with TB 2.

There is one other minor change… I also set the following additional default option in the global prefs.js file:

user_pref("mail.winsearch.enable", true);

This defaults Windows Search integration to “on” in the new account setup dialog.  It is a cool new feature that everyone should use (in my opinion).

The following is the modified build script I now use to quickly repackage new Thunderbird releases into customized versions.  Note that we use the free gnuwin32 version of “sed” to do our javascript modifications (be sure to use the –b (—binary) switch to avoid file munging), and we also need 7-zip (7z.exe) to handle file unpacking and packing:

REM Thunderbird customized build script for UVM.
REM Updated December 2009 for Thunderbird 3.0 support.
REM REQUIRES: 7z.exe, 7zr.exe and sed.exe in parallel "..\bin" directory
REM     Unmodified Thunderbird installer in .\source directory
REM        all required config files in .\config directory
REM     (including 7z control file, ISP Hook RDF file, and modified prefs.js)
REM OUTPUT: Fully modified Thunderbird installer in .\Installer directory.
@echo off

Echo Cleaning up old builds...
del .\Installer\*.exe
rmdir /s /q .\build
set /P tbver=Enter Thunderbird version number to build (i.e. "3.0.0.0"):

Echo Extracting setup files from OEM Installer...
mkdir .\build\temp
..\bin\7zr x .\source\*.exe -o.\build

Echo Extracting messenger.jar contents...
REM messenger.jar is a ZIP type archive... need full 7z executable to unpack.
..\bin\7z x .\build\nonlocalized\chrome\messenger.jar -o.\build\temp

Echo modifying messenger functions...
..\bin\sed.exe --binary "s/NewMailAccount(msgWindow, okCallback);/MsgAccountWizard(okCallback);/" <.\build\temp\content\messenger\msgMail3PaneWindow.js > .\build\temp\content\messenger\msgMail3PaneWindow_new.js
MOVE /Y .\build\temp\content\messenger\msgMail3PaneWindow_new.js .\build\temp\content\messenger\msgMail3PaneWindow.js

Echo Repacking messenger.jar...
REM again, we must use the full 7z executable.  -mx=0 means archive with no compression.
..\bin\7z a -tzip -mx=0 .\build\temp\messenger.jar .\build\temp\*

Echo Copying UVM Custom config files to source...

copy /Y .\config\prefs.js .\build\nonlocalized\defaults\profile\prefs.js
mkdir .\build\nonlocalized\isp\en-US
copy /Y .\config\UVMMail.rdf .\build\nonlocalized\isp\en-US\UVMMail.rdf
copy /Y .\build\temp\messenger.jar .\build\nonlocalized\chrome\messenger.jar

Echo Deleting temporary files...
rmdir /s /q .\build\temp

Echo Repackaging Thunderbird installer...
..\bin\7zr a .\Installer\UVM_Thunderbird_setup_%tbver%.7z .\build\*
copy /b ..\bin\7zS.sfx + .\config\config.txt + .\Installer\UVM_Thunderbird_setup_%tbver%.7z .\Installer\UVM_Thunderbird_setup_%tbver%.exe

Echo Cleaning up installation source...
REM rmdir /s /q .\build
del /f /q .\Installer\UVM_Thunderbird_setup_%tbver%.7z