How I use the Zotero reference manager for collaborative grants or manuscripts

Why Zotero?

Zotero is an excellent and free reference manager that is my go-to for writing grants and manuscripts. It has some very handy features, like word processor plugins, web browser plugins that will grab PDF versions of documents if available, and ability to share “group libraries” with collaborators. It has all of the standard features of reference managers, like auto-formatting of references to meet submission requirements, automatic renumbering of in-line references, etc.

There are some silly things about Zotero’s initial setup that are important to get out of the way. For example, you don’t necessarily need to have a Zotero account to use Zotero (or at least you didn’t when I used it the first time). Make sure that you read about how to get set up with Zotero under the “Zotero” heading on this page. After you do that, come back here and read on!

How to use Zotero for collaborative projects

Zotero works well with MS Word and Google Docs. Examples here are taken from MS Word, but are also applicable to Google Docs. The main difference between Google Docs and MS Word is that the web browser plugin is also the Google Docs plugin. MS Word has a plugin separate from the web browser plugin. Regardless, whenever you use a Zotero plugin (eg the MS Word, Google Docs, or browser extensions), you also need to have the Zotero desktop app open. You’ll get an error if you try to insert a reference into a document or snag a reference from PubMed/a journal website if the Zotero desktop app isn’t also open.

Organizing your folders (“collections”) and subfolders (“subcollections”).

In your desktop app, navigate to the shared library that I’ll send you. Make folders/collections or subfolders/subcollections in there to help stay organized. To make a new folder/collection, right click on the shared group library and click “New Collection…”. To make subfolders/subcollections, right click on that new folder/subfolder that you made.

I suggest making collections/folders by section of your document, and numbering them so they stay in order, so: “01 Introduction”, “02 Methods”, “03 Results”, and “04 Discussion”. If you are writing something that doesn’t follow a usual flow (eg an opinion piece), number/name things by the major sections in your outline. You can always rename these folders/collections and renumber them so they show up in order.

Now, within each of these folders/collections, make specific groupings of subfolders/subcollections by topic. For example, in the introduction, you might have a sentence talking about the epidemiology/population prevalence of hypertension, then the costs (eg DALY lost) of hypertension, then an overview of the pathophysiology of hypertension, then how some biomarker relates to blood pressure. I recommend having a subfolder for each of these concepts separately in the “01 introduction” folder. You can also order these with numbers or letters, but it also might make sense to keep them unordered if you aren’t sure of how the introduction (or any other section) will flow.

Now repeat this for all of the other subfolders. The results folder might be pretty thin because usually (for me at least) there aren’t many references in that section. For the “discussion section”, I recommend including the suggested sections from my “your first epidemiology manuscript” example under “Download” here. It’ll look like this when you are done:

Next: Grabbing citations.

You need to install the Zotero browser extension, and make sure that you have the Zotero desktop app open when you are grabbing citations. I STRONGLY RECOMMEND GRABBING JOURNAL CITATIONS FROM PubMed AND ONLY PubMed. (For textbooks, Google Books works quite well.) Zotero uses metadata from websites to populate the reference. PubMed’s metadata is pristine. Journal websites have metadata, but it’s inconsistent and often incomplete. So, stick with PubMed when grabbing citations.

The first step is to find your article on PubMed. Let’s say you want to grab the AHA’s statistical update document, and using Google you find it on the Journal’s website. Welp, that’s not pubmed so don’t even think about grabbing the reference from here.

Instead, highlight the title and search Google again, appending the word “PubMed” to the search. This will get you to the Pubmed listing for that article.

Now this is very important! Go over to your Zotero desktop app and choose the subcollection/subfolder you want this to go in. This will be in the Introduction/Population Prevalence of HTN subfolder. You’ll notice it’s empty.

Now go back to the PubMed page for your document and click the Zotero plugin button.

Now when you go back to the Zotero Desktop app, you’ll see that the AHA Statistical Update is now saved in your “population prevalence of HTN” folder. If a PDF is available through Unpaywalled (a separate service that’s integrated into Zotero that grabs free/legal copies of journal PDFs), then a PDF will be saved as well.

When you grab references, make sure to sort them into a specific folder along the way. You’ll thank yourself later.

Next: Inserting citations in an MS Word document

Open up your manuscript file in MS Word. Make sure you click/set your cursor in the place in your document where you want the citation to land. (I’m a “citation goes after the period and not before the period” guy myself.) Then, click the Zotero tab. (If you don’t see the Zotero tab, you might need to try to reboot, or manually install the plugin. See the “Setting Things Up/Zotero” section here.) Now, click the “Add/Edit Citation” button. If this is the first citation/reference you are adding to this document, you’ll be prompted to select a formatting style, just pick anything since you can always change it later (I like the American Medical Association one to start with). You won’t see this pop up when adding other references.

Now you’ll see the hovering Zotero window. This isn’t a part of MS Word, it’s actually the Zotero desktop app. Sometimes this gets lost among your various programs/windows on your desktop and you need to go find it, it’ll be under the Zotero icon on your taskbar on windows. Or Alt+Tab until you find it. Anyway, this allows you to find a citation by text search (ie, by typing in the author name or title), but if you select the dropdown menu here, you can use the “classic view” and manually grab citations from your subcollections/subfolders. I recommend adding citations through the classic view.

In the classic view you will see all of your subcollections/subfolders. Navigate to your subcollection/subfolder of interest and click on the citation that you’d like to insert and hit “okay”. Notice in the “Classic View”, you can select multiple references at the same time by clicking the “Multiple Sources…” button at the bottom.

Now your citation is in-line! See the floating “1” after the first sentence. But where is the Reference list? Let’s plop one in. I added a new heading for references in MS Word and we’ll add it there. Click on the line after your “references” header, go to the zotero tab, and click “add/edit bibliography”.

End product is below. This reference list will update while you insert references in your manuscript. The reference numbers will also update automatically as you go.

Good luck!

Descriptive labels of metrics assessing discrimination

Discrimination and calibration of models predicting risk

Risk prediction is common in medicine, eg the Framingham Risk Score and the Pooled Cohort Equation/10-year ASCVD risk prediction models. Machine learning models that also predict risk are growing in popularity.

Discrimination and calibration are discussed in this excellent JAMA article. In brief, discrimination relates to how a model divvies up low and high risk individuals. So, in a population of folks of various risk levels, high discriminating models will score higher risk folks higher than low risk folks. For example, a CVD model should say that a 60 year old with hypertension, hyperlipidemia, and diabetes has a higher risk of CVD than a 20 year old with none of those conditions. Calibration, on the other hand, describes how well a model predicts risk for an individual person.

Discrimination descriptive labels

I have to look this up every time, so I am putting this here. Here’s a widely-used labeling for discrimination, which in this manuscript, we called it the “Hosmer-Lemeshow criteria”. These authors applied this to area under the ROC (AUROC) curves, but I think it’s also fair to apply to C-statistics.

  • 0.5: No discrimination
  • 0.5 to <0.7: Poor discrimination
  • 0.7 to <0.8: Acceptable discrimination
  • 0.8 to <0.9: Excellent discrimination
  • ≥0.9: Outstanding discrimination

The reference is here:

Hosmer JD, Lemeshow S, Sturdivant R. Applied Logistic Regression. Hoboken, NJ, USA: John Wiley & Sons Inc; 2013. 

Here’s the Google Books listing for this, in case you want to grab the metadata for this reference using Zotero or whatnot. You’ll see the above labels on page 177 — you can see this with the “Preview” view.