Risk prediction is common in medicine, eg the Framingham Risk Score and the Pooled Cohort Equation\/10-year ASCVD risk prediction models. Machine learning models that also predict risk are growing in popularity. <\/p>\n\n\n\n
Discrimination and calibration are discussed in this excellent JAMA article<\/a>. In brief, discrimination relates to how a model divvies up low and high risk individuals. So, in a population of folks of various risk levels, high discriminating models will score higher risk folks higher than low risk folks. For example, a CVD model should say that a 60 year old with hypertension, hyperlipidemia, and diabetes has a higher risk of CVD than a 20 year old with none of those conditions. Calibration, on the other hand, describes how well a model predicts risk for an individual person. <\/p>\n\n\n\n I have to look this up every time, so I am putting this here. Here’s a widely-used labeling for discrimination, which in this manuscript<\/a>, we called it the “Hosmer-Lemeshow criteria”. These authors applied this to area under the ROC (AUROC) curves, but I think it’s also fair to apply to C-statistics.<\/p>\n\n\n\n The reference is here: <\/p>\n\n\n\n Here’s the Google Books listing<\/a> for this, in case you want to grab the metadata for this reference using Zotero or whatnot. You’ll see the above labels on page 177 — you can see this with the “Preview” view. <\/p>\n","protected":false},"excerpt":{"rendered":" Discrimination and calibration of models predicting risk Risk prediction is common in medicine, eg the Framingham Risk Score and the Pooled Cohort Equation\/10-year ASCVD risk prediction models. Machine learning models that also predict risk are growing in popularity. Discrimination and calibration are discussed in this excellent JAMA article. In brief, discrimination relates to how a … Continue reading Descriptive labels of metrics assessing discrimination<\/span><\/a><\/p>\n","protected":false},"author":4473,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[502558],"tags":[703372,703369,703368,20138,703376,703374,703375,703373,703370],"class_list":["post-1024","post","type-post","status-publish","format-standard","hentry","category-epidemiology-and-biostatistics","tag-accuracy","tag-auroc","tag-c-statistic","tag-discrimination","tag-machine-learning","tag-prediction","tag-prediction-models","tag-receiver-operating-characteristic","tag-roc"],"_links":{"self":[{"href":"https:\/\/blog.uvm.edu\/tbplante\/wp-json\/wp\/v2\/posts\/1024","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.uvm.edu\/tbplante\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.uvm.edu\/tbplante\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.uvm.edu\/tbplante\/wp-json\/wp\/v2\/users\/4473"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.uvm.edu\/tbplante\/wp-json\/wp\/v2\/comments?post=1024"}],"version-history":[{"count":3,"href":"https:\/\/blog.uvm.edu\/tbplante\/wp-json\/wp\/v2\/posts\/1024\/revisions"}],"predecessor-version":[{"id":1925,"href":"https:\/\/blog.uvm.edu\/tbplante\/wp-json\/wp\/v2\/posts\/1024\/revisions\/1925"}],"wp:attachment":[{"href":"https:\/\/blog.uvm.edu\/tbplante\/wp-json\/wp\/v2\/media?parent=1024"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.uvm.edu\/tbplante\/wp-json\/wp\/v2\/categories?post=1024"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.uvm.edu\/tbplante\/wp-json\/wp\/v2\/tags?post=1024"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Discrimination descriptive labels<\/h2>\n\n\n\n
\n
Hosmer JD, Lemeshow S, Sturdivant R. Applied Logistic Regression. Hoboken, NJ, USA: John Wiley & Sons Inc; 2013. <\/pre>\n\n\n\n