Biostat III exercises in R

Laboratory exercise 7

Suggested solutions by

Author: Johan Zetterqvist, 2015-03-03

Model cause-specific mortality with poisson regression

In this exercise we model, using Poisson regression, cause-specific mortality of patients diagnosed with localised (stage==1) melanoma.

In exercise 9 we model cause-specific mortality using Cox regression and in exercise 28 we use flexible parametric models. The aim is to illustrate that these three methods are very similar.

The aim of these exercises is to explore the similarities and differences to these three approaches to modelling. We will be comparing the results (and their interpretation) as we proceed through the exercises.


Load the diet data using time-on-study as the timescale.

You may have to install the required packages the first time you use them. You can install a package by install.packages("package_of_interest") for each package you require.

require(survival) # for Surv and survfit
require(dplyr)    # for data manipulation
require(foreign)  # for reading data set from Stata


###########################################
### A help function to calculate ###
### and print incidence (hazard) ratios
### from a fitted poisson regression
### from glm
###########################################

IRR <- function(fit){
    summfit <- summary(fit )$coefficients
    IRfit <- exp( cbind( summfit[, 1:2], summfit[, 1] - 1.96*summfit[, 2], summfit[, 1] +
                        1.96*summfit[, 2] ) )
    colnames(IRfit) <- c("IRR", "Std. err", "CI_lower", "CI_upper")
    print(IRfit)
}

Load the melanoma data and explore it.

## Read melanoma data
melanoma <- tbl_df( read.dta("http://biostat3.net/download/melanoma.dta") )

## Create a new dataset with only localised cancer
melanoma.l <- filter(melanoma, stage=="Localised")
head( melanoma.l )
summary(melanoma.l)

Rates can be modelled on different timescales, e.g., attained age, time-since-entry, calendar time. Plot the CHD incidence rates both by attained age and by time-since-entry. Is there a difference? Do the same for CHD hazard by different energy intakes (hieng).

(a) i. Plot Kaplan-Meier estimates of cause-specific survival as a function of calendar period of diagnosis.

## Plot Kaplan-Meier curve using survfit
## Create a new event indicator
melanoma.l <- mutate(melanoma.l,
                   death_cancer = as.numeric(status=="Dead: cancer") )

## Create a fitted object for our subcohort
## using survfit
sfit7a1 <- survfit(Surv(surv_mm, event=death_cancer) ~ year8594,
                   data = melanoma.l )

## Have a look at the fitted object
str(sfit7a1, 1)
## List of 14
##  $ n        : int [1:2] 2145 3173
##  $ time     : num [1:381] 0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 ...
##  $ n.risk   : num [1:381] 2145 2143 2140 2138 2135 ...
##  $ n.event  : num [1:381] 0 3 0 2 2 0 4 5 4 5 ...
##  $ n.censor : num [1:381] 2 0 2 1 3 2 3 2 5 3 ...
##  $ surv     : num [1:381] 1 0.999 0.999 0.998 0.997 ...
##  $ type     : chr "right"
##  $ strata   : Named int [1:2] 249 132
##   ..- attr(*, "names")= chr [1:2] "year8594=Diagnosed 75-84" "year8594=Diagnosed 85-94"
##  $ std.err  : num [1:381] 0 0.000809 0.000809 0.001045 0.001237 ...
##  $ upper    : num [1:381] 1 1 1 1 0.999 ...
##  $ lower    : num [1:381] 1 0.997 0.997 0.996 0.994 ...
##  $ conf.type: chr "log"
##  $ conf.int : num 0.95
##  $ call     : language survfit(formula = Surv(surv_mm, event = death_cancer) ~ year8594,      data = melanoma.l)
##  - attr(*, "class")= chr "survfit"
## Plot the survival curve (with some bells and whistles)
plot(sfit7a1,
     ## No automatic labelling of the curve (we do that ourselves)
     mark.time=F,
     ## Time is measured in months,  but we want to see it in years
     xscale=12,
     ## Make the plot prettier
     xlab="Years since diagnose",
     ylab="S(t)",
     col=c("blue","red"),
     lty=c("solid","dashed"))
## Add legend too
legend("bottomleft",legend=levels(melanoma.l$year8594),col=c("blue","red"),lty=c("solid","dashed"))

### TRY IF YOU WANT ###
## install.packages("survMisc")
## require(survMisc)
## autoplot(sfit7a1)
  1. Now plot the estimated hazard function (cause-specific mortality rate) as a function of calendar period of diagnosis.
## To plot smoothed hazards, we need use the muhaz package
require(muhaz)

## The plot.muhaz cannot rescale so we create a new variable
## Create a new variable base on the number of months
## translated to years (more accurate than surv_yy)
melanoma.l <- mutate( melanoma.l, surv_yy1 = surv_mm/12)

## Since the muhaz interface does not support data frames
## we do this the old school way
hazfit7a.early <- with(melanoma.l,
                       muhaz(times=surv_yy1,
                             delta=status=="Dead: cancer",
                             subset=year8594=="Diagnosed 75-84"))

hazfit7a.late <- with(melanoma.l,
                      muhaz(times=surv_yy1,
                            delta=status=="Dead: cancer",
                            subset=year8594=="Diagnosed 85-94",
                            max.time=max(surv_yy1)))
## Warning in muhaz(times = surv_yy1, delta = status == "Dead: cancer", subset = year8594 == : maximum time > maximum Survival Time
## With bells and whistles
plot(hazfit7a.early,xlab="Years since diagnose",col="blue",lty="solid")
lines(hazfit7a.late,col="red",lty="dashed")
legend("topright",legend=levels(melanoma$year8594),col=c("blue","red"),lty=c("solid","dashed"))

During which calendar period (the early or the latter) is mortality the lowest?

  1. Is the interpretation (with respect to how prognosis depends on period) based on the hazard consistent with the interpretation of the survival plot?
## Compare with Kaplan-Meier plot
par(mfrow=c(1,2)) ## Two graphs in the same window

plot(sfit7a1,
     ## No automatic labelling of the curve (we do that ourselves)
     mark.time=F,
     ## Time is measured in months,  but we want to see it in years
     xscale=12,
     ## Make the plot prettier
     xlab="Years since diagnose",
     ylab="S(t)",
     col=c("blue","red"),
     lty=c("solid","dashed"))
legend("bottomleft",legend=levels(melanoma.l$year8594),col=c("blue","red"),lty=c("solid","dashed"))

plot(hazfit7a.early,xlab="Years since diagnose",col="blue",lty="solid")
lines(hazfit7a.late,col="red",lty="dashed")
legend("topright",legend=levels(melanoma$year8594),col=c("blue","red"),lty=c("solid","dashed"))

(b) Estimate the cause-specific mortality rate for each calendar period.

## Calculate the incidence rate by time of diagnosis
rates_by_diag_yr <- melanoma.l %>%

    ## Stratify on year of diagnosis
    group_by(year8594) %>%

    ## We only need the deaths and the persontime
    select(death_cancer, surv_mm) %>%

    summarise(
        ## Calculate the number of deaths
        D = sum(death_cancer),
        ## Calculate the total person time
        Y = sum(surv_mm)/12/1000,
        ## Calculate the rate
        Rate = D/Y) %>%

    ## Add confidence intervals for the rates
    mutate(CI_low = Rate + qnorm(0.025) * Rate / sqrt(D),
           CI_high = Rate + qnorm(0.975) * Rate / sqrt(D))

rates_by_diag_yr
## Source: local data frame [2 x 6]
## 
##          year8594   D        Y     Rate   CI_low CI_high
## 1 Diagnosed 75-84 572 22.66279 25.23961 23.17122 27.3080
## 2 Diagnosed 85-94 441 15.96379 27.62502 25.04673 30.2033

During which calendar period (the early or the latter) is mortality the lowest? Is this consistent with what you found earlier? If not, why the inconsistency?

(c) The reason for the inconsistency between parts 7a and 7b was confounding by time since diagnosis. The comparison in part 7a was adjusted for time since diagnosis (since we compare the differences between the curves at each point in time) whereas the comparison in part 7b was not. Understanding this concept is central to the remainder of the exercise so please ask for help if you don’t follow.

Two approaches for controlling for confounding are ‘restriction’ and ‘statistical adjustment’. We will first use restriction to control for confounding. We will restrict the potential follow-up time to a maximum of 120 months. Individuals who survive more than 120 months are censored at 120 months.

  1. Estimate the cause-specific mortality rate for each calendar period.
## Calculate the incidence rate by time of diagnosis
## but with new variables
melanoma.l2 <- mutate(melanoma.l,
                       ## Update the death indicator (only count deaths within 120 months)
                       death_cancer = death_cancer * as.numeric(surv_mm<=120),
                       ## Create a new time variable
                       surv_mm = pmin(surv_mm, 120) )

## Calculate the rates on the truncated data
rates_by_diag_yr2 <- melanoma.l2 %>%

    ## Stratify on year of diagnosis
    group_by(year8594) %>%

    ## We only need the deaths and the persontime
    select(death_cancer, surv_mm) %>%

    summarise(
        ## Calculate the number of deaths
        D = sum(death_cancer),
        ## Calculate the total person time
        Y = sum(surv_mm)/12/1000,
        ## Calculate the rate
        Rate = D/Y) %>%

    ## Add confidence intervals for the rates
    mutate(CI_low = Rate + qnorm(0.025) * Rate / sqrt(D),
           CI_high = Rate + qnorm(0.975) * Rate / sqrt(D))

rates_by_diag_yr2
## Source: local data frame [2 x 6]
## 
##          year8594   D        Y     Rate   CI_low  CI_high
## 1 Diagnosed 75-84 519 16.50104 31.45256 28.74661 34.15851
## 2 Diagnosed 85-94 441 15.87562 27.77843 25.18583 30.37104
rates_by_diag_yr
## Source: local data frame [2 x 6]
## 
##          year8594   D        Y     Rate   CI_low CI_high
## 1 Diagnosed 75-84 572 22.66279 25.23961 23.17122 27.3080
## 2 Diagnosed 85-94 441 15.96379 27.62502 25.04673 30.2033

During which calendar period (the early of the latter) is mortality the lowest? Is this consistent with what you found in part 7b?

  1. Calculate by hand the ratio (85–94/75–84) of the two mortality rates (i.e., a mortality rate ratio) and interpret the estimate (i.e., during which period is mortality higher/lower and by how much).
## MRR full data
rates_by_diag_yr2[2, "Rate"] / rates_by_diag_yr2[1, "Rate"]
##        Rate
## 1 0.8831852
## ## MRR truncated data
## rates_by_diag_yr2[2, "Rate"] / rates_by_diag_yr2[1, "Rate"]
  1. Now use Poisson regression to estimate the same mortality rate ratio. Write the linear predictor using pen and paper and draw a graph of the fitted hazard rates.
## Use glm to estimate the rate ratios
## we scale the offset term to personyears
poisson7c <- glm( death_cancer ~ year8594 + offset( log( surv_mm/12/1000 ) ), family=poisson, data=melanoma.l2 )
summary( poisson7c )
## 
## Call:
## glm(formula = death_cancer ~ year8594 + offset(log(surv_mm/12/1000)), 
##     family = poisson, data = melanoma.l2)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -0.7931  -0.7568  -0.5632  -0.3081   3.0550  
## 
## Coefficients:
##                         Estimate Std. Error z value Pr(>|z|)    
## (Intercept)              3.44848    0.04389  78.566   <2e-16 ***
## year8594Diagnosed 85-94 -0.12422    0.06476  -1.918   0.0551 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for poisson family taken to be 1)
## 
##     Null deviance: 4814.3  on 5317  degrees of freedom
## Residual deviance: 4810.6  on 5316  degrees of freedom
## AIC: 6734.6
## 
## Number of Fisher Scoring iterations: 6
## IRR
IRR(poisson7c)
##                                IRR Std. err   CI_lower  CI_upper
## (Intercept)             31.4525600 1.044870 28.8598349 34.278212
## year8594Diagnosed 85-94  0.8831851 1.066905  0.7779032  1.002716
## with confidence intervals (takes some time to compute)
## exp(cbind(coef(poisson7c),confint(poisson7c)))


## Note that the scaling of the offset term only has an impact on the intercept
summary( glm( death_cancer ~ year8594 + offset( log( surv_mm ) ),
             family=poisson, data=melanoma.l2 ) )
## 
## Call:
## glm(formula = death_cancer ~ year8594 + offset(log(surv_mm)), 
##     family = poisson, data = melanoma.l2)
## 
## Deviance Residuals: 
##     Min       1Q   Median       3Q      Max  
## -0.7931  -0.7568  -0.5632  -0.3081   3.0550  
## 
## Coefficients:
##                         Estimate Std. Error  z value Pr(>|z|)    
## (Intercept)             -5.94418    0.04389 -135.426   <2e-16 ***
## year8594Diagnosed 85-94 -0.12422    0.06476   -1.918   0.0551 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for poisson family taken to be 1)
## 
##     Null deviance: 4814.3  on 5317  degrees of freedom
## Residual deviance: 4810.6  on 5316  degrees of freedom
## AIC: 6734.6
## 
## Number of Fisher Scoring iterations: 6

(7d) In order to adjust for time since diagnosis (i.e., adjust for the fact that we expect mortality to depend on time since diagnosis) we need to split the data by this timescale. We will restrict our analysis to mortality up to 10 years following diagnosis.

## Add a new variable for year
melanoma.l2 <- mutate( melanoma.l2, surv_yy1 = surv_mm/12)

## Split follow up by year
melanoma.spl <- survSplit(melanoma.l2, cut=0:9, end="surv_yy1", start="start",
                           event="death_cancer")

## Calculate persontime and
## recode start time as a factor
melanoma.spl <- mutate(melanoma.spl,
                       pt = surv_yy1 - start,
                       fu = as.factor(start) )

(e) Now tabulate (and produce a graph of) the rates by follow-up time.

## Calculate the incidence rate by observation year
yearly_rates <- melanoma.spl %>%

    ## Stratify on follow-up year
    group_by(fu) %>%

    ## We only need the deaths and the persontime
    select(death_cancer, pt) %>%

    summarise(
        ## Calculate the number of deaths
        D = sum(death_cancer),
        ## Calculate the total person time measured in 1000
        Y = sum(pt)/1000,
        ## Calculate the rate
        Rate = D/Y) %>%

    ## Add confidence intervals for the rates
    mutate(CI_low = Rate + qnorm(0.025) * Rate / sqrt(D),
           CI_high = Rate + qnorm(0.975) * Rate / sqrt(D))

yearly_rates
## Source: local data frame [10 x 6]
## 
##    fu   D        Y     Rate    CI_low  CI_high
## 1   0  71 5.257000 13.50580 10.364286 16.64732
## 2   1 228 4.857917 46.93370 40.841621 53.02578
## 3   2 202 4.235458 47.69260 41.115663 54.26953
## 4   3 138 3.711583 37.18090 30.977517 43.38428
## 5   4 100 3.265583 30.62240 24.620520 36.62428
## 6   5  80 2.864667 27.92646 21.806917 34.04600
## 7   6  56 2.524792 22.18005 16.370845 27.98925
## 8   7  35 2.190250 15.97991 10.685856 21.27397
## 9   8  34 1.886375 18.02399 11.965566 24.08241
## 10  9  16 1.583042 10.10712  5.154725 15.05953
## Plot by year
with(yearly_rates, matplot(fu,
                           cbind(Rate, CI_low,
                             CI_high),
                           lty=c("solid","dashed","dashed"),
                           col=c("black","gray","gray"),
                           type="l",
                           main="Cancer deaths per 1000 person-year by years since diagnose",
                           ylab="IR",
                           xlab="Years since diagnose") )

Mortality appears to be quite low during the first year of follow-up. Does this seem reasonable considering the disease with which these patients have been diagnosed?

(f) Compare the plot of the estimated rates to a plot of the hazard rate as a function of continuous time.

hazfit7f <- with(melanoma.l2, muhaz(times=surv_yy1, delta=death_cancer, max.time=10.5) )

# Plot smoothed hazards
frame() # Open new graphics device (window)
plot(hazfit7f,xlab="Years since diagnose",col="blue",lty="solid")