I'm getting inconsistent results when using cor() function. It is easiest to demonstrate with small piece of code:

data("pbc2.id", package = "JM") # Mayo Clinic Primary Biliary Cirrhosis Data
df <- pbc2.id
vars_num1 <- c("years", "age", "serBilir", "serChol", "albumin", 
          "alkaline", "SGOT", "platelets", "prothrombin", "histologic", 
          "status2")
cor(df[vars_num1], use = "complete.obs", method="pearson") # years vs age: -0.17719866
cor(df$years, df$age, use = "complete.obs", method="pearson") # -0.1631033

Other columns do give consistent results such as serBilir vs serChol (0.39675890). I also coded it myself to test it:

v <- function(x,y=x) mean(x*y) - mean(x)*mean(y)
my_corr <- function(x,y) v(x,y) / sqrt(v(x) * v(y))
my_corr(df$years, df$age) # -0.1631033

So why does cor(df[vars_num1], use = "complete.obs", method="pearson") give different results?

I'm getting inconsistent results when using cor() function. It is easiest to demonstrate with small piece of code:

data("pbc2.id", package = "JM") # Mayo Clinic Primary Biliary Cirrhosis Data
df <- pbc2.id
vars_num1 <- c("years", "age", "serBilir", "serChol", "albumin", 
          "alkaline", "SGOT", "platelets", "prothrombin", "histologic", 
          "status2")
cor(df[vars_num1], use = "complete.obs", method="pearson") # years vs age: -0.17719866
cor(df$years, df$age, use = "complete.obs", method="pearson") # -0.1631033

Other columns do give consistent results such as serBilir vs serChol (0.39675890). I also coded it myself to test it:

v <- function(x,y=x) mean(x*y) - mean(x)*mean(y)
my_corr <- function(x,y) v(x,y) / sqrt(v(x) * v(y))
my_corr(df$years, df$age) # -0.1631033

So why does cor(df[vars_num1], use = "complete.obs", method="pearson") give different results?

• r
• correlation
• pearson-correlation

• 2 Because "use=complete.obs" will drop all rows if any variable in the data frame contains NA. There are 28 NA in serChol and 4 in platelets (check with summary(df)) so these will be excluded completely. The second cor command doesn't exclude them, so the results differ. – Edward Commented Mar 11 at 10:31
• 2 Use use = "pairwiseplete.obs" to get the same results for both. – Edward Commented Mar 11 at 10:34
• @Edward thank you, pairwiseplete.obs is working – Hunter Commented Mar 11 at 11:26

1 Answer 1

I think the problem comes from your NA values. In the second case, the cor function keeps more rows than in the first case. Use na.omit and you will see that you find the same thing.

data("pbc2.id", package = "JM") # Mayo Clinic Primary Biliary Cirrhosis Data
df <- pbc2.id

vars_num1 <- c("years", "age", "serBilir", "serChol", "albumin", 
          "alkaline", "SGOT", "platelets", "prothrombin", "histologic", 
          "status2")

df = na.omit(df)
cor(df[vars_num1], use = "complete.obs", method="pearson") # years vs age: -0.17719866
cor(df$years, df$age, use = "complete.obs", method="pearson") # -0.17719866

df[vars_num1]