I’ve been using the R/exams package to design randomised quiz questions containing a mix of numeric and single multiple-choice components, using the cloze format. My goal is to upload them to Canvas LMS.

However, when I upload the generated quiz to Canvas, it appears that only the multiple-choice questions render properly, while the numeric parts are not interpreted correctly. I then tried separating the multiple choice and the numeric questions and created a standalone Rmd for the numeric type, but when uploaded, there is only a single text box for the first question. Could you please provide suggestions how to create a single file that contains both question types?

Below is an example Rmd

```{r, include=FALSE}
# Define the levels
lvs <- c("normal", "abnormal")

# Original vectors
truth <- factor(rep(lvs, times = c(86, 258)), levels = rev(lvs))
pred <- factor(
  c(rep(lvs, times = c(54, 32)), rep(lvs, times = c(27, 231))),
  levels = rev(lvs)
)

library("caret")
cm <- confusionMatrix(truth, pred)
cm$overall["Accuracy"]
cm$byClass["Sensitivity"]
cm$byClass["Specificity"]
cm$overall["Kappa"]

## QUESTION/ANSWER GENERATION
questions <- solutions <- explanations <- points <- rep(list(""), 4)
type <- rep(list("num"), 4)

questions[[1]] <- ""
solutions[[1]] <- explanations[[1]] <- round(as.numeric(cm$overall["Accuracy"]), 3)

questions[[2]] <- ""
solutions[[2]] <- explanations[[2]] <-round(as.numeric(cm$byClass["Sensitivity"]), 3)

questions[[3]] <- ""
solutions[[3]] <- explanations[[3]] <- round(as.numeric(cm$byClass["Specificity"]), 3)

questions[[4]] <- ""
solutions[[4]] <- explanations[[4]] <- round(as.numeric(cm$overall["Kappa"]), 3)

points[c(1:4)] <- 1
tol <- 0.001
```

Question
========
Some blurb here

**The abnormal class is the class of interest**, since these are the cases that require intervention. Your friend developed a classification model and summarised the results using the confusion matrix below:

```{r, echo=FALSE, results="asis"}
library("kableExtra")
tab <- cm$table
colnames(tab) <-  c("Actual Abnormal", "Actual Normal")
rownames(tab) <-  c("Predicted Abnormal", "Predicted Normal")
knitr::kable(tab, "html", align = ('lrrr'), booktabs=T) |> 
    kable_styling(bootstrap_options = c("striped","hover"), full_width = FALSE
    ) |> 
  add_header_above(c(" ", "Confusion Matrix" = 2))
```

Calculate the following performance metrics to 3 decimal places:

Accuracy: ##ANSWER1##

Sensitivity: ##ANSWER2##

Specificity: ##ANSWER3##

Cohen's Kappa: ##ANSWER4##

```{r questionlist, echo = FALSE, results = "asis"}
answerlist(unlist(questions), markup = "markdown")
```

Solution
========

```{r solutionlist, echo = FALSE, results = "asis"}
answerlist(paste(unlist(explanations), ".", sep = ""), markup = "markdown")
```

Meta-information
================
extype: num
exsolution: `r paste(solutions, collapse = "|")`
exclozetype: `r paste(type, collapse = "|")`
extol: `r tol`
expoints: `r paste(points, collapse = "|")`
exname: NUM

Thank you in advance for your help!

I’ve been using the R/exams package to design randomised quiz questions containing a mix of numeric and single multiple-choice components, using the cloze format. My goal is to upload them to Canvas LMS.

However, when I upload the generated quiz to Canvas, it appears that only the multiple-choice questions render properly, while the numeric parts are not interpreted correctly. I then tried separating the multiple choice and the numeric questions and created a standalone Rmd for the numeric type, but when uploaded, there is only a single text box for the first question. Could you please provide suggestions how to create a single file that contains both question types?

Below is an example Rmd

```{r, include=FALSE}
# Define the levels
lvs <- c("normal", "abnormal")

# Original vectors
truth <- factor(rep(lvs, times = c(86, 258)), levels = rev(lvs))
pred <- factor(
  c(rep(lvs, times = c(54, 32)), rep(lvs, times = c(27, 231))),
  levels = rev(lvs)
)

library("caret")
cm <- confusionMatrix(truth, pred)
cm$overall["Accuracy"]
cm$byClass["Sensitivity"]
cm$byClass["Specificity"]
cm$overall["Kappa"]

## QUESTION/ANSWER GENERATION
questions <- solutions <- explanations <- points <- rep(list(""), 4)
type <- rep(list("num"), 4)

questions[[1]] <- ""
solutions[[1]] <- explanations[[1]] <- round(as.numeric(cm$overall["Accuracy"]), 3)

questions[[2]] <- ""
solutions[[2]] <- explanations[[2]] <-round(as.numeric(cm$byClass["Sensitivity"]), 3)

questions[[3]] <- ""
solutions[[3]] <- explanations[[3]] <- round(as.numeric(cm$byClass["Specificity"]), 3)

questions[[4]] <- ""
solutions[[4]] <- explanations[[4]] <- round(as.numeric(cm$overall["Kappa"]), 3)

points[c(1:4)] <- 1
tol <- 0.001
```

Question
========
Some blurb here

**The abnormal class is the class of interest**, since these are the cases that require intervention. Your friend developed a classification model and summarised the results using the confusion matrix below:

```{r, echo=FALSE, results="asis"}
library("kableExtra")
tab <- cm$table
colnames(tab) <-  c("Actual Abnormal", "Actual Normal")
rownames(tab) <-  c("Predicted Abnormal", "Predicted Normal")
knitr::kable(tab, "html", align = ('lrrr'), booktabs=T) |> 
    kable_styling(bootstrap_options = c("striped","hover"), full_width = FALSE
    ) |> 
  add_header_above(c(" ", "Confusion Matrix" = 2))
```

Calculate the following performance metrics to 3 decimal places:

Accuracy: ##ANSWER1##

Sensitivity: ##ANSWER2##

Specificity: ##ANSWER3##

Cohen's Kappa: ##ANSWER4##

```{r questionlist, echo = FALSE, results = "asis"}
answerlist(unlist(questions), markup = "markdown")
```

Solution
========

```{r solutionlist, echo = FALSE, results = "asis"}
answerlist(paste(unlist(explanations), ".", sep = ""), markup = "markdown")
```

Meta-information
================
extype: num
exsolution: `r paste(solutions, collapse = "|")`
exclozetype: `r paste(type, collapse = "|")`
extol: `r tol`
expoints: `r paste(points, collapse = "|")`
exname: NUM

Thank you in advance for your help!

• canvas
• r-exams

1 Answer 1

Source of the problem: Unfortunately, the support of the Canvas LMS for cloze exercises is not as flexible as for other systems (like Moodle or OpenOlat). In particular, there is no good documentation that would explain how to configure exactly the QTI 1.2 XML files for cloze exercises so that they can be imported successfully into Canvas. So far we were only successful in figuring out how to import cloze exercises with multiple schoice elements.

Your example: I edited your post to better format the Rmd file that you shared and I also load the caret and kableExtra packages that are needed. If I copy that Rmd code into a file confusionmatrix.Rmd then I can do the following:

exams2canvas("confusionmatrix.Rmd")
##   adding: canvasquiz_548176981/ (stored 0%)
##   adding: canvasquiz_548176981/canvasquiz_548176981.xml (deflated 74%)
##   adding: canvasquiz_548176981/assessment_meta.xml (deflated 70%)
##   adding: imsmanifest.xml (deflated 67%)
## Warning message:
## In read_metainfo(file, exshuffle = exshuffle) :
##   exclozetype should only be specified for extype 'cloze'

The code "works" but warns that the exclozetype is not meaningful for num exercises. Maybe the warning should be even more explicit that currently the exercise is treated as a num exercise with a single answer only.

To change that you should use extype: cloze. After doing so I get:

exams2canvas("confusionmatrix.Rmd")
## Error in itembody[[type]](exm[[i]][[j]]) : 
##   only cloze questions with schoice elements are supported for Canvas

Thus, this reflects the problem described in the beginning.

Outlook: In principle, it should be possible to import this type of exercise into Canvas but so far we were not able to figure out how. If anyone has been successful in this and has documented how the XML needs to be configured, please let us know.

P.S.: Your Solution section does not really provide any additional insights over the exsolution in the meta-information. Hence I would either expand the Solution section with a proper explanation or omit it entirely.