Function: $\epsilon$–first, Greedy, Decreasing

$\epsilon–first$:

$$ P(x) = \begin{cases} i \le \text{threshold}, & x=1 \\ i > \text{threshold}, & x=0 \end{cases} $$

$\epsilon–greedy$:

$$ P(x) = \begin{cases} \epsilon, & x=1 \\ 1-\epsilon, & x=0 \end{cases} $$

$\epsilon–decreasing$:

$$ P(x) = \begin{cases} \frac{1}{1+\epsilon \cdot i}, & x=1 \\ \frac{\epsilon \cdot i}{1+\epsilon \cdot i}, & x=0 \end{cases} $$

Usage

func_epsilon(rownum, params, ...)

Arguments

rownum: The trial number
params: Parameters used by the model’s internal functions, see params
...: Subject ID, Block ID, Trial ID, and any additional information defined by the user.

Body

func_epsilon <- function(
    rownum,
    params,
    ...
){
  # if you need extra information
  # e.g.
  # Trial <- idinfo["Trial"]
  # Frame <- exinfo["Frame"]

  epsilon   <-  multiRL:::get_param(params, "epsilon")
  threshold <-  multiRL:::get_param(params, "threshold")

  # Determine the model currently in use based on which parameters are free.
  if (is.na(epsilon) && threshold > 0) {
    model <- "first"
  } else if (!(is.na(epsilon)) && threshold == 0) {
    model <- "decreasing"
  } else if (!(is.na(epsilon)) && threshold == 1) {
    model <- "greedy"
  } else {
    stop("Unknown Model! Plase modify your learning rate function")
  }

  set.seed(rownum)
  # Epsilon-First:
  if (rownum <= threshold) {
    try <- 1
  } else if (rownum > threshold && model == "first") {
    try <- 0
    # Epsilon-Greedy:
  } else if (rownum > threshold && model == "greedy"){
    try <- sample(
      c(1, 0),
      prob = c(epsilon, 1 - epsilon),
      size = 1
    )
    # Epsilon-Decreasing:
  } else if (rownum > threshold && model == "decreasing") {
    try <- sample(
      c(1, 0),
      prob = c(
        1 / (1 + epsilon * rownum),
        epsilon * rownum / (1 + epsilon * rownum)
      ),
      size = 1
    )
  }

  return(try)
}

Function: \(\epsilon\)–first, Greedy, Decreasing

Usage

Arguments

Body