In a previous blog post,
I presented time series cross-validation with `crossvalidation::crossval_ts`

. The most
recent version of package `crossvalidation`

, v0.4.1, contains a **new function** called `eval_ts`

.
How does `crossvalidation::eval_ts`

work? As described in the graph below:

`crossvalidation::crossval_ts`

and `crossvalidation::eval_ts`

have the same arguments, except,
an argument `p`

for `crossval_ts`

and an argument `q`

for `eval_ts`

. `p`

is the percentage of original
data used for cross-validation and hyperparameter tuning, and `q`

is the percentage of unseen data used
for model validation. In addition, and most importantly, we **must have p + q = 1**, to avoid overlapping
train/test/validation sets.

Here’s a concrete example of use of `crossvalidation::eval_ts`

:

**Installing packages**

```
options(repos = c(
techtonique = 'https://techtonique.r-universe.dev',
CRAN = 'https://cloud.r-project.org'))
install.packages("ahead")
install.packages("crossvalidation")
install.packages("e1071")
```

**Loading packages**

```
library(ahead) # forecasting tool
library(crossvalidation) # cross-validation and evaluation
library(datasets) # dataset Nile
library(e1071) # for svm
```

**Input data**

```
y <- AirPassengers
```

**Forecasting function**

```
fcast_func <- function(y, h = 5, kernel=c("linear", "radial"))
{
kernel <- match.arg(kernel)
ahead::dynrmf(y=y, h=h, level=95,
fit_func = e1071::svm,
fit_params = list(kernel = kernel),
predict_func = predict)
}
```

We’ll consider that the *hyperparameter* is `kernel`

, which is either “linear” or “radial”.

**Error metric (Root Mean Squared Error, RMSE)**

```
eval_metric <- function(predicted, observed)
{
return(sqrt(mean((observed - predicted) ^ 2, na.rm = FALSE)))
}
```

**Cross-validation on 80% of the data**

With `kernel = "radial"`

```
res_cv_radial <- crossvalidation::crossval_ts(y = y,
fcast_func = fcast_func,
fit_params = list(kernel = "radial"),
initial_window = 20,
horizon = 10, p=0.8,
eval_metric = eval_metric,
show_progress = FALSE)
print(median(res_cv_radial))
```

```
[1] 47.29141
```

With `kernel = "linear"`

```
res_cv_linear <- crossvalidation::crossval_ts(y = y,
fcast_func = fcast_func,
fit_params = list(kernel = "linear"),
initial_window = 20,
horizon = 10, p=0.8,
eval_metric = eval_metric,
show_progress = FALSE)
print(median(res_cv_linear))
```

```
[1] 44.18955
```

**Evaluation with the** ***best hyperparameter*, `kernel = "linear"`

, on unseen data (`q = 1 - p`

) **

```
res_eval_linear <- crossvalidation::eval_ts(y = y,
fcast_func = fcast_func,
fit_params = list(kernel = "linear"),
initial_window = 20,
horizon = 10, q=0.2,
eval_metric = eval_metric,
show_progress = FALSE)
print(median(res_eval_linear))
```

```
[1] 86.47903
```