Time series cross-validation is now available in crossval, using function `crossval::crossval_ts`

. Main parameters for `crossval::crossval_ts`

include:

`fixed_window`

described below in sections 1 and 2, and indicating if the training set’s size is fixed or increasing through cross-validation iterations`initial_window`

: the number of points in the rolling training set`horizon`

: the number of points in the rolling testing set

Yes, this type of functionality exists in packages such as `caret`

, or `forecast`

, but with different flavours. We start by installing crossval from its online repository (in R’s console):

```
library(devtools)
devtools::install_github("thierrymoudiki/crossval")
library(crossval)
```

## 1 - Calling `crossval_ts`

with option `fixed_window = TRUE`

`initial_window`

is the length of the training set, depicted in blue, which is **fixed** through cross-validation iterations. `horizon`

is the length of the testing set, in orange.

### 1 - 1 Using statistical learning functions

```
data("AirPassengers")
# regressors including trend
xreg <- cbind(1, 1:length(AirPassengers))
# cross validation with least squares regression
res <- crossval_ts(y=AirPassengers, x=xreg, fit_func = crossval::fit_lm,
predict_func = crossval::predict_lm,
initial_window = 10,
horizon = 3,
fixed_window = TRUE)
# print results
print(colMeans(res))
```

```
ME RMSE MAE MPE MAPE
0.16473829 71.42382836 67.01472299 0.02345201 0.22106607
```

### 1 - 2 Using time series functions from package `forecast`

```
res <- crossval_ts(y=AirPassengers, initial_window = 10,
horizon = 3,
fcast_func = forecast::thetaf,
fixed_window = TRUE)
print(colMeans(res))
```

```
ME RMSE MAE MPE MAPE
2.657082195 51.427170382 46.511874693 0.003423843 0.155428590
```

## 2 - Calling `crossval_ts`

with option `fixed_window = FALSE`

`initial_window`

is the length of the training set, in blue, which **increases** through cross-validation iterations. `horizon`

is the length of the testing set, depicted in orange.

### 2 - 1 Using statistical learning functions

```
# regressors including trend
xreg <- cbind(1, 1:length(AirPassengers))
# cross validation with least squares regression
res <- crossval_ts(y=AirPassengers, x=xreg, fit_func = crossval::fit_lm,
predict_func = crossval::predict_lm,
initial_window = 10,
horizon = 3,
fixed_window = FALSE)
# print results
print(colMeans(res))
```

```
ME RMSE MAE MPE MAPE
11.35159629 40.54895772 36.07794747 -0.01723816 0.11825111
```

### 2 - 2 Using time series functions from package `forecast`

```
res <- crossval_ts(y=AirPassengers, initial_window = 10,
horizon = 3,
fcast_func = forecast::thetaf,
fixed_window = FALSE)
print(colMeans(res))
```

```
ME RMSE MAE MPE MAPE
2.670281455 44.758106487 40.284267136 0.002183707 0.135572333
```

**Note:** I am currently looking for a *gig*. You can hire me on Malt or send me an email: **thierry dot moudiki at pm dot me**. I can do descriptive statistics, data preparation, feature engineering, model calibration, training and validation, and model outputs’ interpretation. I am fluent in Python, R, SQL, Microsoft Excel, Visual Basic (among others) and French. My résumé? Here!

Under License Creative Commons Attribution 4.0 International.

**Previous publications**

- New version of mlsauce, with Gradient Boosted randomized networks and stump decision trees Jul 3, 2020
- Announcements Jun 26, 2020
- Parallel AdaOpt classification Jun 19, 2020
- Comments section and other news Jun 12, 2020
- Maximizing your tip as a waiter Jun 5, 2020
- AdaOpt classification on MNIST handwritten digits (without preprocessing) May 29, 2020
- AdaOpt (a probabilistic classifier based on a mix of multivariable optimization and nearest neighbors) for R May 22, 2020
- AdaOpt May 15, 2020
- Custom errors for cross-validation using crossval::crossval_ml May 8, 2020
- Documentation+Pypi for the `teller`, a model-agnostic tool for Machine Learning explainability May 1, 2020
- Encoding your categorical variables based on the response variable and correlations Apr 24, 2020
- Linear model, xgboost and randomForest cross-validation using crossval::crossval_ml Apr 17, 2020
- Grid search cross-validation using crossval Apr 10, 2020
- Documentation for the querier, a query language for Data Frames Apr 3, 2020
- Time series cross-validation using crossval Mar 27, 2020
- On model specification, identification, degrees of freedom and regularization Mar 20, 2020
- Import data into the querier (now on Pypi), a query language for Data Frames Mar 13, 2020
- R notebooks for nnetsauce Mar 6, 2020
- Version 0.4.0 of nnetsauce, with fruits and breast cancer classification Feb 28, 2020
- Create a specific feed in your Jekyll blog Feb 21, 2020
- Git/Github for contributing to package development Feb 14, 2020
- Feedback forms for contributing Feb 7, 2020
- nnetsauce for R Jan 31, 2020
- A new version of nnetsauce (v0.3.1) Jan 24, 2020
- ESGtoolkit, a tool for Monte Carlo simulation (v0.2.0) Jan 17, 2020
- Search bar, new year 2020 Jan 10, 2020
- 2019 Recap, the nnetsauce, the teller and the querier Dec 20, 2019
- Understanding model interactions with the `teller` Dec 13, 2019
- Using the `teller` on a classifier Dec 6, 2019
- Benchmarking the querier's verbs Nov 29, 2019
- Composing the querier's verbs for data wrangling Nov 22, 2019
- Comparing and explaining model predictions with the teller Nov 15, 2019
- Tests for the significance of marginal effects in the teller Nov 8, 2019
- Introducing the teller Nov 1, 2019
- Introducing the querier Oct 25, 2019
- Prediction intervals for nnetsauce models Oct 18, 2019
- Using R in Python for statistical learning/data science Oct 11, 2019
- Model calibration with `crossval` Oct 4, 2019
- Bagging in the nnetsauce Sep 25, 2019
- Adaboost learning with nnetsauce Sep 18, 2019
- Change in blog's presentation Sep 4, 2019
- nnetsauce on Pypi Jun 5, 2019
- More nnetsauce (examples of use) May 9, 2019
- nnetsauce Mar 13, 2019
- crossval Mar 13, 2019