Chapter 2 Small Scale Example
Relationship between: Wind Speed and Energy Production of Wind Turbines
2.1 Intro
will insert a introduction here
…
We will use wind speed data from 6 locations. We want to see whether using temporal and nonlinear features improve our forecast accuracy.
The relationships that we want to analyse are as follows:
- Linear relationship between the wind speed and energy production
- Temporal (e.g. features from past and (forecasts) from future)
- Non-linear (e.g. square of features)
- Combination of Temporal and Non-linear
- Polynomial
- Lagged version of Polynomial
- Polynomials of Temporal features
In the following sections we will show that all these relationships matter for wind power forecasting.
Also we will see that dimensions increase exponentially as we try to derive more features. So we will have a motivation for dimension reduction.
2.1.1 The Data
Our data has much more locations and ranges over a longer time. However, for this report we have reduced size for easiness, also we will see that even reduced size will not be small enough.
All 6 datasets have 6 months of observations. First 5 months will be used for training and the last month will be used for test.
5 of our 6 datasets have 9 locations (3x3 grid) and dataset-1 has 4 locations (2x2).
Here is a look of our datasets.
head(dt1)## date hour production loc1 loc2 loc3 loc4
## 1: 2017-01-02 0 0 1.490839 1.184061 1.141621 0.4410216
## 2: 2017-01-02 1 0 1.700052 1.307793 1.385103 0.6289321
## 3: 2017-01-02 2 0 1.910143 1.432977 1.628786 0.8169388
## 4: 2017-01-02 3 0 2.120849 1.559263 1.872592 1.0049876
## 5: 2017-01-02 4 0 2.225062 1.552668 2.059827 0.9891467
## 6: 2017-01-02 5 0 2.332381 1.547259 2.249153 0.9741549head(dt2)## date hour production loc1 loc2 loc3 loc4 loc5 loc6 loc7 loc8 loc9
## 1: 2017-01-02 0 7.00 2.420620 2.644277 2.922619 2.624881 3.144026 2.994879 2.288274 2.910344 3.524599
## 2: 2017-01-02 1 7.00 2.415885 2.607095 2.823119 2.617361 3.118926 3.009090 2.321575 2.935492 3.521327
## 3: 2017-01-02 2 10.00 2.411182 2.570002 2.723692 2.612287 3.095949 3.023605 2.354877 2.962141 3.521237
## 4: 2017-01-02 3 20.00 2.406512 2.533002 2.624348 2.609674 3.075142 3.038421 2.388179 2.990251 3.524330
## 5: 2017-01-02 4 27.00 2.522292 2.622637 2.607378 2.641233 3.119695 3.146457 2.564588 3.112134 3.559182
## 6: 2017-01-02 5 17.09 2.638823 2.712524 2.590410 2.672959 3.164253 3.254569 2.741650 3.234055 3.5942192.1.2 Linear
It is clear that wind speed can give us some idea about the wind energy production. Let’s see:
dfit1 = cv.glmnet(x = as.matrix(dt1[,-c(1:3)]), y = dt1$production, type.measure = "mae")
pred.test1 = predict(dfit1, as.matrix(dte1[,-c(1:3)]), s = "lambda.1se")
predict(dfit1,type="coef") # coefficients
## 5 x 1 sparse Matrix of class "dgCMatrix"
## lambda.1se
## (Intercept) -13.913447
## loc1 -2.136017
## loc2 -1.279383
## loc3 -1.319780
## loc4 11.030869
print(paste("Train Error: ", min(dfit1$cvup)), sep = "") # train error
## [1] "Train Error: 16.08646661956"
print(paste("Test Error: ", mean(abs(pred.test1 - dte1$production))), sep = "") # test error
## [1] "Test Error: 13.4394665091161"We can see that our model does not only depend on the “intercept” also we have seen that test error is not too bad (actually it is better than the train error which we will discuss a little later.)
Before looking at the all results, here is our workflow:
# fit
dfit1 = cv.glmnet(x = as.matrix(dt1[,-c(1:3)]), y = dt1$production, type.measure = "mae")
# train error
error.train1 = min(dfit1$cvup)
# prediction
pred.test1 = predict(dfit1, as.matrix(dte1[,-c(1:3)]), s = "lambda.1se")
# test error
error.test1 = mean(abs(pred.test1 - dte1$production))
# result
res1 = data.table(station = "aliaga", model = "base", train_error = error.train1, test_error = error.test1)
# big result table
dresults = rbind(dresults, res1)Now, let’s look at the results from all 6 stations:
dresults %>% filter(model == "base") # we are filtering it but you don't need to as we just have results of the base models right now.## station model train_error test_error
## 1: aliaga base 16.115594 13.446084
## 2: bares base 8.368780 6.469539
## 3: dinar base 25.054550 19.934007
## 4: geycek base 21.612130 18.296658
## 5: soke base 7.570961 8.648788
## 6: soma base 14.451429 13.176678These results are our base results. We are expecting to outperform the base models in the following sections.
You may have noticed that test error is less than train error in 5 of 6 stations. It is highly likely because of the train/test splits we are using. The train set have observations from january to may which contains more instability compared to test dates (june). This is actually not a good practice but we ensure you that in the “bigger” project we did not have such results. Just for the reduced version, we may compare our models with their train results and use the test results just for checking a possible overfit.
2.1.3 Lagged
Thinking about the nature of wind as a fluid, we think that the wind speed from past and (forecasts) from the future would be also related.
2.1.3.1 Some Details:
A custom function for deriving lagged datasets:
createLagged <- function(data, lags, other.head = 0, other.tail = 0) {
# remove 0 if it exists
# get min/max for removing first/last rows for lag/lead
minlag = min(0,lags)
maxlag = max(0,lags)
startN = 1 - minlag
endN = nrow(data) - maxlag
# seperating others
other = c(other.head, other.tail)
tobe.head = data[(startN:endN),..other.head]
tobe.tail = data[(startN:endN),..other.tail]
dat = data.table() # empty data table
# adding lagged values one by one (column by column)
for(i in lags){
lagdat = data[((startN+i):(endN+i)),-..other]
if(i<0){
lagname = paste('lag',-i,sep='')
colnames(lagdat) = paste(colnames(lagdat),lagname,sep='_')
} else if(i>0){
lagname = paste('lead',i,sep='')
colnames(lagdat) = paste(colnames(lagdat),lagname,sep='_')
}
dat = cbind(dat,lagdat)
}
dat = cbind(tobe.head,dat,tobe.tail)
return(dat)
}# deriving lagged datasets
dt_lag1 = createLagged(data = dt1, lags = c(-1:1), other.head = c(1:3), other.tail = 0)head(dt_lag1)
## date hour production loc1_lag1 loc2_lag1 loc3_lag1 loc4_lag1 loc1 loc2 loc3 loc4 loc1_lead1
## 1: 2017-01-02 1 0 1.490839 1.184061 1.141621 0.4410216 1.700052 1.307793 1.385103 0.6289321 1.910143
## 2: 2017-01-02 2 0 1.700052 1.307793 1.385103 0.6289321 1.910143 1.432977 1.628786 0.8169388 2.120849
## 3: 2017-01-02 3 0 1.910143 1.432977 1.628786 0.8169388 2.120849 1.559263 1.872592 1.0049876 2.225062
## 4: 2017-01-02 4 0 2.120849 1.559263 1.872592 1.0049876 2.225062 1.552668 2.059827 0.9891467 2.332381
## 5: 2017-01-02 5 0 2.225062 1.552668 2.059827 0.9891467 2.332381 1.547259 2.249153 0.9741549 2.442396
## 6: 2017-01-02 6 0 2.332381 1.547259 2.249153 0.9741549 2.442396 1.543049 2.440082 0.9600521 2.311337
## loc2_lead1 loc3_lead1 loc4_lead1
## 1: 1.432977 1.628786 0.8169388
## 2: 1.559263 1.872592 1.0049876
## 3: 1.552668 2.059827 0.9891467
## 4: 1.547259 2.249153 0.9741549
## 5: 1.543049 2.440082 0.9600521
## 6: 1.345251 2.382790 0.6608412
dim(dt_lag1)
## [1] 3598 15
dim(dt_lag2)
## [1] 3598 30Notice that looking at only t-1 and t+1 triples the size of our data. Thinking about a time window of 7 hours may be problematic with bigger datasets.
2.1.3.2 Results:
Our approach is again same. We again used cv.glmnet().
dresults[order(station)] %>% filter(model %in% c("base","lagged"))## station model train_error test_error
## 1: aliaga base 16.115594 13.446084
## 2: aliaga lagged 15.819267 13.337983
## 3: bares base 8.368780 6.469539
## 4: bares lagged 8.115166 6.476837
## 5: dinar base 25.054550 19.934007
## 6: dinar lagged 24.704932 19.893682
## 7: geycek base 21.612130 18.296658
## 8: geycek lagged 21.413134 17.767683
## 9: soke base 7.570961 8.648788
## 10: soke lagged 7.451066 8.643861
## 11: soma base 14.451429 13.176678
## 12: soma lagged 14.248784 13.019966We see better results in all of the stations.
2.1.4 Powered
Recall the formula for “kinetic energy”: \[ E_k = \frac{1}{2} m v^2 \text{, where $m$ is mass and $v$ is velocity.} \]
We can see that square of velocity is related with kinetic energy. Also previous studies show that cube of wind velocity is also correlated with wind energy production however we will be only using squares for sake of simplicity.
Some information about the powered datasets:
head(dt_pow1)
## date hour production loc1 loc2 loc3 loc4 loc1_p2 loc2_p2 loc3_p2 loc4_p2
## 1: 2017-01-02 0 0 1.490839 1.184061 1.141621 0.4410216 2.222600 1.402001 1.303299 0.1945001
## 2: 2017-01-02 1 0 1.700052 1.307793 1.385103 0.6289321 2.890178 1.710323 1.918511 0.3955556
## 3: 2017-01-02 2 0 1.910143 1.432977 1.628786 0.8169388 3.648644 2.053423 2.652944 0.6673890
## 4: 2017-01-02 3 0 2.120849 1.559263 1.872592 1.0049876 4.498000 2.431300 3.506600 1.0100000
## 5: 2017-01-02 4 0 2.225062 1.552668 2.059827 0.9891467 4.950900 2.410778 4.242889 0.9784111
## 6: 2017-01-02 5 0 2.332381 1.547259 2.249153 0.9741549 5.440000 2.394011 5.058689 0.9489778
dim(dt_pow1)
## [1] 3600 11
dim(dt_pow2)
## [1] 3600 21Note that this also doubles the size of our dataset. (what will happen when we will combine lag and power :) )
Skipping the details, here are our results:
dresults[order(station)] %>% filter(model %in% c("base","lagged","powered"))## station model train_error test_error
## 1: aliaga base 16.115594 13.446084
## 2: aliaga lagged 15.819267 13.337983
## 3: aliaga powered 15.929062 13.466087
## 4: bares base 8.368780 6.469539
## 5: bares lagged 8.115166 6.476837
## 6: bares powered 8.187200 6.652139
## 7: dinar base 25.054550 19.934007
## 8: dinar lagged 24.704932 19.893682
## 9: dinar powered 23.836830 20.203461
## 10: geycek base 21.612130 18.296658
## 11: geycek lagged 21.413134 17.767683
## 12: geycek powered 20.743009 18.897305
## 13: soke base 7.570961 8.648788
## 14: soke lagged 7.451066 8.643861
## 15: soke powered 6.883059 8.060418
## 16: soma base 14.451429 13.176678
## 17: soma lagged 14.248784 13.019966
## 18: soma powered 13.869595 13.363834We again see that adding powered features gives an improvement over the base case. Also, there is no visible ranking between lagged and powered models. In some stations lagged model is better, in others powered one significantly improves the performance.
Let’s combine them:
2.1.5 Lagged and Powered
We predict that using both temporal and nonlinear features will give us a better model. However with this section the dimensions will start to increase a lot.
dim(dt_lagpow1)
## [1] 3598 27
dim(dt_lagpow2)
## [1] 3598 57We used to have 4 or 9 features. Now we have 24 and 54 features. And note that we could have used a wider time window and take the cube of these features.
With more detailed features our performance increased in all stations:
dresults[order(station)] %>% filter(model %in% c("base","lagged","powered", "lagged + powered"))## station model train_error test_error
## 1: aliaga base 16.115594 13.446084
## 2: aliaga lagged 15.819267 13.337983
## 3: aliaga powered 15.929062 13.466087
## 4: aliaga lagged + powered 15.613828 13.392450
## 5: bares base 8.368780 6.469539
## 6: bares lagged 8.115166 6.476837
## 7: bares powered 8.187200 6.652139
## 8: bares lagged + powered 8.003236 6.630835
## 9: dinar base 25.054550 19.934007
## 10: dinar lagged 24.704932 19.893682
## 11: dinar powered 23.836830 20.203461
## 12: dinar lagged + powered 23.287946 19.776765
## 13: geycek base 21.612130 18.296658
## 14: geycek lagged 21.413134 17.767683
## 15: geycek powered 20.743009 18.897305
## 16: geycek lagged + powered 20.441426 18.363553
## 17: soke base 7.570961 8.648788
## 18: soke lagged 7.451066 8.643861
## 19: soke powered 6.883059 8.060418
## 20: soke lagged + powered 6.789591 8.074436
## 21: soma base 14.451429 13.176678
## 22: soma lagged 14.248784 13.019966
## 23: soma powered 13.869595 13.363834
## 24: soma lagged + powered 13.551715 13.056804
## station model train_error test_error2.1.6 Polynomial
As we have shown that all these derived features improve our performance, also again thinking about the fluid dynamics, we think that polynomials of wind speed features would give another nonlinear and temporal perspective.
We will be using polym() function for creating our polynomial data:
dt_poly1 = cbind(dt1[,1:3], polym(as.matrix(dt1[,-c(1:3)]),degree = 2, raw = TRUE)) # raw=TRUE means:: dont' scaleChecking the dimensions:
head(dt_poly1)
## date hour production 1.0.0.0 2.0.0.0 0.1.0.0 1.1.0.0 0.2.0.0 0.0.1.0 1.0.1.0 0.1.1.0 0.0.2.0
## 1: 2017-01-02 0 0 1.490839 2.222600 1.184061 1.765245 1.402001 1.141621 1.701973 1.351750 1.303299
## 2: 2017-01-02 1 0 1.700052 2.890178 1.307793 2.223317 1.710323 1.385103 2.354748 1.811428 1.918511
## 3: 2017-01-02 2 0 1.910143 3.648644 1.432977 2.737190 2.053423 1.628786 3.111214 2.334013 2.652944
## 4: 2017-01-02 3 0 2.120849 4.498000 1.559263 3.306960 2.431300 1.872592 3.971484 2.919862 3.506600
## 5: 2017-01-02 4 0 2.225062 4.950900 1.552668 3.454782 2.410778 2.059827 4.583243 3.198228 4.242889
## 6: 2017-01-02 5 0 2.332381 5.440000 1.547259 3.608798 2.394011 2.249153 5.245881 3.480023 5.058689
## 0.0.0.1 1.0.0.1 0.1.0.1 0.0.1.1 0.0.0.2
## 1: 0.4410216 0.6574921 0.5221967 0.5034797 0.1945001
## 2: 0.6289321 1.0692175 0.8225132 0.8711358 0.3955556
## 3: 0.8169388 1.5604695 1.1706544 1.3306185 0.6673890
## 4: 1.0049876 2.1314268 1.5670396 1.8819315 1.0100000
## 5: 0.9891467 2.2009124 1.5358163 2.0374714 0.9784111
## 6: 0.9741549 2.2721002 1.5072702 2.1910234 0.9489778
dim(dt_poly1)
## [1] 3600 17
dim(dt_poly2)
## [1] 3600 57With polynomials, the dimensions increase even faster. (It is a fun exercise to write a function for calculating how many features there will be for different degrees of polynomials.)
Not to bore you with details our results are as follows:
dresults[order(station)] %>% filter(model %in% c("base","lagged","powered", "lagged + powered", "poly(2)"))## station model train_error test_error
## 1: aliaga base 16.115594 13.446084
## 2: aliaga lagged 15.819267 13.337983
## 3: aliaga powered 15.929062 13.466087
## 4: aliaga lagged + powered 15.613828 13.392450
## 5: aliaga poly(2) 15.284348 13.547195
## 6: bares base 8.368780 6.469539
## 7: bares lagged 8.115166 6.476837
## 8: bares powered 8.187200 6.652139
## 9: bares lagged + powered 8.003236 6.630835
## 10: bares poly(2) 8.024843 6.630476
## 11: dinar base 25.054550 19.934007
## 12: dinar lagged 24.704932 19.893682
## 13: dinar powered 23.836830 20.203461
## 14: dinar lagged + powered 23.287946 19.776765
## 15: dinar poly(2) 23.456418 20.640282
## 16: geycek base 21.612130 18.296658
## 17: geycek lagged 21.413134 17.767683
## 18: geycek powered 20.743009 18.897305
## 19: geycek lagged + powered 20.441426 18.363553
## 20: geycek poly(2) 20.410482 19.122731
## 21: soke base 7.570961 8.648788
## 22: soke lagged 7.451066 8.643861
## 23: soke powered 6.883059 8.060418
## 24: soke lagged + powered 6.789591 8.074436
## 25: soke poly(2) 6.717682 7.934136
## 26: soma base 14.451429 13.176678
## 27: soma lagged 14.248784 13.019966
## 28: soma powered 13.869595 13.363834
## 29: soma lagged + powered 13.551715 13.056804
## 30: soma poly(2) 13.487807 13.253549
## station model train_error test_errorWe were hoping to outperform the powered models. We sure did. However we see that in some stations lagged+powered models are better. So we think that using polynomial features from the past can improve our model even more!
2.1.7 Polynomial then Lagged
Did you notice something weird in the title? Why didn’t we just say Polynomial + Lagged or Lagged + Polynomial, just like in Lagged + Powered?
Because now we are only interested in the polynomial combinations of our features and their results in different time indices. It is much easier than checking the polynomial combinations of lagged results.
colnames(dt_polylag1)
## [1] "date" "hour" "production" "1.0.0.0_lag1" "2.0.0.0_lag1" "0.1.0.0_lag1" "1.1.0.0_lag1"
## [8] "0.2.0.0_lag1" "0.0.1.0_lag1" "1.0.1.0_lag1" "0.1.1.0_lag1" "0.0.2.0_lag1" "0.0.0.1_lag1" "1.0.0.1_lag1"
## [15] "0.1.0.1_lag1" "0.0.1.1_lag1" "0.0.0.2_lag1" "1.0.0.0" "2.0.0.0" "0.1.0.0" "1.1.0.0"
## [22] "0.2.0.0" "0.0.1.0" "1.0.1.0" "0.1.1.0" "0.0.2.0" "0.0.0.1" "1.0.0.1"
## [29] "0.1.0.1" "0.0.1.1" "0.0.0.2" "1.0.0.0_lead1" "2.0.0.0_lead1" "0.1.0.0_lead1" "1.1.0.0_lead1"
## [36] "0.2.0.0_lead1" "0.0.1.0_lead1" "1.0.1.0_lead1" "0.1.1.0_lead1" "0.0.2.0_lead1" "0.0.0.1_lead1" "1.0.0.1_lead1"
## [43] "0.1.0.1_lead1" "0.0.1.1_lead1" "0.0.0.2_lead1"
dim(dt_polylag1)
## [1] 3598 45
dim(dt_polylag2)
## [1] 3598 165Now the dimensions of our datasets are tripled compared to the polynomial data. It is again a huge increase but the following section will be scarier.
Looking at the results:
dresults[order(station)] %>% filter(model %in% c("base","lagged","powered", "lagged + powered", "poly(2)", "poly(2) > lag"))## station model train_error test_error
## 1: aliaga base 16.115594 13.446084
## 2: aliaga lagged 15.819267 13.337983
## 3: aliaga powered 15.929062 13.466087
## 4: aliaga lagged + powered 15.613828 13.392450
## 5: aliaga poly(2) 15.284348 13.547195
## 6: aliaga poly(2) > lag 15.061376 13.356530
## 7: bares base 8.368780 6.469539
## 8: bares lagged 8.115166 6.476837
## 9: bares powered 8.187200 6.652139
## 10: bares lagged + powered 8.003236 6.630835
## 11: bares poly(2) 8.024843 6.630476
## 12: bares poly(2) > lag 7.841424 6.523750
## 13: dinar base 25.054550 19.934007
## 14: dinar lagged 24.704932 19.893682
## 15: dinar powered 23.836830 20.203461
## 16: dinar lagged + powered 23.287946 19.776765
## 17: dinar poly(2) 23.456418 20.640282
## 18: dinar poly(2) > lag 22.247506 19.667580
## 19: geycek base 21.612130 18.296658
## 20: geycek lagged 21.413134 17.767683
## 21: geycek powered 20.743009 18.897305
## 22: geycek lagged + powered 20.441426 18.363553
## 23: geycek poly(2) 20.410482 19.122731
## 24: geycek poly(2) > lag 20.116372 18.523240
## 25: soke base 7.570961 8.648788
## 26: soke lagged 7.451066 8.643861
## 27: soke powered 6.883059 8.060418
## 28: soke lagged + powered 6.789591 8.074436
## 29: soke poly(2) 6.717682 7.934136
## 30: soke poly(2) > lag 6.647820 7.965204
## 31: soma base 14.451429 13.176678
## 32: soma lagged 14.248784 13.019966
## 33: soma powered 13.869595 13.363834
## 34: soma lagged + powered 13.551715 13.056804
## 35: soma poly(2) 13.487807 13.253549
## 36: soma poly(2) > lag 13.065487 12.976022
## station model train_error test_errorGreat results! We have seen nice improvements in all stations.
Now we have another question in our mind: can polynomial combinations of features from different times be useful ,too, for our forecasting?
2.1.8 Polynomial of Lagged (Lagged then Polynomial)
Now we will combine features from different times. As we are looking at a time window of only size 3 and on 4 locations, things should not be very big. However with a wider time window and more locations this exercise may only make things complicated.
While trying to create polynomials for stations 2 to 6 we get the following error: > Error: vector memory exhausted (limit reached?)
As we can see our memory is not enough. This problem can be solved but we will stop there for now and in the next chapter we will try to handle this.
However we were able to create the dataset for station1, let’s see its performance:
fit.lagpoly1 = cv.glmnet(x = as.matrix(dt_lagpoly1[,-c(1:3)]), y = dt_lagpoly1$production, type.measure = 'mae')
error.train1 = min(fit.lagpoly1$cvup)
pred.test1 = predict(fit.lagpoly1, as.matrix(dte_lagpoly1[,-c(1:3)]), s = "lambda.1se")
error.test1 = mean(abs(pred.test1 - dte_lagpoly1$production))
res1 = data.table(station = "aliaga", model = "lag > poly(2)", train_error = error.train1, test_error = error.test1)
rbind(dresults %>% filter(station == "aliaga") %>% filter(model %in% c("base","lagged","powered", "lagged + powered", "poly(2)", "poly(2) > lag")), res1)## station model train_error test_error
## 1: aliaga base 16.11559 13.44608
## 2: aliaga lagged 15.81927 13.33798
## 3: aliaga powered 15.92906 13.46609
## 4: aliaga lagged + powered 15.61383 13.39245
## 5: aliaga poly(2) 15.28435 13.54719
## 6: aliaga poly(2) > lag 15.06138 13.35653
## 7: aliaga lag > poly(2) 14.90441 13.37074We were able to reduce our error, once more. This is a good motivation for us to move on. Before finalizing this section let’s try one more thing.
2.1.9 Polynomial of degree 3
Just for future motivation, lets try the base model of polynomial with degree 3:
dresults[order(station)]## station model train_error test_error
## 1: aliaga base 16.115594 13.446084
## 2: aliaga lagged 15.819267 13.337983
## 3: aliaga powered 15.929062 13.466087
## 4: aliaga lagged + powered 15.613828 13.392450
## 5: aliaga poly(2) 15.284348 13.547195
## 6: aliaga poly(2) > lag 15.061376 13.356530
## 7: aliaga poly(3) 13.932274 12.602298
## 8: bares base 8.368780 6.469539
## 9: bares lagged 8.115166 6.476837
## 10: bares powered 8.187200 6.652139
## 11: bares lagged + powered 8.003236 6.630835
## 12: bares poly(2) 8.024843 6.630476
## 13: bares poly(2) > lag 7.841424 6.523750
## 14: bares poly(3) 7.366621 6.156572
## 15: dinar base 25.054550 19.934007
## 16: dinar lagged 24.704932 19.893682
## 17: dinar powered 23.836830 20.203461
## 18: dinar lagged + powered 23.287946 19.776765
## 19: dinar poly(2) 23.456418 20.640282
## 20: dinar poly(2) > lag 22.247506 19.667580
## 21: dinar poly(3) 21.762741 18.844985
## 22: geycek base 21.612130 18.296658
## 23: geycek lagged 21.413134 17.767683
## 24: geycek powered 20.743009 18.897305
## 25: geycek lagged + powered 20.441426 18.363553
## 26: geycek poly(2) 20.410482 19.122731
## 27: geycek poly(2) > lag 20.116372 18.523240
## 28: geycek poly(3) 18.541052 17.200868
## 29: soke base 7.570961 8.648788
## 30: soke lagged 7.451066 8.643861
## 31: soke powered 6.883059 8.060418
## 32: soke lagged + powered 6.789591 8.074436
## 33: soke poly(2) 6.717682 7.934136
## 34: soke poly(2) > lag 6.647820 7.965204
## 35: soke poly(3) 6.599493 7.894552
## 36: soma base 14.451429 13.176678
## 37: soma lagged 14.248784 13.019966
## 38: soma powered 13.869595 13.363834
## 39: soma lagged + powered 13.551715 13.056804
## 40: soma poly(2) 13.487807 13.253549
## 41: soma poly(2) > lag 13.065487 12.976022
## 42: soma poly(3) 12.364350 11.836208
## station model train_error test_errorWow! Our poly(3) models are outperforming poly(2)>lag models. However combination of lag and poly(3) would again create dimension problems. So we stop here.
# dlevel_order is factorized version of "models". needed for ordered model names
plot1 = ggplot(dresults) +
geom_point(mapping = aes(x = dlevel_order, y = train_error, color = station)) +
labs(x = "Models", y = "Mean Absolute Error", title = "Final Results", subtitle = "Error Comparison")
print(plot1)
2.1.10 Final Words
In this section, we have shown that there are many ways to derive new features that increases performance. However with more features more problems occur, especially because of size. In the following section we will try to keep our dimensions low while trying to make use of these new features.
We will talk about:
- Sparse Polynomials
- Principal Component Analysis (PCA)
- Kronecker PCA
- Sparse Kronecker PCA
- Kernel PCA …