Advantages and disadvantages of naive forecasting method

In time series forecasting, the naive forecast - where the forecast for all future periods is set equal to the value from the current period - is the most simple of all forecasting methods. It generally gets a bad wrap, due in large part to this simplicity - how can such a basic methodology be any good? In many applications, its reputation is often justified by - at best - average performance, especially when forecasting many periods ahead. Forecasting inventories (such as reservoir storage levels) is one obvious example. Forecasting inventory levels in 6 months to be equal to todays leaves much information on the table that could easily be used to get a better, more accurate forecast.

However, in a number of applications the naive forecast performs surprisingly well. One example of this are market prices, of which power market prices are one example. Indeed, more complex forecasting methods often struggle to do better (and sometimes do even worse) than the naive in these cases.

Why might this be? A (partial) answer lies in the markets themselves. In well functioning markets, the market price at any time represents the summation of all information available to the market. It can be thought of as an aggregate market “view” of what the price is, given such “underlying” information. For the next period, assuming relatively moderate changes in the underlying information, we may reasonably expect relatively moderate changes in market prices. In such a case, the naive forecast can be a fairly good short term predictor.

A Simple Example

To illustrate this, lets look at the day-ahead market power price in the Nordic markets. To make things a little simpler, we drop the weekend data, and build a simple naive model to predict the next-weekday’s system spot price:

forecast(t+1) = observed(t)

Thus, for example, the forecast for Thursday is Wednesday’s price, and for Monday is the previous Friday’s price. The predictions for our test dataset (May-August 2021) are shown below.

We can test this against a simple feed forward neural network model, trained on a set of historical price features that exhibit some (auto)correlation with current price. The forecasts from this model for our test data set are given below.

There is very little difference in the forecasts of the two models - indeed, the NN model has essentially learned a naive forecast with occasional small modifications. We can see this in more detail examining a set of forecast metrics for the two models:

This is not an atypical result in time-series forecasting using standardised machine learning - such models that are based on recent historical data and historical data patterns end up learning essentially a simple ARIMA-type model. The naive forecast is of course essentially already a simple autoregressive model, and thus performs fairly similarly to these simple ML alternatives.

This highlights a very common issue with many ML time-series forecasting models (in power markets and elsewhere) that appear fairly accurate, but are in reality offering little more (or even no more) than the naive forecast. And it raises the obvious next question is: how do we beat the naive forecast? We will tackle this issue in our next post.

Dirk Soehnholz and Marcus Burkert find that ‘naïve’ strategic asset allocation offers improved risk/return characteristics over traditional ‘pseudo-optimised’ asset allocation

The current financial market crisis again shows that asset allocation is extremely important. But how much do we really know about the optimal asset allocation?

Most professional investors agree that asset allocation accounts for the majority of investment success. They usually refer to Brinson, Beebower and Hood (1), 1986, which found that asset allocation accounts for about 90% of investment results. But this study only makes a statement about the variation in total pension plan returns and not about the impact of asset allocation on the total performance of the portfolio. Also, the study analyses a very limited number of asset classes, and it does not explicitly differentiate between asset allocation, manager or fund selection and security/stock selection (see Nuttal (2)).

Existing studies do not really help to explain some interesting institutional investor success stories such as US university endowment successes, like Yale’s.

We could not find good research on different asset allocation models that explicitly compare the benefits of increasing asset class diversification in a broad and open way. Most studies start with allocations to cash, bonds and equities and then optimise these portfolios. Then, the effects of allocation to one additional asset class, mostly real estate as the next most likely one, are analysed. The results are always quite similar. One additional asset class - independently of which one is chosen - improves portfolio performance. However, we could find an answer neither to the question of how many asset classes to invest in overall nor to the ideal order of adding classes. We suggest the application of a so-called zero-based budgeting (3) approach which can be termed zero-based asset allocation.

Since most asset allocation or liability modelling is done on the basis of return, risk and correlation prognosis, we were initially interested whether a naïve asset allocation may improve risk-return characteristics of portfolios.

Therefore, we developed a simple model in which we start with the most traditional asset classes that are increased in a naïve way over time, varying the additional asset classes.

One of our hypothetical assumptions was that the impact of diversification decreases above a certain number of asset classes, even though we do not explicitly consider transaction costs. We decided to use euro money market rates, European government bonds and European equities for the three basic or initial asset classes cash, bonds and equities. We also decided to focus on risk adjusted returns. We chose the Sharpe ratio as the target function, although we acknowledge the problems of this indicator (for example, the assumption of normal return distribution). It has to be stressed that a sort of ‘risk minimising’ approach (such as is advocated by many regulators) or a ‘return maximising’ approach (especially for shorter time periods, but also the usage of historical rather than expected data) may lead to very different results. However, we believe that the risk/return ratio optimisation is the most rational approach for most institutional investors, at least over the medium to long term.

First, we had to select the additional asset classes for analysis. In order to reduce complexity, we decided to use only 10 asset classes in total which meant adding convertibles, corporate bonds, asset backed securities, real estate, commodities, hedge funds, and private equity to the three initial classes. Our choice is based on what we find as main allocations in institutional investor portfolios. We are aware that hedge funds, for example, are not asset class as such. However, we used a coherent and different set of risk, return and correlation data similar to the other asset classes we used for the analysis, for hedge funds, as well.

For these, we used long-term volatility, return and correlation expectations derived from our proprietary capital market model. The co-variances in our capital market model change over time and therefore cannot easily be made transparent here. We decided not to use historic returns, risks and correlations, because - dependent on the time frame chosen - they usually result in momentum driven asset allocations. This means that asset classes which had ‘good’ model characteristics in the past would also receive high allocations for the future. Also, for many asset classes there are not enough available reliable and representative reference indices and past data.

We acknowledge that our approach is very sensitive to the input. For example, we use a high correlation between globally listed equities and private equities as well as relatively low private equity returns and high volatility and are thus theoretically compensating for the illiquidity of private equity. This leads to allocation disadvantages for this asset class which can be seen in some of the results mentioned below. We also have to point out that the results will obviously not hold in every market environment - especially in the short term. It is well known that in times of general crisis correlations tend to increase and diversification effects tend to disappear for many asset classes at least temporarily. Usually, we use a complex multi-period, abnormally distributed, capital market model for our analysis. In this case we wanted to make the results easier to replicate and decided to use a standard and simple one period Markowitz optimisation model, which is still used by many investors. We acknowledge that changes in the assumptions and models may have impacts on the results.

The next question concerned the sequence of adding new asset classes. We decided to first model a ‘Naïve Basic Three’ portfolio (cash, European bonds and equities) plus one additional asset class, chosen from the different asset classes of the range mentioned above.

It is no great surprise that in our setting four equally weighted asset classes result in a better Sharpe ratio than three in almost all cases. Similarly, we achieve a better Sharpe ratio if we switch from a naïve (equally weighted) four-asset class portfolio to a five-asset class portfolio, and from a five to a six-asset class portfolio. The potential increase in the Sharpe ratio declines with the number of asset classes, but negative ‘over-diversification’ is very unlikely.

In a second step, we optimised the Sharpe ratio of the Basic Three portfolio with a simple Markowitz model. Then we first fixed the allocation ratio of the three basic asset classes before we reduced the overall allocation to them in the total portfolio and added one new asset class.

Again, all additional asset classes improve the Sharpe ratio of the portfolio. Only private equity allocations above 10% worsen the Sharpe ratio compared to the optimised basic three asset class case.

To state it technically: Naïve diversification to x+1 asset classes improved the Sharpe ratio of a portfolio compared to x asset classes in most cases in our simple model. In the equally weighted model with three asset classes, cash will generate a Sharpe ratio of approximately zero. Other asset classes have higher expected Sharpe ratios. Increasing the number of asset classes might lead to better results due to the fact that the zero Sharpe ratio category cash is reduced. Starting from an already optimised allocation, further naïve diversification showed positive effects in all cases if the allocation to a new asset class did not become predominant. Also, the overall Sharpe ratio of the optimised plus naïvely diversified portfolio improved in all cases compared to the overall naïvely diversified portfolio with the same asset classes.

 Overall, in our set up we find strong support for our hypothesis that:

1. ‘Naïve’ diversification improves the risk/return characteristics of a portfolio

2. The more asset classes already in a portfolio, the lower is the effect of diversification,

3. If the starting point is not a naïve diversification, but an already optimised diversification, the probability of successful further naïve diversification rises even more.

Dr Dirk Soehnholz is managing partner at Feri Institutional Advisors and Marcus Burkert is head of consulting at Feri Institutional Advisors and managing director at Heubeck-Feri Pension Asset Consulting. A German version of this article was first published earlier this year

1 Brinson, Gary P. / Hood, L. Randolph / Beebower, Gilbert L.

2 Nuttal, John

3 The intention of zero-base budgeting: The constant orienting on prior year figures with only marginal adjustments up or down shall be resolved. Source: www.wirtschaftslexikon24.net/d/zero-base-budgeting/zero-base-budgeting.htm