Aggregation of Forecasts and Recommendations of Financial Analysts - - PowerPoint PPT Presentation

Aggregation of Forecasts and Recommendations

• f Financial Analysts in the Framework
• f Evidence Theory

Ekaterina Kutynina Alexander Lepskiy

National Research University - Higher School of Economics, Moscow, Russia

The 10th conference of the European Society for Fuzzy Logic and Technology - EUSFLAT 2017, September 11 - 15, 2017, Warszawa, Poland

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 1 / 24

Preamble

The Problem of Aggregation of Recommendations and Forecasts

Among the tasks associated with evaluating the recommendations of financial analysts, the important challenge is to aggregate recommendations and forecasts. We have: a number of financial analysts’ recommendations and forecasts regarding the expected potential for growth in the prices of shares

• f a particular company (”sell”, ”hold”, ”buy” + target prices of

shares); the data on the real value of the shares of this company in the certain period in the past; the history of financial analysts’ recommendations in the past. How we can aggregate financial analysts’ recommendations in the best way?

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 2 / 24

Preamble

Some Studies on the Aggregation of Analysts’ Recommendations

in [Berkman &Yang 2016] it was demonstrated that taking into account the recommendations aggregated at the country level improves profitability on the international stock market; in [Howe et al. 2009] it is shown that the inclusion of changes in aggregated recommendations on average positively affects for income and profit; in [Huang et al. 2009] was showed the importance of the recommendations combining and forecasts of the target price for building a profitable investment strategy; in [Kim et al. 2001] it was argued that the average value is inefficient for aggregating forecasts. This inefficiency increases with the growth in the number of forecasts in aggregation. Also there were offered some procedures for selecting ”good” forecasts for aggregation.

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 3 / 24

Preamble

Evidence Theory in Financial and Economic Analysis

A weighted consensus forecast is usually used as a general method of

• aggregation. The actual problem is to choose such aggregation

procedures, which most fully take into account information from individual analysts (uncertainty), the history of their forecasts (reliability), their correlation (conflict), etc. All the noted features can be described within the theory of evidence. The theory of evidence is widely used in financial and economic analysis, for example: in forecasting of investments’ profitability on the basis of interval expert assessments [Utkin 2006]; in marketing analysis of data [Kanjanatarakul et al. 2014]; in forecasting of income on the stock market [Autchariyapanitkul et al. 2014]; in foresight studies [Xu et al. 2014], etc.

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 4 / 24

Preamble

Outline of Presentation

the background of evidence theory; the survey of financial analysts’ data on the Russian Stock Market; the construction of bodies of evidence of financial analysts’ and implementation of different aggregation procedure; experimental results.

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 5 / 24

Evidence Theory and Combining Rules

Background of Evidence Theory

Ω be a some universal set of all possibility values of experimental results, P(Ω) be a powerset of Ω; a mass function m : P(Ω) → [0, 1],

A∈P(Ω) m(A) = 1;

A ⊆ Ω is called a focal element if m(A) > 0; the pair F = (A, m) is called a body of evidence, BE; F(Ω) be a set of all BE on Ω. BE is said to be categorical (is denoted as FA = (A, 1)) if it has

• nly one focal element; BE FΩ = (Ω, 1) is said to be vacuous;

if Fj = (Aj, mj) ∈ F(Ω) and

j αj = 1, αj ∈ [0, 1] ∀j, then

F =

j αjFj = (A, m) ∈ F(Ω), where A = j Aj,

m(A) =

j αjmj(A); we have F = A∈A m(A)FA ∀F = (A, m);

BE is said to be simple if it has the form F ω

A = (1 − ω)FA + ωFΩ,

ω ∈ [0, 1].

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 6 / 24

Evidence Theory and Combining Rules

Combining Rules

Suppose there are two independent groups of experts who provide their forecasts, given in the form of two bodies of evidence F1 = (A1, m1) and F2 = (A2, m2) on Ω. We want to combine these two BE in one BE. There are a few popular combining rules. The mass function m = m1 ⊕ m2 of new BE obtained with the help of Dempster’s rule is calculated by the formula m(A) = 1 1 − K

• B∩C=A

m1(A)m2(B), A = ∅, where K = K(F1, F2) = m(∅) =

B∩C=∅ m1 (A) m2 (B). The value K

characterizes the amount of conflict in two information sources. If K = 1 then it means that information sources are absolutely conflict and Dempster’s rule cannot be applied. Since the combining rule ⊕ is an associative operation, then any finite number of BE can be combined.

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 7 / 24

Evidence Theory and Combining Rules

Boundaries of Expectation and Discounting

Let all focal elements of BE F = (A, m) are bounded sets in R. Then the lower and upper boundaries of expectation of belonging of true alternative can be calculated: E[F] =

• A∈A m(A) inf{A},

E[F] =

• A∈A m(A) sup{A}.

The degree of reliability of information source be taken into account with the help of discount coefficient α ∈ [0, 1] [Shafer, 1976]: mα(A) = (1 − α)m(A) ∀A = Ω, mα(Ω) = α + (1 − α)m(Ω). If α = 1, then it means that information source is absolutely not

• reliable. If α = 0, then information source is absolutely reliable. The

some combining rule applied after discounting of initial BE.

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 8 / 24

The Survey Data

The Survey Data

As a rule, an expert estimation of the share price behaviour consists of two indicators: the target price and direct recommendation. Analysts’ information: the target price is the share price expected by the expert at the end of the forecast period; recommendations of analysts can take the values ”sell”, ”hold”, ”buy”; 7 Russian banks and 3 analytical companies that provide their annual forecasts for 16 Russian companies represented on the Russian stock market during January 2010 – May 2016; the data on the real value of the shares of these companies in the period from January 2010 to May 2016.

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 9 / 24

The Survey Data

Determination of Focal Elements

• 1. We used the relative target price

Crv( stock, t )= target price of the share stock actual price of stock on the date of the forecast t.

• 2. Boundary values of focal elements are calculated as a solution to the

problem of minimization an error in the incorrect classification of recommendations

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 10 / 24

Determination of BE

Determination of Bodies of Analysts’ Evidence

During one year one analytical company gives several

• recommendations. For each analytical company i and each stock

BE can be constructed Fi,stock = (Ai,stock, mi,stock). Each BE has not more than three focal elements Si,stock, Hi,stock, Bi,stock, and mass functions mi,stock(A) equal to relative frequency

• f recommendation.

The set Ω is added to the set focal elements in the case of discounting.

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 11 / 24

Determination of BE

The Problem of Finding the Optimal BE

Let we have n categorical BE (recommendation of i-th source in during a year) that ordered by the time FAs, where As ∈ {Si,stock, Hi,stock, Bi,stock}, s = 1, . . . , n. We will consider a BE F(α1, . . . , αn) = ⊕n

s=1F αs As,

1 ≥ α1 ≥ · · · ≥ αn ≥ 0 for finding of recommendation of i-th source on the end a year with account of revision of forecasts. Here F αs

As = (1 − αs)FAs + αsFΩ.

Criteria for optimization C(α1, . . . , αn) = (E0[F(α1, . . . , αn)] − p)2 → min, where p is an actual last ”pre-forecast” relative price of the share, E0[F] = 1

2 (E[F] + E[F]) is the middle value of the interval of

expectation of the forecast price.

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 12 / 24

Determination of BE

Example

Let n = 4, F1 = F2 = F4 = FS (”sell”) and F3 = FH (”hold”). Then F(α1, α2, α3, α4)=F α1

S ⊕F α2 S ⊕F α3 H ⊕F α4 S =m(S)FS+m(H)FH+m(Ω)FΩ.

The conflict of discounting BE is equal K =K

• F α1

S , F α2 S , F α3 H , F α4 S

• =

=(1 − α3)(1 − α1α2α4); the values of a mass function are equal: m(S)= α3(1 − α1α2α4) 1 − K , m(H)= α1α2(1 − α3)α4 1 − K , m(Ω)= α1α2α3α4 1 − K . Consequently, we have C(α1, α2, α3, α4) = (E0[F(α1, α2, α3, α4)] − p)2 = α3(1 − α1α2α4)S0 + α1α2(1 − α3)α4H0 + α1α2α3α4Ω0 α3 + α1α2α4 − α1α2α3α4 − p 2 , where S0, H0, Ω0 are middles of intervals of relative prices.

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 13 / 24

Determination of BE

We will solve a problem C(α1, α2, α3, α4) → min subject to the condition 1 ≥ α1 ≥ α2 ≥ α3 ≥ α4 ≥ 0 and we will find the optimal evidence F. For example, if S0 = 0.7, H0 = 1.1, Ω0 = 0.9 and p = 0.8, then we

• btain optimal coefficients α1 = α2 = 1, α3 ≈ 0.34, α4 ≈ 0.13 and

F ≈ 0.7FS + 0.2FH + 0.1FΩ. If we combines I bodies of evidence of the form m(S)FS + m(H)FH + m(B)FB + m(Ω)FΩ, then we will obtain a new evidence, in which there can be up to 4I of the focal elements.

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 14 / 24

Different Combining Strategies

Different Combining Strategies

It is necessary to define the rules according to which the sources for combining will be selected. Two alternative rules for selecting sources were considered:

• 1. All sources were ranked by the increase in the degree of conflict and

the combination of evidence began with a pair of the least conflicting

• sources. The algorithm of ranking is:

to choose the pair of BE (Fi, Fj) = arg min

F ′=F ′′ K(F ′, F ′′);

to choose three BE (Fi, Fj, Fs) = arg min

F=Fi,F=Fj

K(Fi, Fj, F) and so on, where K(Fi1, ..., Fis) =

Ai1∩...∩Ais=∅ mi1(Ai1)...mis(Ais);

the algorithm stops when K(Fi, Fj, ..., Fs) > K0.

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 15 / 24

Different Combining Strategies

• 2. For each data source i the degree of forecast’s reliance was

evaluated for each share stock basing on data of a previous period δi,stock = 1 N

• t

|Crvreal(stock, t) − Crvforecast(stock, t)| max{Crvreal(stock, t), Crvforecast(stock, t)}, where N is the number of forecasts during the period, Crvreal(stock, t) is the actual relative price, Crvforecast(stock, t) is the forecasted relative price. All sources were ranked by the increase δi,stock and the combination of evidence began with a pair of the most reliable sources. If the data from the first two sources of the ranked row were in conflict, then the first source from the row was selected. Further, the following in turn among sources, which did not in the conflict with already combined, was chosen.

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 16 / 24

Different Combining Strategies

The value δi,stock ∈ [0, 1] was chosen as a coefficient of discounting. In case when in current period one of data sources represents forecasts for the first time, it is possible to consider several strategies:

1 (optimistic) this scenario assumes a high degree of belief to new

recommendations; the discounting coefficient can be equal, for example, δi,stock = 0.1;

2 (neutral) this scenario assumes a smaller belief to new

recommendations, δi,stock = 0.5;

3 (pessimistic) this scenario assumes a very low degree of belief to

new recommendations, δi,stock = 0.75. Rather ”old” sources of information we will keep to two strategies:

1 ”with censorship” – for a combination there were chosen only

those sources, which coefficient of discounting is less than a threshold value α;

2 ”without censorship” – in case of a combination there were used

all available sources.

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 17 / 24

Forecast Error Estimation

Forecast Error Estimation

For calculation of errors of the forecasts the functionality of mean absolute error (MAE) was used MAEstock(Crvforecast) = 1 N

• t

|Crvreal(stock, t) − Crvforecast(stock, t)|, in which forecast value of a relative share price is calculated as follows: consensus forecast (CF) or weighted consensus forecast (WCF, with weights that are equal to reliability of recommendations in previous periods); E[F], E[F], E0[F] = 1

2

• E[F] + E[F]
• values received by means of

the Dempster’s rule with/without discounting, with/without censorship, optimistic-neutral-pessimistic attitude to new sources.

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 18 / 24

The Results

The Results

Example of forecasting for the share price of the Transneft company (TRNFP). Figure show the results of applying the Dempster’s rule with the choice of the least conflicting sources.

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 19 / 24

The Results

Figure show the results of combining with discounting in cases of neutral attitude to new sources and ”with censorship” α = 0.75.

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 20 / 24

The Results

The mean absolute errors for some strategies of recommendations’ combining are presented in Table.

MAEstock WCF CF E, OSWC E, OSWC E0, OSWC E, NSWC E, NSWC E0, NSWC E, CLCS E, CLCS E0, CLCS GAZP 0,941 0,456 0,686 0,715 0,653 0,745 0,772 0,711 0,275 0,359 0,308 LKOH 0,499 0,27 0,19 0,99 0,59 0,234 0,918 0,524 0,193 0,309 0,263 ROSN 0,587 0,412 0,153 0,486 0,302 0,194 0,529 0,265 0,11 0,257 0,191 SBER 0,55 0,413 0,137 0,539 0,33 0,178 0,633 0,349 0,116 0,319 0,233 MAGN 0,298 0,205 0,385 0,937 0,661 0,347 1,014 0,681 0,345 0,47 0,422 SNGSP 0,485 0,332 0,327 0,84 0,546 0,243 1,037 0,598 0,263 0,405 0,33 GMKN 0,512 0,46 0,382 0,673 0,447 0,349 0,672 0,345 0,283 0,314 0,267 VTBR 0,637 0,249 0,451 0,376 0,344 0,525 0,273 0,286 0,269 0,213 0,207 TRNFP 0,33 0,234 0,146 0,356 0,18 0,199 0,416 0,18 0,119 0,215 0,141 TATN 0,568 0,3 0,198 0,998 0,589 0,183 1,027 0,596 0,2 0,421 0,331 MTSS 0,516 0,371 0,268 0,352 0,249 0,39 0,397 0,259 0,214 0,205 0,210 CHMF 0,311 0,203 0,17 0,369 0,227 0,196 0,405 0,222 0,192 0,24 0,225 ALRS 0,216 0,14 0,119 0,204 0,116 0,156 0,308 0,16 0,115 0,162 0,11 NVTK 0,236 0,395 0,28 0,396 0,196 0,28 0,495 0,149 0,273 0,198 0,178 AFLT 0,123 0,033 0,477 0,186 0,222 0,552 0,365 0,193 0,376 0,076 0,218 URKA 0,6 0,523 0,504 0,216 0,357 0,654 0,285 0,339 0,338 0,244 0,271 MAE 0,463 0,312 0,305 0,539 0,376 0,339 0,597 0,366 0,23 0,275 0,244

WCF – weighted consensus forecast; CF – consensus forecast; the Dempster’s rule with discounting and (a) optimistic scenario without censorship (OSWC); (b) neutral scenario without censorship (NSWC); (c) with the choice of the least conflicting sources (CLCS).

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 21 / 24

Summary and Conclusion

Summary and Conclusion

1 in most cases forecasts (with the exception of two shares) the

aggregating methods with using the Dempster’s rule turned out to be more accurate than the consensus forecast;

2 in most cases forecasts E[F] and E0[F], obtained by using the

Dempster’s rule without discounting with the choice of the least conflicting sources, turned out to be more accurate than similar estimates with discounting under a neutral or pessimistic attitude to new sources;

3 for at least half of the shares, the most accurate estimate was E[F]. Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 22 / 24

References

Autchariyapanitkul, K., Chanaim, S., Sriboonchitta, S., Denoeux, T.: Predicting stock returns in the capital asset pricing model using quantile regres. and belief func. In: F. Cuzzolin (ed.) BELIEF 2014, LNCS, vol. 8764, pp. 219–226, Springer (2014). Berkman, H., Yang, W.: Analyst recommendations and international stock market

• returns. July 1, 2016. Available at SSRN: https://ssrn.com/abstract=2811871.

Bronevich, A., Lepskiy, A., Penikas, H.: The Application of conflict measure to estimating incoherence of analyst’s forecasts about the cost of shares of Russian

• companies. Proc. Comp. Sc. 55, 1113–1122 (2015).

Howe, J.S., Unlu, E., Yan, X.: The predictive content of aggregate analyst

• recommendations. J. of Account. Res. 47(3), 799–821 (2009).

Huang, J., Mian, G.M., Sankaraguruswamy, S.: The value of combining the inf. content of analyst recommend. and target prices. J. of Fin. Mark. 12, 754–777 (2009). Kanjanatarakul, O., Sriboonchitta, S., Denoeux, T.: Forecasting using belief func.: An application to marketing econometrics. Int. J. of Appr. Reas. 55(5), 1113–1128 (2014). Kim, O., Lim, S.C., Shaw, K.W.: The inefficiency of the mean forecast as a summary forecast of earnings. J. of Account. Res. 39, 329–336 (2001). Utkin, L.V.: Cautious analysis of project risks by interval-valued initial data. Int. J. of Uncert., Fuz. and Knowl.-Based Sys. 14(6), 663–685 (2006). Xu, Y., Wu, L., Wu, X., Xu, Z.: Belief fusion of predict. of industr. in China’s stock

• mark. In: Cuzzolin (ed.) BELIEF 2014, LNAI, vol. 8764, pp. 348–355, Springer (2014).

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 23 / 24

Thanks for you attention

ekytinina@gmail.com, alex.lepskiy@gmail.com http://lepskiy.ucoz.com

Kutynina & Lepskiy (HSE) Aggregation of Forecasts EUSFLAT 2017 24 / 24