I regularly publish papers on arXiv.org, an open-access archive for research in physics, math, computer science, and (recently) quantitative finance. I also subscribe to digest updates on recently published research.
Edit: It’s unclear whether the in-sample issue actually affects the prediction or whether this is only used to compare OF and GPOMS. Though the regressions are written using X_t and not Z_{X_t}, Figure 3 and its accompanying interpretation clearly compare the z-scores of $DJIA and their chosen signals.
I noticed an interesting paper hit the digest tonight: Twitter mood predicts the stock market. Though I haven’t read it in detail, the paper suggests that sentiment analysis of Twitter can be used to improve the prediction of market direction. My quick scan of the paper found it to be mostly out-of-sample, though it appears that the OpinionFinder (OF) and Google Profile (GPOMS) data are normalized with symmetric windows that do incorporate in-sample data. However, the degree of improvement in prediction suggests to me that this sentiment analysis might improve prediction even when this issue corrected. Below is the abstract and full citation:
“Behavioral economics tells us that emotions can profoundly affect individual behavior and decision-making. Does this also apply to societies at large, i.e., can societies experience mood states that affect their collective decision making? By extension is the public mood correlated or even predictive of economic indicators? Here we investigate whether measurements of collective mood states derived from large-scale Twitter feeds are correlated to the value of the Dow Jones Industrial Average (DJIA) over time. We analyze the text content of daily Twitter feeds by two mood tracking tools, namely OpinionFinder that measures positive vs. negative mood and Google-Profile of Mood States (GPOMS) that measures mood in terms of 6 dimensions (Calm, Alert, Sure, Vital, Kind, and Happy). We cross-validate the resulting mood time series by comparing their ability to detect the public’s response to the presidential election and Thanksgiving day in 2008. A Granger causality analysis and a Self-Organizing Fuzzy Neural Network are then used to investigate the hypothesis that public mood states, as measured by the OpinionFinder and GPOMS mood time series, are predictive of changes in DJIA closing values. Our results indicate that the accuracy of DJIA predictions can be significantly improved by the inclusion of specific public mood dimensions but not others. We find an accuracy of 87.6% in predicting the daily up and down changes in the closing values of the DJIA and a reduction of the Mean Average Percentage Error by more than 6%.”
Johan Bollen, Huina Mao, Xiao-Jun Zeng. Twitter mood predicts the stock market. arXiv:1010.3003