Identifying Predictive Causal Factors from News Streams
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Identifying Predictive Causal Factors from News Streams
Ananth Balashankar1 , Sunandan Chakraborty2 , Samuel Fraiberger1,3 , and
Lakshminarayanan Subramanian1
1
Courant Institute of Mathematical Sciences, New York University
2
School of Informatics and Computing, Indiana University-Indianapolis
3
World Bank, Washington DC
ananth@nyu.edu, sunchak@iu.edu, sfraiberger@worldbank.org, lakshmi@nyu.edu
Abstract a single document (Kalyani et al., 2016; Shynke-
vich et al., 2015). However, relationships between
We propose a new framework to uncover the
relationship between news events and real
events affecting stock prices can be quite com-
world phenomena. We present the Predictive plex, and their mentions can be spread across mul-
Causal Graph (PCG) which allows to detect tiple documents. For instance, market volatility is
latent relationships between events mentioned known to be triggered by recessions; this relation-
in news streams. This graph is constructed ship may be reflected with a spike in the frequency
by measuring how the occurrence of a word of the word ”recession” followed by a spike in
in the news influences the occurrence of an- the frequency of the word ”volatility” a few weeks
other (set of) word(s) in the future. We show
later. Existing methods are not well-equipped to
that PCG can be used to extract latent fea-
tures from news streams, outperforming other deal with these cases.
graph-based methods in prediction error of 10 This paper aims to uncover latent relationships
stock price time series for 12 months. We between words describing events in news streams,
then extended PCG to be applicable for longer allowing us to unveil hidden links between events
time windows by allowing time-varying fac- spread across time, and integrate them into a
tors, leading to stock price prediction error news-based predictive model for stock prices. We
rates between 1.5% and 5% for about 4 years.
propose the Predictive Causal Graphs (PCG), a
We then manually validated PCG, finding that
67% of the causation semantic frame argu- framework allowing us to detect latent relation-
ments present in the news corpus were directly ships between words when such relationships are
connected in the PCG, the remaining being not directly observed. PCG differs from existing
connected through a semantically relevant in- relationship extraction (Das et al., 2010) and rep-
termediate node. resentational frameworks (Mikolov et al., 2013)
across two dimensions. First, PCG identifies un-
1 Introduction
supervised causal relationships based on consis-
Contextual embedding models (Devlin et al., tent time series prediction instead of association,
2018) have managed to produce effective repre- allowing us to uncover paths of influence between
sentations of words, achieving state-of-the-art per- news items. Second, PCG finds inter-topic in-
formance on a range of NLP tasks. In this pa- fluence relationships outside the “context” or the
per, we consider a specific task of predicting vari- confines of a single document. Construction of
ations in stock prices based on word relationships PCG naturally leads to news-dependent predictive
extracted from news streams. Existing word em- models for numerous variables, like stock prices.
bedding techniques are not suited to learn relation- We construct PCG by identifying Granger
ships between words appearing in different docu- causal pairs of words (Granger et al., 2000) and
ments and contexts (Le and Mikolov, 2014). Ex- combining them to form a network of words using
isting work on stock price prediction using news the Lasso Granger method (Arnold et al., 2007). A
have typically relied on extracting features from directed edge in the network therefore represents a
financial news (Falinouss, 2007; Hagenau et al., potential influence between words. While predic-
2013), or sentiments expressed on Twitter (Mao tive causality is not true causality (Maziarz, 2015),
et al., 2011; Rao and Srivastava, 2012; Bernardo identification of predictive causal factors which
et al., 2018), or by focusing on features present in prove to be relevant predictors over long periods of
2338
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing
and the 9th International Joint Conference on Natural Language Processing, pages 2338–2348,
Hong Kong, China, November 3–7, 2019. c 2019 Association for Computational Linguisticstime provides guidance for future causal inference lar approach can be observed in (Kozareva, 2012;
studies. We achieve this consistency by proposing Do et al., 2011), whereas CATENA system (Mirza
a framework for Longitudinal Predictive Causal and Tonelli, 2016) used a hybrid approach consist-
Factor identification based on methods of honest ing of a rule-based component and a supervised
estimation (Athey and Imbens, 2016). Here, we classifier. PCG differs from these approaches as
first estimate a universe of predictive causal fac- it explores latent inter-topic causal relationships in
tors on a relatively long time series and then iden- an unsupervised manner from the entire vocabu-
tify time-varying predictive causal factors based lary of words and collocated N-grams.
on constrained estimation on multiple smaller time
Apart from using causality, there are many other
series. We also augment our model with an or-
methods explored to extract information from
thogonal spike correction ARIMA (Brockwell and
news and are used in time series based forecasting.
Davis, 2002) model, allowing us to overcome the
Amodeo et al. (Amodeo et al., 2011) proposed a
drawback of slow recovery in smaller time series.
hybrid model consisting of time-series analysis, to
We constructed PCG from news streams of predict future events using the New York Times
around 700, 000 articles from Google News API corpus. FBLG (Cheng et al., 2014) focused on
and New York Times spread across over 6 years discovering temporal dependency from time series
and evaluated it to extract features for stock price data and applied it to a Twitter dataset mention-
predictions. We obtained two orders lower predic- ing the Haiti earthquake. Similar work by Luo et
tion error compared to a similar semantic causal al. (Luo et al., 2014) showed correlations between
graph-based method (Kang et al., 2017). The lon- real-world events and time-series data for inci-
gitudinal PCG provided insights into the variation dent diagnosis in online services. Other similar
in importance of the predictive causal factors over works like, Trend Analysis Model (TAM) (Kawa-
time, while consistently maintaining a low predic- mae, 2011) and Temporal-LDA (TM-LDA) (Wang
tion error rate between 1.5-5% in predicting 10 et al., 2012) model the temporal aspect of topics in
stock prices. Using full text of more than 1.5 mil- social media streams like Twitter. Structured data
lion articles of Times of India news archives for extraction from news have also been used for stock
over 10 years, we performed a fine-grained qual- price prediction using techniques of information
itative analysis of PCG and validated that 67% retrieval in (Ding et al., 2014; Xie et al., 2013;
of the semantic causation arguments found in the Ding et al., 2015; Chang et al., 2016; Ding et al.,
news text is connected by a direct edge in PCG 2016). Vaca et al. (Vaca et al., 2014) used a collec-
while the rest were linked by a path of length 2. tive matrix factorization method to track emerg-
In summary, PCG provides a powerful framework ing, fading and evolving topics in news streams.
for identifying predictive causal factors from news PCG is inspired by such time series models and
streams to accurately predict and interpret price leverages the Granger causality detection frame-
fluctuations. work for the trend prediction task.
2 Related Work Deriving true causality from observational stud-
ies has been studied extensively. One of the
Online news articles are a popular source most widely used algorithm is to control for vari-
for mining real-world events, including extrac- ables which satisfy the backdoor criterion (Pearl,
tion of causal relationships. Radinsky and 2009). This however, requires a knowledge of the
Horvitz (Radinsky and Horvitz, 2013) proposed causal graph and the unconfoundedness assump-
a framework to find causal relationships between tion that there are no other unobserved confound-
events to predict future events from News but ing variables. While the unconfoundedness as-
caters to a small number of events. Causal re- sumption is to some extent valid when we ana-
lationships extracted from news using Granger lyze all news streams (under the assumption that
causality have also been used for predicting vari- all significant events are reported), it is still hard
ables, such as stock prices (Kang et al., 2017; to get away from the causal graph requirement.
Verma et al., 2017; Darrat et al., 2007). A similar Propensity score based matching aims to control
causal relationship generation model has been pro- for most confounding variables by using an ex-
posed by Hashimoto et al. (2015) to extract causal ternal method for estimating and controlling for
relationships from natural language text. A simi- the likelihood of outcomes (Olteanu et al., 2017).
2339More recently, (Wang and Blei, 2018) showed that 3.1 Selecting Informative Words:
with multiple causal factors, it is possible to lever- Only a small percentage of the words appearing
age the correlation of those multiple causal fac- in news can be used for meaningful information
tors and deconfound using a latent variable model. extraction and analysis (Manning et al., 1999; Ho-
This setting is similar to the one we consider, and vold, 2005). Specifically, we eliminated too fre-
is guaranteed to be truly causal if there is no con- quent (at least once in more than 50% of the days)
founder which links a single cause and the out- or too rare (appearing in less than 100 articles)
come. This assumption is less strict than the un- (Manning et al., 2008). Many common English
confoundedness assumption and makes the case nouns, adjectives and verbs, whose contribution
for using predictive causality in such scenarios. to semantics is minimal (Forman, 2003) were also
Another approach taken by (Athey and Imbens, removed from the vocabulary. However, named-
2016) estimates heterogeneous treatment effects entities were retained for their newsworthiness and
by honest estimation where the model selection a set of “trigger” words were retained that de-
and factor weight estimation is done on two sub- pict events (e.g. flood, election) using an existing
populations of data by extending regression trees. “event trigger” detection algorithm (Ahn, 2006).
Our work is motivated by these works and ap- The vocabulary set was enhanced by adding bi-
plies methodologies for time series data extracted grams that are significantly collocated in the cor-
from news streams. PCG can offer the following pus, such as, ‘fuel price’ and ‘prime minister’ etc.
benefits for using news for predictive analytics –
(1) Detection of influence path, (2) Unsupervised 3.2 Time-series Representation of Words:
feature extraction, (3) Hypothesis testing for ex- Consider a corpus D of news articles indexed by
periment design. time t, such that Dt is the collection of news arti-
cles published at time t. Each article d ∈ D is a
collection of words Wd , where ith word wd,i ∈
3 Predictive Causal Graph
Wd is drawn from a vocabulary V of size N .
The set of articles published at time t can be ex-
Predictive Causal Graph (PCG) addresses the dis-
pressed in terms of the words appearing in the
covery of influence between words that appear in
articles as {α1t , α2t , ..., αN
t }, where αt is the sum
i
news text. The identification of influence link be-
of frequency of the word wi ∈ V across all ar-
tween words is based on temporal co-variance,
ticles published at time t. αit corresponding to
that can help answer questions of the form: “Does µt
the appearance of word x influence the appear- wi ∈ V is defined as, αit = PT i t where µti =
t=1 µi
ance of word y after δ days?”. The influence of P|Dt | t
d=1 T F (wd,i ). αi is normalized by using the
one word on another is determined based on pair- frequency distribution of wi in the entire time pe-
wise causal relationships and is computed using riod. T (wi ) represents the time series of the word
the Granger causality test. Following the identifi- wi , where i varies from 1 to N , the vocabulary
cation of Granger causal pairs of words, such pairs size.
are combined together to form a network of words,
where the directed edges depict potential influence 3.3 Measuring Influence between Words
between words. In the final network, an edge or a Given two time-series X and Y , the Granger
path between a word pair represents a flow of in- causality test checks whether the X is more effec-
fluence from the source word to the final word and tive in predicting Y , than using just Y and if this
this influence depicts an increase in the appearance holds then the test concludes X “Granger-causes”
of the final words when the source word was ob- Y (Granger et al., 2000). However, if both X and
served in news data. Y are driven by a common third process with dif-
Construction of PCG from the raw unstructured ferent lags, one might still fail to reject the alter-
news data, finding pairwise causal links and even- native hypothesis of Granger causality. Hence, in
tually building the influence network involves nu- PCG, we explore the possibility of causal links be-
merous challenges. In the rest of the section, we tween all word pairs and detect triangulated rela-
discuss the design methodologies used to over- tions to eliminate the risk of ignoring confounding
come these challenges and describe some proper- variables, otherwise not considered in the Granger
ties of the PCG. causality test.
2340coordinate committee 19 provided relief Dirichlet Allocation (LDA)(Blei et al., 2003). In-
2 21
fluence is generalized to topic level by calculating
the weight of inter-topic influence relationships as
slum rehabilitation a total number of edges between vertices of two
topics. If we define θu and θv to be two topics
Figure 1: PCG highlighting the underlying cause
in our topic model and |θu | represents the size of
topic θu , i.e. the number of words in the topic
However, constructing PCG using an exhaus- whose topic-probability is greater than a thresh-
tive set of word pairs does not scale, as even af- old (0.001), then the strength of influence between
ter using a reduced set of words and including the topics θu and θv is defined as,
collocated phrases, the vocabulary size is around
39, 000. One solution to this problem is consid- Φ(θu , θv ) =
# Edges between words in θu and θv
(2)
ering the Lasso Granger method (Arnold et al., (|θu | × |θv |)
2007) that applies regression to the neighborhood
selection problem for any word, given the fact that Φ(θu , θv ) is termed as strong if its value is in
the best regressor for that variable with the least the 99th percentile of Φ for all topics. Any
squared error will have non-zero coefficients only edge in the original PCG is removed if there are
for the lagged variables in the neighborhood. The no strong topic edges between the corresponding
Lasso algorithm for linear regression is an incre- word nodes. This filtered topic graph has only
mental algorithm that embodies a method of vari- edges between topics which have high influence
able selection (Tibshirani, 1994). strength. This combination of inter-document
If we define V to be the input vocabulary from temporal and intra-document distributional simi-
the news dataset, N is the vocabulary size, x is the larity is critical to obtaining temporally and se-
list of all lagged variables (each word is multivari- mantically consistent predictive causal factors.
ate with a maximum lag of 30 days per word) of
the vocabulary, w is the weight vector denoting the
4 Prediction Models using PCG
influence of each variable, y is the predicted time In this section, we present three approaches for
series variable and λ is a sparsity constraint hyper- building prediction models using PCG namely (1)
parameter to be fine-tuned, then minimizing the direct estimation using PCG (2) longitudinal pre-
regression loss below leads to weights that charac- diction which incorporates short term temporal
terize the influential links between words in x that variations and (3) spike augumented prediction
predicts y, which estimates spikes over a longer time window.
1 4.1 Direct Prediction from PCG
w = argmin Σ(x,y)∈V |w.x − y|2 + λ||w|| (1)
N
One straightforward way of using PCG for predic-
To set λ, we use the method based on consistent tion modeling is to use the Lasso regression equa-
estimation used in (Meinshausen and Bühlmann, tion used for identifying the predictive causal fac-
2006). We select the variables that have non-zero tors directly. We first adopt this approach by re-
co-efficients and choose the best lag for a given stricting the construction of PCG to the nodes of
variable based on the maximum absolute value of concern, which significantly speeds up the com-
a word’s co-efficient. We then, draw an edge from putation. This inherently ignores any predictive
all these words to the predicted word with the an- causal factor which only has an indirect link to
notations of the optimal time lag (in days) and in- the outcome node, as theorized by the Granger
crementally construct the graph as illustrated in Causality framework. In this case, we split the
Figure 1. data into a contiguous training data, and evaluate
on the remaining testing data. If y represents the
3.4 Topic Influence Compression target stock time series variable and x represents a
To arrive at a sparse graphical representation of multivariate vector of all lagged feature variables,
PCG, we compress the graph based on topics (50 w represents the coefficient weight vector indexed
topics in our case). Topics are learned from the by the feature variable z ∈ x and time lag m, p, q
original news corpus using unsupervised Latent in days, a represent a bias constant and t repre-
2341sents an i.i.d noise variable, then we predict future indicators have a monthly cycle and hence respon-
values of y as follows. siveness within that cycle is desired.
q
r
m
X XX E(T re , f ) = E M SE(w, f ) (5)
yt = a + wy,i yt−i + wz,j zt−j + t (3) w∈T re ,|w|=W
i=1 z∈x j=p
fLP C (T rm , T re ) = minf ∈F (T rm ) E(T re , f ) (6)
4.2 Longitudinal Prediction via Honest
Estimation We then evaluate the model on an unseen time
series T e, where the learnt predictive causal fac-
In scenarios where heterogenous causal effects are
tors and their weights are used for inference.
to be estimated, it is important to adjust by parti-
tioning the time series into subpopulations which 4.3 Spike Prediction
vary in the magnitude of the causal effects (Athey One drawback of using a specific time window for
and Imbens, 2016). In a news stream, this amounts estimating the weights of the predictive causal fac-
to constructing the word influence networks given tors is the lack of representative data in the win-
a context specified by a time window. This naive dow used for training. This could mean that pre-
extension however can be quite computationally dicting abrupt drops or spikes in the data would
expensive and can limit the speed of inference be hard. To overcome this limitation, we train a
considerably. However, if the set of potential composite model to predict the residual error from
causal factors are identified over a larger time se- honest estimation by training on differences in
ries, learning their time varying weights over a consecutive values of the time series. Let (∆y =
shorter training period can significantly decrease yt − yt−1 , ∆f = ft − ft−1 ) denote time series of
the computation required. the differences of the consecutive values of the la-
Hence, we do a two staged honest estimation bels and the feature variables and let [∆y, ∆f ] de-
approach similar to (Athey and Imbens, 2016). In note the concatenated input variables of the model.
the first stage, multiple sets (instead of trees as We use a multivariate ARIMA model M of or-
in (Athey and Imbens, 2016)) of predictive causal der (p, d, q) (Brockwell and Davis, 2002) where
factors F (T rm ), that provide overall reduction p controls the number of time lags included in
in root mean squared error (RM SE), over train- the model, q denotes the number of times differ-
ing data T rm , are gathered for model selection ences are taken between consecutive values and r
through repeated random initialization of regres- denotes the time window of the moving average
sion weights. For any f ∈ F (T rm ), we define f taken to incorporate sudden spikes in values. Let
to be a set of predictive factors which when trained the actual values of the time series of label y be
over data T rm to predict future time series values y ∗ , the predictions of the honest estimation model
of the target y, achieves RM SE(T rm , f ) < δ, for be ŷ, a training sample of significantly longer time
a hyperparameter δ > 0. window T rs with |T rs | >> W , then the com-
[ posite model is trained to predict the residuals,
F (T rm ) = {RM SE(T rm , f ) < δ} (4) res = y ∗ − ŷ = E(T rs , f ).
f
From these sets of predictive causal fac-
M = ARIM A(p, d, q) (7)
tors F (T rm ), we choose the set of features
M.f it(E(T rs , f ), [∆y, ∆f ]) (8)
fLP C (T rm , T re ), which gives the least expected
res
ˆ = M.predict(T re ) (9)
root mean squared error on time windows w of
length W uniformly sampled from the unseen Augmenting this predicted residual (res) ˆ back
training data used for estimation T re through to ŷT re gives us the spike-corrected estimate yˆs .
cross validation. This model selection procedure
depends on the size of the smaller time windows yˆs = ŷT re + res
ˆ (10)
W used to fine-tune the factors. The size of this
5 Results
time window is a hyperparameter to trade off long-
term stability and short-term responsiveness of the In this section, we present the results from direc-
predictive causal factors. We chose the time win- tion prediction models from PCG, followed by im-
dow based of 30 days for our stock price predic- provement in stock price prediction due to longitu-
tion due to prior knowledge that many financial dinal and spike prediction from news streams and
2342compare it to a manually tuned semantic causal Table 1: 30 day windowed average of stock price pre-
graph method. We analyze the time varying fac- diction error using PCG
tors to explain the gains achieved via honest esti-
Step size Cbest P CGuni P CGbi P CGboth
mation. 1 1.96 0.022 0.023 0.020
3 3.78 0.022 0.023 0.022
5.1 Data and Metrics 5 5.25 0.022 0.023 0.021
The news dataset1 we used for stock price predic-
tion contains news crawled from 2010 to 2013 us-
regressive model on the combined time series of
ing Google News APIs and New York Times data
text and historical stock values.
from 1989 to 2007. We construct PCG from the
time series representation of its 12,804 unigrams Comparison with the baseline: Compared to
and 25,909 bigrams over the entire news corpora the baseline, note that our features and topics were
of more than 23 years, as well as the 10 stock chosen purely based on distributional semantics of
prices2 from 2010 to 2012 for training and 2013 the word time series. Once the features are ex-
as test data for prediction. The prediction is done tracted from PCG, we use the past values of stock
with varying step sizes (1,3,5), which indicates the prices and time series corresponding to the incom-
time lag between the news data and the day of the ing word edges of PCG to predict the future values
predicted stock price in days. The results shown in of the stock prices using the multivariate regres-
Table 1 is the root mean squared error (RMSE) in sion equation used to determine Granger Causal-
predicted stock value calculated on a 30 day win- ity. As compared to their best error, PCG from uni-
dow averaged by moving it by 10 days over the pe- grams, bigrams or both obtain two orders lower er-
riod and directly comparable to our baseline (Kang ror and significantly outperforms Cbest . The mean
et al., 2017). To evaluate the time-varying fac- absolute error (MAE) for the same set of evalu-
tors over a larger time window, we present average ations is within 0.003 of the RMSE, which indi-
monthly cross validation RMSE % sampled over a cates that the variance of the errors is also low.
4 year window of 2010-13 in Table 3. Please note We attribute this gain to the flexibility of PCG’s
that the results in Table 3 are not comparable with Lasso Granger method to produce sparse graphs
(Kang et al., 2017) as we report a cross validation as compared to CGRAPH’s Vector Auto Regres-
error over a longer time window. sive model which used a fixed number (10) of in-
coming edges per node already pre-filtered by a
5.2 Prediction Performance of PCG semantic parser. This imposes an artificial bound
on sparsity thereby losing valuable latent informa-
To evaluate the causal links generated by PCG, we
tion. We overcome this in PCG using a suitable
use it to extract features for predicting stock prices
penalty term (λ) in the Lasso method.
using the exact data and prediction setting used in
Key PCG factors for 2013: The causal links
Kang et al. (2017) as our baseline.
in PCG are more generic (Table 2) than the ones
Baseline: Kang et al. (2017) extract relevant described in CGRAPH, supporting the hypothesis
variables based on a semantic parser - SEMAFOR that latent word relationships do exist that go be-
(Das et al., 2010) by filtering causation related yond the scope of a single news article. The nodes
frames from news corpora, topics and sentiments of CGRAPH are tuples extracted from a seman-
from tweets. To overcome the problem of low re- tic parser (SEMAFOR (Das et al., 2010)) based
call, they adopt a topic-based knowledge base ex- on evidence of causality in a sentence. PCG poses
pansion. This expanded dataset is then used to no such restriction and derives topical (unfriended,
train a neural reasoning model which generates se- FB) and inter-topical (healthcare, AMZN), sparse,
quence of cause-effect statements using an atten- latent and semantic relationships.
tion model where the words are represented using
Inspecting the links and paths of PCG gives us
word2vec vectors. (Kang et al., 2017)’s CGRAPH
qualitative insights into the context in which the
based forecasting model - Cbest model uses the
word-word relationships were established. Since
top 10 such generated cause features, given the
PCG is also capable of representing other stock
stock name as the effect and apply a vector auto-
time series as potential influencers in the network,
1
https://github.com/dykang/cgraph we can use this to model the propagation of shocks
2
https://finance.yahoo.com in the market as shown in Figure 2. However,
2343Table 2: Stock price predictive factors for 2013 in PCG
0.009
0.008
Stock symbol Prediction indicators
AAPL workplace, shutter, music 0.007
0.006
Importance weight
AMZN healthcare, HBO, cloud
FB unfriended, troll, politician 0.005
GOOG advertisers, artificial intelligence, shake-up
HPQ China, inventions, Pittsburg 0.004
IBM 64 GB, redesign, outage 0.003
MSFT interactive, security, Broadcom 0.002
ORCL corporate, investing, multimedia
0.001
TSLA prices, testers, controversy
YHOO entertainment, leadership, investment 0.000
0 10 20 30 40 50 60 70 80
time (in weeks)
FB (a) “podcast” for predicting AAPL stock
8
HP 14 AAPL 0.7
8
0.6
AMZN
0.5
Importance weight
Figure 2: Inter-stock influencer links where one stock’s
0.4
movement indicates future movement of other stocks
(time lag annotated edges) 0.3
0.2
0.1
these links were not used for prediction perfor-
0.0
mance to maintain parity with our baseline. 0 10 20 30 40 50 60 70 80
time (in weeks)
5.3 Time Varying Causal Analysis (b) “unfollowers” for predicting GOOG stock
We quantitatively evaluate the time varying variant Figure 3: Temporal variation in importance weight of
of PCG by using it to extract features for stock predictive causal factors
price prediction for a longer time window.
We present average root mean squared errors in igated when the time window for training is in-
Table 3 for different values of time windows of creased. Increasing the window more than 100 did
size W (50,100). For model selection, we used not improve the RMSE and came at the cost of
50% of the time series and then used multiple time training time. But, incorporating the spike resid-
series of length W , disjoint from the ones used for ual PCG model, which predicts the leftover price
time-varying factor selection and took average of value from the first model, provides significant im-
the test error on the next 30 data points using the provements over the model without spike correc-
weights learnt. We repeat this using K-fold cross tion as seen in the last column on Table 3. Thus,
validation (K=10) for choosing the model selec- we are able to achieve significant gains with an
tion data and present the average errors. The vari- unsupervised approach without any domain spe-
ation in importance weights of predictive causal cific feature engineering by estimating using an
factors for stock prices (“podcast”, AAPL) and ARIMA model (p, d, q) = (30, 1, 10).
(“unfollowers”, GOOG) is shown in Figure 3
which illustrates several peaks (for weeks) when 6 Interpretation of Predictive Causal
the factor in the news was particularly predictive Factors
of the company’s stock price and not significant
during other weeks. In order to qualitatively validate that the latent
inter-topic edges learnt from the news stream is
The time series and error graph shown for mul-
also humanly interpretable, we constructed PCG
tiple stocks shows that the RMSE errors range be-
from the online archives of Times of India (TOI)
tween 1.5% 5% for all the test time series as shown 3 , the most circulated Indian English newspaper.
in Figure 4. However, sudden spikes tend to dis-
We used this dataset as, unlike the previous dataset
play higher error rates due to the lack of train-
3
ing data which contain spikes. This issue is mit- https://timesofindia.indiatimes.com/archive.cms
23445.5 AAPL Stock Prediction 110 3.0 HPQ Stock Prediction 14 4.0 ORCL Stock Prediction 38
5.0 13 3.5 36
100 2.5
4.5 12 3.0 34
4.0 90 2.0 11
2.5
32
stock price
stock price
stock price
3.5 10
RMSE
RMSE
RMSE
80 1.5 2.0
3.0 9 30
1.5
2.5 70 1.0 8
1.0 28
2.0 7
60 0.5 26
1.5 6 0.5
1.0 50 0.0 5 0.0 24
0 100 200 300 400 500 600 700 800 900 0 100 200 300 400 500 600 700 800 900 0 100 200 300 400 500 600 700 800 900
time (days) time (days) time (days)
Figure 4: RMSE of stocks for longitudinal causal factor prediction without spike correction. Spikes in RMSE can
be seen along with spikes in stock prices like HPQ.
Table 3: Variation in stock price prediction error tion of causality like “caused”, “effect”, “due to”
(RMSE) % with window size and spike correction and manually verified that these word pairs were
indeed causal. We then searched the shortest path
Stock W=50 W=100 W=100 + spike
AABA 2.87 2.07 1.61
in PCG between these word pairs. For example,
AAPL 2.95 2.84 2.28 one of the news article mentioned that “Gujarat
AMZN 3.03 2.99 2.41 government has set aside a suggestion for price
GOOG 2.67 2.36 1.91
hike in electricity due to the Mundra Ultra Mega
HPQ 6.77 3.34 2.44
IBM 2.19 2.07 1.65 Power Project.” and these corresponding causa-
MSFT 3.03 9.45 4.80 tion arguments were captured by a direct link in
ORCL 2.94 2.21 1.65 PCG as shown in Table 4. 67% of the word pairs
TSLA 5.56 5.52 4.32
which were manually identified to be causal in
the news text through causal indicator words such
which provided just the time series of words, we as “caused”, were linked in PCG through direct
also have the raw text of the articles, which al- edges, while the rest were linked through an in-
lowed us to perform manual causal signature ver- termediate relevant node. As seen in Table 4, the
ification. This dataset contains all the articles bi-gram involving the words and the intermediate
published in their online edition between Jan- words in the path provide the relevant context un-
uary 1, 2006 and December 31, 2015 containing der which the causality is established. The time
1,538,932 articles. lags in the path show that the influence between
events are at different time lags. We also qualita-
6.1 Inter-topic edges of PCG tively verified that two unrelated words are either
The influence network we constructed from the not connected or have a path length greater than
TOI dataset has 18,541 edges and 7,190 uni-grams 2, which makes the relationship weak. The abil-
and bi-gram vertices. We were interested in the ity of PCG to validate such humanly understood
inter-topic non-associative relationships that PCG causal pairs with temporal predictive causality can
is expected to capture. We observe that a few top- be used for better hypothesis testing.
ics (5) influence or are influenced by a large num- Table 4: Comparison with manually identified influ-
ber of topics. Some of these highly influential top- ence from news articles
ics are composed of words describing “Agricul-
ture”, “Politics”, “Crime”, etc. The ability of PCG Pairs in news Relevant paths in PCG
to learn these edges between topical word clusters price, project price-hike –(19)– power-project
land, budget allot-land –(22)– railway-budget
purely based on temporal predictive causal predic- price, land price-hike –(12)– land
tion further validates its use for design of extensive strike, law terror-strike –(25)– law ministry
land, bill land-reform –(25)– bill-pass
causal inference experiments. election, strike election –(21)– Kerala government –(10)– strike
election, strike election –(18)– Mumbai University –(14)– strike
6.2 Causal evidence in PCG election, strike election –(20)– Shiromani Akali –(13)– strike
To validate the causal links in PCG, we extracted
56 causation semantic frame (Baker et al., 1998)
7 Conclusion
arguments which depict direct causal relationships
in the news corpus. We narrowed down the search We presented PCG, a framework for building pre-
to words surrounding verbs which depict the no- dictive causal graphs which capture hidden rela-
2345tionships between words in text streams. PCG Ching-Yun Chang, Yue Zhang, Zhiyang Teng, Zahn
overcomes the limitations of contextual represen- Bozanic, and Bin Ke. 2016. Measuring the Informa-
tion Content of Financial News. In COLING. ACL,
tation approaches and provides the framework to
3216–3225.
capture inter-document word and topical relation-
ships spread across time to solve complex min- Dehua Cheng, Mohammad Taha Bahadori, and Yan
ing tasks from text streams. We demonstrated Liu. 2014. FBLG: A Simple and Effective Ap-
the power of these graphs in providing insights to proach for Temporal Dependence Discovery from
Time Series Data (KDD ’14). 382–391. https:
answer causal hypotheses and extracting features //doi.org/10.1145/2623330.2623709
to provide consistent, interpretable and accurate
stock price prediction from news streams through Ali F. Darrat, Maosen Zhong, and Louis T.W.
honest estimation on unseen time series data. Cheng. 2007. Intraday volume and volatility re-
lations with and without public news. Jour-
nal of Banking and Finance 31, 9 (2007), 2711
References – 2729. https://doi.org/10.1016/j.
jbankfin.2006.11.019
David Ahn. 2006. The stages of event extrac-
tion. In Proceedings of the Workshop on Annotat- Dipanjan Das, Nathan Schneider, Desai Chen, and
ing and Reasoning about Time and Events. As- Noah A. Smith. 2010. Probabilistic Frame-
sociation for Computational Linguistics, Sydney, Semantic Parsing. In Human Language Tech-
Australia, 1–8. https://www.aclweb.org/ nologies: The 2010 Annual Conference of the
anthology/W06-0901 North American Chapter of the Association for
Computational Linguistics. Association for Com-
Giuseppe Amodeo, Roi Blanco, and Ulf Brefeld. 2011. putational Linguistics, Los Angeles, California,
Hybrid Models for Future Event Prediction (CIKM 948–956. https://www.aclweb.org/
’11). 1981–1984. anthology/N10-1138
Andrew Arnold, Yan Liu, and Naoki Abe. 2007.
Temporal Causal Modeling with Graphical Granger Jacob Devlin, Ming-Wei Chang, Kenton Lee, and
Methods. In Proceedings of the 13th ACM SIGKDD Kristina Toutanova. 2018. BERT: Pre-training
International Conference on Knowledge Discovery of Deep Bidirectional Transformers for Language
and Data Mining (KDD ’07). ACM, New York, NY, Understanding. CoRR abs/1810.04805 (2018).
USA, 66–75. https://doi.org/10.1145/ arXiv:1810.04805 http://arxiv.org/abs/
1281192.1281203 1810.04805
Susan Athey and Guido Imbens. 2016. Recursive par- Xiao Ding, Yue Zhang, Ting Liu, and Junwen Duan.
titioning for heterogeneous causal effects: Table 2014. Using Structured Events to Predict Stock
1. Proceedings of the National Academy of Sci- Price Movement: An Empirical Investigation.
ences 113, 7353–7360. https://doi.org/
10.1073/pnas.1510489113 Xiao Ding, Yue Zhang, Ting Liu, and Junwen Duan.
Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 2015. Deep Learning for Event-Driven Stock Pre-
1998. The Berkeley FrameNet Project. In Pro- diction. In IJCAI. AAAI Press, 2327–2333.
ceedings of the 36th Annual Meeting of the Asso-
ciation for Computational Linguistics and 17th In- Xiao Ding, Yue Zhang, Ting Liu, and Junwen Duan.
ternational Conference on Computational Linguis- 2016. Knowledge-Driven Event Embedding for
tics - Volume 1 (ACL ’98/COLING ’98). Associ- Stock Prediction. In COLING. ACL, 2133–2142.
ation for Computational Linguistics, Stroudsburg,
PA, USA, 86–90. https://doi.org/10. Quang Xuan Do, Yee Seng Chan, and Dan Roth.
3115/980845.980860 2011. Minimally Supervised Event Causal-
ity Identification. In Proceedings of the Con-
Ivo Bernardo, Roberto Henriques, and Victor Lobo. ference on Empirical Methods in Natural Lan-
2018. Social Market: Stock Market and Twit- guage Processing (EMNLP ’11). Association for
ter Correlation. In Intelligent Decision Technologies Computational Linguistics, Stroudsburg, PA, USA,
2017, Ireneusz Czarnowski, Robert J. Howlett, and 294–303. http://dl.acm.org/citation.
Lakhmi C. Jain (Eds.). Springer International Pub- cfm?id=2145432.2145466
lishing, Cham, 341–356.
David M. Blei, Andrew Y. Ng, and Michael I. Jordan. Pegah Falinouss. 2007. Stock Trend Prediction using
2003. Latent Dirichlet Allocation. J. Mach. Learn. News Events. Masters thesis (2007).
Res. 3 (March 2003), 993–1022.
George Forman. 2003. An extensive empirical study
Peter J. Brockwell and Richard A. Davis. 2002. In- of feature selection metrics for text classification.
troduction to Time Series and Forecasting (2nd ed.). Journal of machine learning research 3, Mar (2003),
Springer. 1289–1305.
2346Clive WJ Granger, Bwo-Nung Huangb, and Chin-Wei Press, 100123. https://doi.org/10.1017/
Yang. 2000. A bivariate causality between stock CBO9780511809071.007
prices and exchange rates: evidence from recent
Asianflu. The Quarterly Review of Economics and Christopher D Manning, Hinrich Schütze, et al. 1999.
Finance 40, 3 (2000), 337–354. Foundations of statistical natural language process-
ing. Vol. 999. MIT Press.
Michael Hagenau, Michael Liebmann, and Dirk Neu-
mann. 2013. Automated news reading: Stock price Huina Mao, Scott Counts, and Johan Bollen. 2011.
prediction based on financial news using context- Predicting Financial Markets: Comparing Survey,
capturing features. Decision Support Systems 55, News, Twitter and Search Engine Data. Arxiv
3 (2013), 685 – 697. https://doi.org/10. preprint (2011).
1016/j.dss.2013.02.006
Mariusz Maziarz. 2015. A review of the Granger-
Chikara Hashimoto, Kentaro Torisawa, Julien Kloet- causality fallacy. The Journal of Philosoph-
zer, and Jong-Hoon Oh. 2015. Generating Event ical Economics 8, 2 (2015), 6. https:
Causality Hypotheses Through Semantic Relations. //EconPapers.repec.org/RePEc:bus:
In Proceedings of the Twenty-Ninth AAAI Confer- jphile:v:8:y:2015:i:2:n:6
ence on Artificial Intelligence (AAAI’15). AAAI
Press, 2396–2403. http://dl.acm.org/ Nicolai Meinshausen and Peter Bühlmann. 2006.
citation.cfm?id=2886521.2886654 High-dimensional graphs and variable selection with
the lasso. The annals of statistics (2006), 1436–
Johan Hovold. 2005. Naive Bayes Spam Filtering Us- 1462.
ing Word-Position-Based Attributes.. In CEAS. 41–
48. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Cor-
rado, and Jeffrey Dean. 2013. Distributed Repre-
Joshi Kalyani, H. N. Bharathi, and Rao Jyothi. sentations of Words and Phrases and Their Com-
2016. Stock trend prediction using news sen- positionality. In Proceedings of the 26th Interna-
timent analysis. CoRR abs/1607.01958 (2016). tional Conference on Neural Information Processing
arXiv:1607.01958 http://arxiv.org/abs/ Systems - Volume 2 (NIPS’13). Curran Associates
1607.01958 Inc., USA, 3111–3119. http://dl.acm.org/
citation.cfm?id=2999792.2999959
Dongyeop Kang, Varun Gangal, Ang Lu, Zheng Chen,
and Eduard Hovy. 2017. Detecting and Explaining Paramita Mirza and Sara Tonelli. 2016. CATENA:
Causes From Text For a Time Series Event. In Pro- CAusal and TEmporal relation extraction from
ceedings of the 2017 Conference on Empirical Meth- NAtural language texts. In Proceedings of COL-
ods in Natural Language Processing. Association ING 2016, the 26th International Conference
for Computational Linguistics, Copenhagen, Den- on Computational Linguistics: Technical Pa-
mark, 2758–2767. https://www.aclweb. pers. The COLING 2016 Organizing Commit-
org/anthology/D17-1292 tee, 64–75. http://www.aclweb.org/
Noriaki Kawamae. 2011. Trend analysis model: trend anthology/C16-1007
consists of temporal words, topics, and timestamps
Alexandra Olteanu, Onur Varol, and Emre Kiciman.
(WSDM ’11). 317–326.
2017. Distilling the Outcomes of Personal Experi-
Zornitsa Kozareva. 2012. Cause-effect Relation Learn- ences: A Propensity-scored Analysis of Social Me-
ing. In Workshop Proceedings of TextGraphs-7 on dia. In Proceedings of The 20th ACM Conference on
Graph-based Methods for Natural Language Pro- Computer-Supported Cooperative Work and Social
cessing (TextGraphs-7 ’12). Association for Com- Computing (computer-supported cooperative work
putational Linguistics, Stroudsburg, PA, USA, 39– and social computing ed.). Association for Comput-
43. http://dl.acm.org/citation.cfm? ing Machinery, Inc.
id=2392954.2392961
Judea Pearl. 2009. Causal inference in statistics: An
Quoc V. Le and Tomas Mikolov. 2014. Distributed overview. Statistics Surveys (2009).
Representations of Sentences and Documents.
CoRR abs/1405.4053 (2014). arXiv:1405.4053 Kira Radinsky and Eric Horvitz. 2013. Mining the web
http://arxiv.org/abs/1405.4053 to predict future events (WSDM ’13). ACM, 255–
264.
Chen Luo, Jian-Guang Lou, Qingwei Lin, Qiang Fu,
Rui Ding, Dongmei Zhang, and Zhe Wang. 2014. Tushar Rao and Saket Srivastava. 2012. Ana-
Correlating Events with Time Series for Incident Di- lyzing Stock Market Movements Using Twitter
agnosis (KDD ’14). 1583–1592. https://doi. Sentiment Analysis. In Proceedings of the 2012
org/10.1145/2623330.2623374 International Conference on Advances in Social
Networks Analysis and Mining (ASONAM 2012)
Christopher D. Manning, Prabhakar Raghavan, and (ASONAM ’12). IEEE Computer Society, Washing-
Hinrich Schtze. 2008. Scoring, term weighting, ton, DC, USA, 119–123. https://doi.org/
and the vector space model. Cambridge University 10.1109/ASONAM.2012.30
2347Y. Shynkevich, T. M. McGinnity, S. Coleman, and
A. Belatreche. 2015. Stock price prediction based
on stock-specific and sub-industry-specific news ar-
ticles. In 2015 International Joint Conference on
Neural Networks (IJCNN). 1–8. https://doi.
org/10.1109/IJCNN.2015.7280517
Robert Tibshirani. 1994. Regression Shrinkage and Se-
lection Via the Lasso. Journal of the Royal Statisti-
cal Society, Series B 58 (1994), 267–288.
Carmen K Vaca, Amin Mantrach, Alejandro Jaimes,
and Marco Saerens. 2014. A time-based collective
factorization for topic discovery and monitoring in
news (WWW ’14). 527–538.
Ishan Verma, Lipika Dey, and Hardik Meisheri.
2017. Detecting, Quantifying and Accessing Im-
pact of News Events on Indian Stock Indices.
In Proceedings of the International Conference
on Web Intelligence (WI ’17). ACM, New York,
NY, USA, 550–557. https://doi.org/10.
1145/3106426.3106482
Yu Wang, Eugene Agichtein, and Michele Benzi. 2012.
Tm-lda: efficient online modeling of latent topic
transitions in social media (KDD ’12). ACM, 123–
131.
Yixin Wang and David M. Blei. 2018. The Blessings of
Multiple Causes. CoRR abs/1805.06826. http:
//dblp.uni-trier.de/db/journals/
corr/corr1805.html#abs-1805-06826
Boyi Xie, Rebecca J. Passonneau, Leon Wu, and
Germán Creamer. 2013. Semantic Frames to Predict
Stock Price Movement. In ACL (1). The Association
for Computer Linguistics, 873–883.
2348You can also read
