Online media coverage before and after the Bundestag elections in 2017
Franziska Löw
06 Sep 2020
1. Introduction
In this paper I develop and apply a machine learning approach based on structural topic modeling in combination with cosine similarity and a regression discontinuity framework in order to identify similarities between topics addressed in press releases and online news articles of different media sources before and after the Bundestag elections that took place the 24th of Sep 2017.
Research question:
How does media coverage change after the election day for different parties?
Method:
Estimate a structural topic model to obtain a topic distribution for each document (press release or news article)
Estimate the cosine similarity of topic distribution between documents of the same week (Hier könnte man überlegen ob es mehr Sinn macht eher die Dokumente +/-7 Tage zu nehmen)
Cosine similarity is built on the geometric definition of the dot product of two vectors. It is a measure for the distance between two vectors and is defined between zero and one; values towards 1 indicate similarity.
\[ \text{cosine similarity} = \text{cos}(\theta)=\frac{a*b}{||a|| ||b||} \] As topic proportions per news article / press release are vectors of the same length, the cosine similarity allows a comparison of the topic distribution between two documents.
Applications of cosine similarity to compare of topic model outcomes: Ramage et al. 2010, Rehs 2020
- Estimate a regression discontinuity model for each media outlet \(i\)
\[ \text{CosineSimilarity}_{i}=\beta_0+\beta_1T_i+\beta_nDn_{j}+\beta_{n+n}T_iDn_{j}+\epsilon_i \] where the \(D2_{j}, D3_{j}, ... ,Dn_{j}\) represent dummy variables for a political party \(j\)
\[ T_i = 1 \text{ if date } >= \text{election date} \\ T_i = 0 \text{ if date } < \text{election date} \] so that the receipt of treatment \(T_i\) is determined by the threshold \(c\) (election day) of the continuous variable \(W_i\) (date), the so called running variable. The idea of regression regression discontinuity design is to use observations with a \(W_i\) close to \(c\) for the estimation of \(\beta_1\). \(\beta_1\) is the average treatment effect for observations with \(W_i = c\) which is assumed to be a good approximation to the overall treatment effect. In other words, \(\beta_1\) gives us the average change of news coverage of media outlet \(i\) after the election day. Since interaction terms \(T_iDn_{j}\) are included, we can estimate the treatment effect for each party.
2. Structural topic model
Parties want the media agenda to be congruent with their own agenda to define the issue-based criteria on which they will be evaluated by voters ( Eberl, 2017 ). Thus, parties instrumentalize their press releases in order to highlight issues that they are perceived to be competent on, that they “own” and that are important to their voters ( Kepplinger & Maurer, 2004 ). Editors can select from this universe and decide which of these topics will be discussed in the news. In that sense the ideological content of a newspaper refers to the extent to which the topics promoted by the parties correlate with the topics discussed in the news articles.
To discover the latent topics in the corpus of press releases and news articles, a structural topic modeling (STM) developed by Roberts (2016) is applied. The STM is an unsupervised machine learning approach that models topics as multinomial distributions of words and documents as multinomial distributions of topics, allowing to incorporate external variables that effect both, topical content and topical prevalence.
2.1 Results of the STM
To explore the words associated with each topic we use the words with the highest probability in each topic. As we included the source type (press release or news paper) as a control for the topical content (the word distribution of each topic), we have two different labels for each topic.
Each document has a probability distribution over all topics, e.g.
3. Cosine similarity
Cosine similarity is built on the geometric definition of the dot product of two vectors.
\[ \text{cosine similarity} = \text{cos}(\theta)=\frac{a*b}{||a|| ||b||} \]
calc_cos_sim <- function(doc_code,
gamma_mat = gamma) {
# find the doc
doc_col_index <- which(colnames(gamma_mat) == as.character(doc_code))
# get the week of the doc
year_week_target <- filter(model_df, doc_index == doc_code)$year_week
# get all doc_codes that are in the same week
doc_code_pairs <- filter(model_df, year_week == year_week_target)$doc_index
# calculate cosine similarity for each document based on gamma
# apply(..., 2) iterates over the columns of a matrix
cos_sims <- apply(gamma_mat[, doc_code_pairs], 2,
FUN = function(y) cosine_sim(gamma_mat[, doc_col_index], y)
)
# return results as data frame
data_frame(
year_week = year_week_target,
doc_code1 = doc_code,
doc_code2 = as.numeric(names(cos_sims)),
cos_sim = cos_sims
) %>%
filter(doc_code2 != doc_code) # remove self reference
}Calculate the cosine similarity for our sample document and all other documents and display the top 5 documents (the 5 documents most similar).
calc_cos_sim(sample_doc) %>%
mutate(doc_index = doc_code2) %>%
left_join(.,
model_df %>%
select(title, source, doc_index, type),
by = "doc_index"
) %>%
filter(type == "press") %>%
arrange(desc(cos_sim)) %>%
top_n(5, cos_sim) %>%
select(title, source, cos_sim)Finally, the function will be applied to all documents.
run_cos_sim_loop <- function(target) {
small_df <- model_df %>% filter(source == target)
doc_indices <- small_df$doc_index
print(paste("Number of documents:", length(doc_indices)))
rm(cosine_distances)
i <- 1
for (x in doc_indices) {
print(i)
temp <- calc_cos_sim(x, gamma)
# append the results to a data frame
cosine_distances <- if (!exists("cosine_distances")) temp else rbind(cosine_distances, temp)
i <- i + 1
}
cosine_distances
}4. Estimate Model
library(broom)
library(magrittr)
library(tidyverse)
rm(list = ls())
load("../output/model40_weekly_source.Rda")
election_date <- as.Date("24Sep2017", "%d%b%Y")
model_df <- model_df %>%
dplyr::mutate(
doc_index = as.numeric(rownames(.)),
election_dummy = as.factor(ifelse(date <= election_date, "pre", "post")),
year_week = lubridate::floor_date(date, "1 week")
)df_prep <- function(cosine_df) {
cosine_df %>%
left_join(., model_df %>%
transmute(
doc_code1 = doc_index,
source1 = source, type1 = type,
date1 = date
),
by = "doc_code1"
) %>%
left_join(., model_df %>%
transmute(
doc_code2 = doc_index,
source2 = source, type2 = type,
date2 = date
),
by = "doc_code2"
) %>%
group_by(date1, source2, type2) %>%
summarise(cos_sim = mean(cos_sim, na.rm = TRUE)) %>%
ungroup() %>%
mutate(election_dummy = as.factor(ifelse(date1 <= election_date, "pre", "post")))
}4.2. OLS dummy
calc_dumm <- function(dataframe, target_source) {
temp_df <- dataframe %>%
filter(type2 == "press")
model_outcome <- temp_df %$%
lm(log(cos_sim) ~ source2)
}# ----------------------------
target_source <- "DIE WELT"
load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(welt)
welt_model <- calc_dumm(cosine_distances_df, target_source)
target_source <- "stern.de"
load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(stern)
stern_model <- calc_dumm(cosine_distances_df, target_source)
target_source <- "ZEIT ONLINE"
load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(zeit)
zeit_model <- calc_dumm(cosine_distances_df, target_source)
target_source <- "Handelsblatt"
load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(handelsblatt)
handelsblatt_model <- calc_dumm(cosine_distances_df, target_source)
target_source <- "FOCUS Online"
load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(focus)
focus_model <- calc_dumm(cosine_distances_df, target_source)
target_source <- "Bild.de"
load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(bild)
bild_model <- calc_dumm(cosine_distances_df, target_source)
target_source <- "SPIEGEL ONLINE"
load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(spiegel)
spiegel_model <- calc_dumm(cosine_distances_df, target_source)stargazer::stargazer(welt_model, stern_model, zeit_model, handelsblatt_model, focus_model, bild_model, spiegel_model,
column.labels = c("DIE WELT", "stern.de", "ZEIT ONLINE", "Handelsblatt", "FOCUS Online", "Bild.de", "SPIEGEL ONLINE"),
dep.var.labels = paste0("Cosine similarity of topic distribution"),
single.row = TRUE,
#font.size = "tiny",
type = "text")##
## ============================================================================================================================================================================================
## Dependent variable:
## ------------------------------------------------------------------------------------------------------------------------------------------------------------------------
## Cosine similarity of topic distribution
## DIE WELT stern.de ZEIT ONLINE Handelsblatt FOCUS Online Bild.de SPIEGEL ONLINE
## (1) (2) (3) (4) (5) (6) (7)
## --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
## source2B90/GRÜNE -0.249*** (0.036) -0.235*** (0.045) -0.213*** (0.055) 0.049 (0.048) -0.333*** (0.038) -0.326*** (0.054) -0.229*** (0.044)
## source2CDU -0.290*** (0.033) -0.203*** (0.042) -0.262*** (0.051) 0.014 (0.045) -0.313*** (0.036) -0.311*** (0.051) -0.248*** (0.041)
## source2DIE LINKE -0.298*** (0.033) -0.150*** (0.042) -0.222*** (0.051) 0.059 (0.045) -0.315*** (0.036) -0.311*** (0.051) -0.225*** (0.041)
## source2FDP -0.267*** (0.033) -0.177*** (0.042) -0.197*** (0.051) 0.050 (0.045) -0.321*** (0.036) -0.323*** (0.051) -0.227*** (0.041)
## source2SPD -0.304*** (0.034) -0.193*** (0.043) -0.259*** (0.052) 0.011 (0.045) -0.350*** (0.036) -0.357*** (0.051) -0.296*** (0.042)
## Constant -1.746*** (0.024) -1.972*** (0.030) -1.749*** (0.036) -1.835*** (0.031) -1.812*** (0.025) -1.730*** (0.036) -1.861*** (0.029)
## --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
## Observations 1,376 1,382 1,408 1,204 1,432 1,326 1,428
## R2 0.082 0.027 0.025 0.003 0.088 0.050 0.042
## Adjusted R2 0.078 0.023 0.021 -0.002 0.085 0.046 0.038
## Residual Std. Error 0.366 (df = 1370) 0.463 (df = 1376) 0.570 (df = 1402) 0.457 (df = 1198) 0.401 (df = 1426) 0.547 (df = 1320) 0.461 (df = 1422)
## F Statistic 24.339*** (df = 5; 1370) 7.507*** (df = 5; 1376) 7.137*** (df = 5; 1402) 0.617 (df = 5; 1198) 27.594*** (df = 5; 1426) 13.769*** (df = 5; 1320) 12.406*** (df = 5; 1422)
## ============================================================================================================================================================================================
## Note: *p<0.1; **p<0.05; ***p<0.014.2. Regression discontinuity model
Calculate a regression discontinuity model for each newspaper \(i\).
\[ \text{CosineSimilarity}_{i}=\beta_0+\beta_1T_i+\beta_2W_{centered}+\beta_nDn_{j}+\beta_{n+n}T_iDn_{j}+\epsilon_i \]
where \(D2_{j}, D3_{j}, ... ,Dn_{j}\) represent dummy variables for a political party \(j\) and $W_{centered} = $ date - election day.
\[ T_i = 1 \text{ if date } >= \text{election date} \\ T_i = 0 \text{ if date } < \text{election date} \]
plot_data <- function(dataframe) {
cosine_distances_df %>%
filter(type2 == "press") %>%
ggplot(aes(date1, cos_sim, color = election_dummy)) +
geom_point(show.legend = F) +
geom_vline(xintercept = election_date) +
ggthemes::theme_clean() +
labs(y = "Cosine similarity of topic distribution", x = "") +
facet_wrap(~source2)
}calc_rd_dummy<- function(dataframe, target_source) {
temp_df <- dataframe %>%
filter(type2 == "press") %>%
mutate(
X_centered = I(date1 - election_date),
treatedTRUE = ifelse(date1 >= election_date, 1, 0))
model_outcome <- temp_df %$%
lm(log(cos_sim) ~ treatedTRUE + source2 + X_centered)
}
calc_rd_dummy_interaction <- function(dataframe, target_source) {
temp_df <- dataframe %>%
filter(type2 == "press") %>%
mutate(
X_centered = I(date1 - election_date),
treatedTRUE = ifelse(date1 >= election_date, 1, 0))
model_outcome <- temp_df %$%
lm(log(cos_sim) ~ treatedTRUE * source2 + X_centered)
}target_source <- "DIE WELT"
#load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(welt)
welt_model1 <- calc_rd_dummy(cosine_distances_df, target_source)
welt_model2 <- calc_rd_dummy_interaction(cosine_distances_df, target_source)
target_source <- "stern.de"
#load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(stern)
stern_model1 <- calc_rd_dummy(cosine_distances_df, target_source)
stern_model2 <- calc_rd_dummy_interaction(cosine_distances_df, target_source)
target_source <- "ZEIT ONLINE"
#load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(zeit)
zeit_model1 <- calc_rd_dummy(cosine_distances_df, target_source)
zeit_model2 <- calc_rd_dummy_interaction(cosine_distances_df, target_source)
target_source <- "Handelsblatt"
#load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(handelsblatt)
handelsblatt_model1 <- calc_rd_dummy(cosine_distances_df, target_source)
handelsblatt_model2 <- calc_rd_dummy_interaction(cosine_distances_df, target_source)
target_source <- "FOCUS Online"
#load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(focus)
focus_model1 <- calc_rd_dummy(cosine_distances_df, target_source)
focus_model2 <- calc_rd_dummy_interaction(cosine_distances_df, target_source)
target_source <- "Bild.de"
#load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(bild)
bild_model1 <- calc_rd_dummy(cosine_distances_df, target_source)
bild_model2 <- calc_rd_dummy_interaction(cosine_distances_df, target_source)
target_source <- "SPIEGEL ONLINE"
#load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(spiegel)
spiegel_model1 <- calc_rd_dummy(cosine_distances_df, target_source)
spiegel_model2 <- calc_rd_dummy_interaction(cosine_distances_df, target_source)4.1.1. Without interaction terms
\[ \text{CosineSimilarity}_{i}=\beta_0+\beta_1T_i+\beta_nDn_{j}+\epsilon_i \]
stargazer::stargazer(welt_model1, stern_model1, zeit_model1, handelsblatt_model1, focus_model1, bild_model1, spiegel_model1,
column.labels = c("DIE WELT", "stern.de", "ZEIT ONLINE", "Handelsblatt", "FOCUS Online", "Bild.de", "SPIEGEL ONLINE"),
dep.var.labels = paste0("Cosine similarity of topic distribution"),
single.row = TRUE,
#font.size = "tiny",
type = "text")##
## ==============================================================================================================================================================================================
## Dependent variable:
## --------------------------------------------------------------------------------------------------------------------------------------------------------------------------
## Cosine similarity of topic distribution
## DIE WELT stern.de ZEIT ONLINE Handelsblatt FOCUS Online Bild.de SPIEGEL ONLINE
## (1) (2) (3) (4) (5) (6) (7)
## ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
## treated -0.102** (0.040) -0.123** (0.051) -0.173*** (0.058) -0.193*** (0.052) -0.032 (0.042) -0.003 (0.057) 0.025 (0.048)
## source2B90/GRÜNE -0.262*** (0.035) -0.238*** (0.045) -0.243*** (0.054) 0.034 (0.048) -0.337*** (0.038) -0.321*** (0.055) -0.228*** (0.044)
## source2CDU -0.290*** (0.033) -0.203*** (0.042) -0.262*** (0.050) 0.014 (0.044) -0.313*** (0.036) -0.311*** (0.051) -0.248*** (0.041)
## source2DIE LINKE -0.298*** (0.033) -0.150*** (0.042) -0.222*** (0.050) 0.059 (0.044) -0.315*** (0.036) -0.311*** (0.051) -0.225*** (0.041)
## source2FDP -0.267*** (0.033) -0.177*** (0.042) -0.197*** (0.050) 0.050 (0.044) -0.321*** (0.036) -0.323*** (0.051) -0.227*** (0.041)
## source2SPD -0.307*** (0.034) -0.191*** (0.043) -0.266*** (0.051) 0.010 (0.045) -0.350*** (0.036) -0.354*** (0.051) -0.296*** (0.042)
## X_centered 0.00002 (0.0003) 0.001*** (0.0003) -0.001* (0.0004) 0.001*** (0.0003) 0.0001 (0.0003) 0.001 (0.0004) -0.0002 (0.0003)
## Constant -1.692*** (0.030) -1.919*** (0.038) -1.646*** (0.045) -1.743*** (0.040) -1.796*** (0.032) -1.738*** (0.045) -1.873*** (0.037)
## ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
## Observations 1,376 1,382 1,408 1,204 1,432 1,326 1,428
## R2 0.099 0.035 0.078 0.014 0.089 0.055 0.042
## Adjusted R2 0.094 0.030 0.073 0.008 0.085 0.050 0.037
## Residual Std. Error 0.363 (df = 1368) 0.461 (df = 1374) 0.555 (df = 1400) 0.455 (df = 1196) 0.401 (df = 1424) 0.545 (df = 1318) 0.462 (df = 1420)
## F Statistic 21.427*** (df = 7; 1368) 7.158*** (df = 7; 1374) 16.901*** (df = 7; 1400) 2.457** (df = 7; 1196) 19.874*** (df = 7; 1424) 10.939*** (df = 7; 1318) 8.903*** (df = 7; 1420)
## ==============================================================================================================================================================================================
## Note: *p<0.1; **p<0.05; ***p<0.01Interpretation:
DIE WELT
- Treatment effect: After the election, cosine similarity of news coverage drops by 0.01 on average.
- Dummies: Compared to AfD, news coverage with all other parties is less similar
Same interpretation for SPIEGEL ONLINE, ZEIT ONLINE
stern.de
- Treatment effect: not significant
- Dummies: Compared to AfD, news coverage with all other parties is less similar
Same interpretation for Bild.de, FOCUS Online
Handelsblatt
- Treatment effect: not significant
- Dummies: Compared to AfD, news coverage with all other parties is more similar
4.1.2. With interaction terms
\[ \text{CosineSimilarity}_{i}=\beta_0+\beta_1T_i+\beta_nDn_{j}+\beta_{n+n}T_iDn_{j}+\epsilon_i \]
The interaction term \(T_i*Dn\) means that the slope can vary on either side of the treatment threshold for each party.
- The coefficient \(\beta_1\) is how the intercept jumps (the RDD effect)
- \(\beta_3\) is how the slope changes for each party
stargazer::stargazer(welt_model2, stern_model2, zeit_model2, handelsblatt_model2, focus_model2, bild_model2, spiegel_model2,
column.labels = c("DIE WELT", "stern.de", "ZEIT ONLINE", "Handelsblatt", "FOCUS Online", "Bild.de", "SPIEGEL ONLINE"),
dep.var.labels = paste0("Cosine similarity of topic distribution"),
single.row = TRUE,
#font.size = "tiny",
type = "text")##
## ==============================================================================================================================================================================================================
## Dependent variable:
## ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
## Cosine similarity of topic distribution
## DIE WELT stern.de ZEIT ONLINE Handelsblatt FOCUS Online Bild.de SPIEGEL ONLINE
## (1) (2) (3) (4) (5) (6) (7)
## --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
## treated -0.113* (0.058) -0.099 (0.074) -0.092 (0.086) -0.089 (0.077) -0.064 (0.062) -0.170* (0.087) 0.046 (0.071)
## source2B90/GRÜNE -0.292*** (0.048) -0.237*** (0.061) -0.208*** (0.075) 0.106* (0.062) -0.401*** (0.053) -0.458*** (0.078) -0.245*** (0.061)
## source2CDU -0.286*** (0.048) -0.145** (0.061) -0.194*** (0.075) 0.131** (0.062) -0.298*** (0.053) -0.382*** (0.078) -0.195*** (0.061)
## source2DIE LINKE -0.285*** (0.048) -0.121** (0.061) -0.156** (0.075) 0.123** (0.062) -0.298*** (0.053) -0.410*** (0.078) -0.159*** (0.061)
## source2FDP -0.300*** (0.048) -0.220*** (0.061) -0.177** (0.075) 0.047 (0.062) -0.384*** (0.053) -0.447*** (0.078) -0.277*** (0.061)
## source2SPD -0.302*** (0.048) -0.164*** (0.061) -0.194*** (0.075) 0.077 (0.062) -0.371*** (0.053) -0.508*** (0.078) -0.287*** (0.061)
## X_centered -0.00001 (0.0003) 0.001*** (0.0003) -0.001* (0.0004) 0.001*** (0.0003) 0.00004 (0.0003) 0.001 (0.0004) -0.0002 (0.0003)
## treatedTRUE:source2B90/GRÜNE 0.072 (0.072) 0.005 (0.091) -0.060 (0.108) -0.161* (0.098) 0.145* (0.077) 0.257** (0.110) 0.048 (0.089)
## treatedTRUE:source2CDU -0.007 (0.066) -0.111 (0.084) -0.124 (0.101) -0.234*** (0.088) -0.027 (0.072) 0.123 (0.102) -0.098 (0.083)
## treatedTRUE:source2DIE LINKE -0.024 (0.066) -0.055 (0.084) -0.121 (0.101) -0.128 (0.088) -0.031 (0.072) 0.171* (0.102) -0.124 (0.083)
## treatedTRUE:source2FDP 0.061 (0.066) 0.082 (0.084) -0.037 (0.101) 0.007 (0.088) 0.116 (0.072) 0.215** (0.102) 0.093 (0.083)
## treatedTRUE:source2SPD -0.011 (0.067) -0.053 (0.085) -0.134 (0.102) -0.136 (0.090) 0.038 (0.073) 0.270*** (0.103) -0.017 (0.084)
## Constant -1.687*** (0.037) -1.932*** (0.047) -1.690*** (0.057) -1.795*** (0.049) -1.778*** (0.041) -1.641*** (0.059) -1.884*** (0.047)
## --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
## Observations 1,376 1,382 1,408 1,204 1,432 1,326 1,428
## R2 0.101 0.039 0.080 0.023 0.095 0.062 0.049
## Adjusted R2 0.093 0.031 0.072 0.014 0.088 0.053 0.041
## Residual Std. Error 0.363 (df = 1363) 0.461 (df = 1369) 0.555 (df = 1395) 0.454 (df = 1191) 0.400 (df = 1419) 0.545 (df = 1313) 0.461 (df = 1415)
## F Statistic 12.766*** (df = 12; 1363) 4.691*** (df = 12; 1369) 10.091*** (df = 12; 1395) 2.375*** (df = 12; 1191) 12.446*** (df = 12; 1419) 7.174*** (df = 12; 1313) 6.029*** (df = 12; 1415)
## ==============================================================================================================================================================================================================
## Note: *p<0.1; **p<0.05; ***p<0.01Interpretation:
DIE WELT
ZEIT ONLINE
Handelsblatt
FOCUS Online - Treatment effect: No significant change (on average)
Dummy variables: The cosine similarity is on average less for all parties compared to AfD. E.g. compared to the AfD, the cosine similarity of topics is 45,8% lower for B90/Grüne, meaning that topic coverage of Bild.de is 45,8% lower for B90/Grüne compared to AfD
Interaction terms: The effect of the election on topic coverage is 25,7% higher for B90/Grüne than for AfD.
\[ \text{CosineSimilarity}_{FOCUS}(D_{B90}=1)=\beta_0-0.17*T-0.458+0.257*T \\ \text{CosineSimilarity}_{FOCUS}(D_{B90}=1)= \beta_0-0.458*T(-0.17+0.257) \] \[ \text{CosineSimilarity}_{Welt}(D_{B90}=0)=\beta_0-0.17*T \\ \]
Bild.de - Treatment effect: The cosine similarity drops after the election day by 17% on average (over all parties)
SPIEGEL ONLIINE
Compare models
DIE WELT
The result shows a Df of 5 (indicating that the more complex model has 5 additional parameter) and a p-value of 0.274. This means that adding the interaction terms to the model did NOT lead to a significantly improved fit over the model.
We can do the same check for all other media outlet models
stern.de
ZEIT ONLINE
Handelsblatt
FOCUS ONLINE
Bild.de
SPIEGEL ONLINE
4.1.4 Isolated models for each party / media pair
\[ \text{CosineSimilarity}_{i}=\beta_0+\beta_1T_i+\beta_2W_{centered}+\beta_3T_i*W_{centered}+\epsilon_i \]
where \(W_{centered}\) = date - election day. The interaction term \(T_i*W_{centered}\) means that the slope can vary on either side of the treatment threshold.
- The coefficient \(\beta_1\) is how the intercept jumps (the RDD effect)
- \(\beta_3\) is how the slope changes
calc_rd_models <- function(dataframe, target_source) {
temp_df <- dataframe %>%
mutate(
X_centered = I(date1 - election_date),
treatedTRUE = ifelse(date1 >= election_date, 1, 0))
afd <- temp_df %>%
filter(source2 == "AfD") %$%
lm(cos_sim ~ treatedTRUE * X_centered)
cdu <- temp_df %>%
filter(source2 == "CDU") %$%
lm(cos_sim ~ treatedTRUE * X_centered)
spd <- temp_df %>%
filter(source2 == "SPD") %$%
lm(cos_sim ~ treatedTRUE * X_centered)
fdp <- temp_df %>%
filter(source2 == "FDP") %$%
lm(cos_sim ~ treatedTRUE * X_centered)
linke <- temp_df %>%
filter(source2 == "DIE LINKE")%$%
lm(cos_sim ~ treatedTRUE * X_centered)
gruene <- temp_df %>%
filter(source2 == "B90/GRÜNE") %$%
lm(cos_sim ~ treatedTRUE * X_centered)
stargazer::stargazer(afd, cdu, spd, fdp, gruene, linke,
column.labels = c("AfD", "CDU", "SPD", "FDP", "B90/GRÜNE", "DIE LINKE"),
dep.var.labels = paste0("Cosine similarity of topic distribution - ",target_source),
single.row = TRUE,
font.size = "tiny",
type = "text")
}DIE WELT
# ----------------------------
target_source <- "DIE WELT"
#load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(welt)
calc_rd_models(cosine_distances_df, target_source)##
## ===========================================================================================================================================================
## Dependent variable:
## ------------------------------------------------------------------------------------------------------------------------------------
## Cosine similarity of topic distribution - DIE WELT
## AfD CDU SPD FDP B90/GRÜNE DIE LINKE
## (1) (2) (3) (4) (5) (6)
## -----------------------------------------------------------------------------------------------------------------------------------------------------------
## treated -0.020 (0.012) -0.028** (0.014) -0.024* (0.014) 0.004 (0.015) 0.045** (0.018) -0.017 (0.011)
## X_centered 0.0001 (0.0001) -0.0002 (0.0002) 0.0001 (0.0001) -0.0002 (0.0002) -0.0003* (0.0002) -0.0002 (0.0001)
## treatedTRUE:X_centered -0.0003* (0.0002) 0.0005** (0.0002) -0.0001 (0.0002) 0.0002 (0.0002) -0.0001 (0.0002) 0.0003* (0.0002)
## Constant 0.201*** (0.009) 0.139*** (0.010) 0.153*** (0.010) 0.136*** (0.011) 0.131*** (0.010) 0.137*** (0.008)
## -----------------------------------------------------------------------------------------------------------------------------------------------------------
## Observations 240 240 227 240 189 240
## R2 0.069 0.059 0.035 0.008 0.053 0.065
## Adjusted R2 0.058 0.047 0.022 -0.004 0.038 0.053
## Residual Std. Error 0.046 (df = 236) 0.054 (df = 236) 0.052 (df = 223) 0.058 (df = 236) 0.055 (df = 185) 0.042 (df = 236)
## F Statistic 5.863*** (df = 3; 236) 4.909*** (df = 3; 236) 2.676** (df = 3; 223) 0.660 (df = 3; 236) 3.468** (df = 3; 185) 5.494*** (df = 3; 236)
## ===========================================================================================================================================================
## Note: *p<0.1; **p<0.05; ***p<0.01Interpretation:
- After the election, the topic coverage is less similar with CDU & SPD and MORE similar with B90/Grüne
stern.de
target_source <- "stern.de"
#load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(stern)
calc_rd_models(cosine_distances_df, target_source)##
## ======================================================================================================================================================
## Dependent variable:
## -------------------------------------------------------------------------------------------------------------------------------
## Cosine similarity of topic distribution - stern.de
## AfD CDU SPD FDP B90/GRÜNE DIE LINKE
## (1) (2) (3) (4) (5) (6)
## ------------------------------------------------------------------------------------------------------------------------------------------------------
## treated -0.014 (0.012) -0.006 (0.015) 0.004 (0.015) 0.015 (0.017) 0.017 (0.018) 0.002 (0.014)
## X_centered 0.0003** (0.0001) -0.0004** (0.0002) -0.0001 (0.0002) -0.00005 (0.0002) -0.00001 (0.0002) -0.0003* (0.0001)
## treatedTRUE:X_centered -0.0002 (0.0002) 0.001*** (0.0002) 0.0002 (0.0002) 0.0003 (0.0002) 0.00001 (0.0002) 0.001*** (0.0002)
## Constant 0.160*** (0.009) 0.106*** (0.011) 0.119*** (0.011) 0.116*** (0.012) 0.115*** (0.010) 0.113*** (0.010)
## ------------------------------------------------------------------------------------------------------------------------------------------------------
## Observations 241 241 228 241 190 241
## R2 0.023 0.055 0.005 0.050 0.023 0.033
## Adjusted R2 0.011 0.043 -0.008 0.038 0.007 0.021
## Residual Std. Error 0.046 (df = 237) 0.056 (df = 237) 0.057 (df = 224) 0.066 (df = 237) 0.054 (df = 186) 0.051 (df = 237)
## F Statistic 1.873 (df = 3; 237) 4.555*** (df = 3; 237) 0.383 (df = 3; 224) 4.141*** (df = 3; 237) 1.432 (df = 3; 186) 2.738** (df = 3; 237)
## ======================================================================================================================================================
## Note: *p<0.1; **p<0.05; ***p<0.01Interpretation:
- No significant change
ZEIT ONLINE
target_source <- "ZEIT ONLINE"
#load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(zeit)
calc_rd_models(cosine_distances_df, target_source)##
## =============================================================================================================================================================
## Dependent variable:
## --------------------------------------------------------------------------------------------------------------------------------------
## Cosine similarity of topic distribution - ZEIT ONLINE
## AfD CDU SPD FDP B90/GRÜNE DIE LINKE
## (1) (2) (3) (4) (5) (6)
## -------------------------------------------------------------------------------------------------------------------------------------------------------------
## treated -0.026 (0.016) -0.033 (0.022) -0.012 (0.019) 0.009 (0.024) 0.028 (0.027) -0.017 (0.017)
## X_centered 0.0004** (0.0002) -0.001** (0.0002) -0.0003 (0.0002) -0.001** (0.0003) -0.0005** (0.0002) -0.0005** (0.0002)
## treatedTRUE:X_centered -0.001*** (0.0002) 0.001*** (0.0003) 0.0001 (0.0003) 0.001** (0.0003) 0.0002 (0.0003) 0.001** (0.0002)
## Constant 0.228*** (0.012) 0.147*** (0.017) 0.159*** (0.014) 0.149*** (0.018) 0.147*** (0.016) 0.149*** (0.013)
## -------------------------------------------------------------------------------------------------------------------------------------------------------------
## Observations 246 246 233 246 191 246
## R2 0.111 0.073 0.082 0.024 0.057 0.100
## Adjusted R2 0.100 0.062 0.070 0.012 0.042 0.089
## Residual Std. Error 0.063 (df = 242) 0.088 (df = 242) 0.074 (df = 229) 0.096 (df = 242) 0.087 (df = 187) 0.066 (df = 242)
## F Statistic 10.092*** (df = 3; 242) 6.355*** (df = 3; 242) 6.784*** (df = 3; 229) 1.974 (df = 3; 242) 3.796** (df = 3; 187) 8.998*** (df = 3; 242)
## =============================================================================================================================================================
## Note: *p<0.1; **p<0.05; ***p<0.01Interpretation:
- No significant change
Handelsblatt
target_source <- "Handelsblatt"
#load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(handelsblatt)
calc_rd_models(cosine_distances_df, target_source)##
## ===================================================================================================================================================
## Dependent variable:
## ----------------------------------------------------------------------------------------------------------------------------
## Cosine similarity of topic distribution - Handelsblatt
## AfD CDU SPD FDP B90/GRÜNE DIE LINKE
## (1) (2) (3) (4) (5) (6)
## ---------------------------------------------------------------------------------------------------------------------------------------------------
## treated 0.011 (0.013) -0.063*** (0.022) -0.005 (0.019) -0.00000 (0.023) 0.041* (0.024) -0.008 (0.018)
## X_centered 0.0001 (0.0001) -0.0001 (0.0002) -0.00005 (0.0002) -0.0001 (0.0003) -0.0002 (0.0002) -0.0002 (0.0002)
## treatedTRUE:X_centered -0.0002 (0.0002) 0.001** (0.0003) 0.0001 (0.0003) 0.001 (0.0003) -0.0002 (0.0003) 0.0005* (0.0002)
## Constant 0.169*** (0.010) 0.189*** (0.017) 0.177*** (0.014) 0.172*** (0.018) 0.171*** (0.015) 0.171*** (0.013)
## ---------------------------------------------------------------------------------------------------------------------------------------------------
## Observations 211 211 198 211 162 211
## R2 0.018 0.080 0.002 0.044 0.034 0.023
## Adjusted R2 0.004 0.066 -0.013 0.031 0.015 0.009
## Residual Std. Error 0.045 (df = 207) 0.079 (df = 207) 0.068 (df = 194) 0.082 (df = 207) 0.070 (df = 158) 0.063 (df = 207)
## F Statistic 1.293 (df = 3; 207) 5.970*** (df = 3; 207) 0.133 (df = 3; 194) 3.208** (df = 3; 207) 1.829 (df = 3; 158) 1.621 (df = 3; 207)
## ===================================================================================================================================================
## Note: *p<0.1; **p<0.05; ***p<0.01Interpretation:
- After the election, the topic coverage is less similar with CDU and MORE similar with B90/Grüne
FOCUS Online
target_source <- "FOCUS Online"
#load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(focus)
calc_rd_models(cosine_distances_df, target_source)##
## ======================================================================================================================================================
## Dependent variable:
## -------------------------------------------------------------------------------------------------------------------------------
## Cosine similarity of topic distribution - FOCUS Online
## AfD CDU SPD FDP B90/GRÜNE DIE LINKE
## (1) (2) (3) (4) (5) (6)
## ------------------------------------------------------------------------------------------------------------------------------------------------------
## treated -0.008 (0.012) -0.016 (0.014) -0.017 (0.012) -0.005 (0.015) 0.047*** (0.017) -0.018 (0.011)
## X_centered 0.0002 (0.0001) -0.0002 (0.0002) 0.0002 (0.0001) 0.00002 (0.0002) -0.00005 (0.0002) -0.0001 (0.0001)
## treatedTRUE:X_centered -0.0004*** (0.0002) 0.0004* (0.0002) -0.0003 (0.0002) 0.0001 (0.0002) -0.0004* (0.0002) 0.0002 (0.0002)
## Constant 0.189*** (0.009) 0.127*** (0.011) 0.141*** (0.009) 0.129*** (0.011) 0.121*** (0.010) 0.129*** (0.008)
## ------------------------------------------------------------------------------------------------------------------------------------------------------
## Observations 250 250 237 250 195 250
## R2 0.053 0.029 0.017 0.007 0.051 0.038
## Adjusted R2 0.042 0.018 0.005 -0.005 0.036 0.026
## Residual Std. Error 0.047 (df = 246) 0.057 (df = 246) 0.050 (df = 233) 0.060 (df = 246) 0.055 (df = 191) 0.044 (df = 246)
## F Statistic 4.596*** (df = 3; 246) 2.486* (df = 3; 246) 1.378 (df = 3; 233) 0.580 (df = 3; 246) 3.417** (df = 3; 191) 3.245** (df = 3; 246)
## ======================================================================================================================================================
## Note: *p<0.1; **p<0.05; ***p<0.01Interpretation:
- After the election, the topic coverage is MORE similar with B90/Grüne
Bild.de
target_source <- "Bild.de"
#load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(bild)
calc_rd_models(cosine_distances_df, target_source)##
## ==================================================================================================================================================
## Dependent variable:
## ---------------------------------------------------------------------------------------------------------------------------
## Cosine similarity of topic distribution - Bild.de
## AfD CDU SPD FDP B90/GRÜNE DIE LINKE
## (1) (2) (3) (4) (5) (6)
## --------------------------------------------------------------------------------------------------------------------------------------------------
## treated -0.034** (0.016) 0.003 (0.017) -0.003 (0.018) 0.004 (0.021) 0.050* (0.027) -0.004 (0.017)
## X_centered 0.0002 (0.0002) -0.0004** (0.0002) 0.0001 (0.0002) -0.0003 (0.0002) -0.0004 (0.0003) -0.0003 (0.0002)
## treatedTRUE:X_centered -0.0003 (0.0002) 0.001** (0.0002) -0.00002 (0.0003) 0.001* (0.0003) 0.0001 (0.0003) 0.001** (0.0002)
## Constant 0.216*** (0.012) 0.127*** (0.013) 0.145*** (0.014) 0.130*** (0.016) 0.129*** (0.017) 0.132*** (0.013)
## --------------------------------------------------------------------------------------------------------------------------------------------------
## Observations 232 232 220 232 178 232
## R2 0.055 0.033 0.002 0.023 0.022 0.021
## Adjusted R2 0.043 0.020 -0.012 0.010 0.005 0.008
## Residual Std. Error 0.062 (df = 228) 0.067 (df = 228) 0.072 (df = 216) 0.081 (df = 228) 0.086 (df = 174) 0.065 (df = 228)
## F Statistic 4.445*** (df = 3; 228) 2.564* (df = 3; 228) 0.139 (df = 3; 216) 1.786 (df = 3; 228) 1.292 (df = 3; 174) 1.658 (df = 3; 228)
## ==================================================================================================================================================
## Note: *p<0.1; **p<0.05; ***p<0.01Interpretation:
- After the election, the topic coverage is LESS simiar with AfD and MORE similar with B90/Grüne
SPIEGEL ONLINE
target_source <- "SPIEGEL ONLINE"
#load(paste0("../output/cosine_dist_", target_source, ".Rda"))
cosine_distances_df <- df_prep(spiegel)
calc_rd_models(cosine_distances_df, target_source)##
## =====================================================================================================================================================
## Dependent variable:
## ------------------------------------------------------------------------------------------------------------------------------
## Cosine similarity of topic distribution - SPIEGEL ONLINE
## AfD CDU SPD FDP B90/GRÜNE DIE LINKE
## (1) (2) (3) (4) (5) (6)
## -----------------------------------------------------------------------------------------------------------------------------------------------------
## treated 0.005 (0.013) -0.015 (0.016) 0.004 (0.013) 0.034* (0.017) 0.079*** (0.021) -0.013 (0.012)
## X_centered 0.0001 (0.0001) -0.0003* (0.0002) 0.0001 (0.0001) -0.0002 (0.0002) -0.001*** (0.0002) -0.0001 (0.0001)
## treatedTRUE:X_centered -0.0003 (0.0002) 0.001*** (0.0002) -0.0002 (0.0002) 0.0003 (0.0002) 0.0001 (0.0003) 0.0003 (0.0002)
## Constant 0.171*** (0.010) 0.124*** (0.012) 0.133*** (0.010) 0.116*** (0.013) 0.104*** (0.013) 0.132*** (0.009)
## -----------------------------------------------------------------------------------------------------------------------------------------------------
## Observations 249 249 237 249 195 249
## R2 0.012 0.043 0.013 0.031 0.078 0.031
## Adjusted R2 -0.0003 0.031 -0.0001 0.019 0.063 0.019
## Residual Std. Error 0.052 (df = 245) 0.062 (df = 245) 0.051 (df = 233) 0.070 (df = 245) 0.067 (df = 191) 0.047 (df = 245)
## F Statistic 0.978 (df = 3; 245) 3.628** (df = 3; 245) 0.995 (df = 3; 233) 2.609* (df = 3; 245) 5.354*** (df = 3; 191) 2.578* (df = 3; 245)
## =====================================================================================================================================================
## Note: *p<0.1; **p<0.05; ***p<0.01Interpretation:
- After the election, the topic coverage is MORE similar with FDP & B90/Grüne