Coalition analysis (KOALA) 2017

class: center, middle, inverse, title-slide

# Coalition analysis (KOALA) 2017
## German election coalition probabilities
### Alexander Bauer | Andreas Bender
### StaBLab, LMU München
### 2017/09/20

---

background-image: url("image/Koala_Logo_Schrift_hell.png")
background-position: 97% 2%
background-size: 150px
class: inverse
# Outline
<hr style='border:solid darkgray 1px; height:0px; margin-top:-20px; margin-bottom:50px'/>

- Motivation

- Implementation (Backend)

- Implementation (Frontend)

- Outlook & sources

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
# Motivation
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>
When covering the election, media outlets (TV and print) mostly focus on
questions like

> Which parties will pass the 5% threshold and enter the "Bundestag" (German parliament)?

and

> Which parties will form the governing coalition
(currently Union - SPD, so called *grand coalition*)?

For the 2017 election also of special interest

> Which party will have the 3rd largest share of votes?

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
# Motivation
To answer these questions, pundits and writers usually focus on raw voting intention polls:
&nbsp;
&nbsp;
&nbsp;

> "Which party would you vote for if election was today?"

&nbsp;
&nbsp;

<table>
<caption>Last FORSA poll before the 2013 election (September, 20th); n = 1995</caption>
 <thead>
  <tr>
   <th style="text-align:right;"> Union </th>
   <th style="text-align:right;"> SPD </th>
   <th style="text-align:right;"> Greens </th>
   <th style="text-align:right;"> FDP </th>
   <th style="text-align:right;"> The Left </th>
   <th style="text-align:right;"> Pirates </th>
   <th style="text-align:right;"> AfD </th>
   <th style="text-align:right;"> Others </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:right;"> 40% </td>
   <td style="text-align:right;"> 26% </td>
   <td style="text-align:right;"> 10% </td>
   <td style="text-align:right;"> 5% </td>
   <td style="text-align:right;"> 9% </td>
   <td style="text-align:right;"> 2% </td>
   <td style="text-align:right;"> 4% </td>
   <td style="text-align:right;"> 4% </td>
  </tr>
</tbody>
</table>

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
# Motivation
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>
Interpretation of raw polls is problematic for several reasons

1. Sample uncertainty is ignored (even if the sample is representative, we would
expect individual polls to deviate from the true shares).
  - Exhibit 1.a: <a href="https://twitter.com/btwahltrend/status/905031505671585793"> pollytix - Koalitionsrechner</a>
  - Exhibit 1.b: <a href="https://twitter.com/tagesschau/status/910485863649472512">
	Tagesschau </a>

2. Redistribution of votes is ignored (all votes for parties that do not
pass the 5% threshold are redistributed proportionally to parties that pass
the threshold).
  - Exhibit 2: <a href="http://www.faz.net/aktuell/politik/bundestagswahl/bundestagswahl-schwarz-gelb-verliert-die-mehrheit-15143172.html">FAZ</a>

3. Overreaction to individual polls (Some polls can be "off" or only depict
the voting intention in a very short time-period; different weighting methods
used by different pollsters)
  - see Exhibits 1.b and 2

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
# Example: BTW13
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>
<table>
<caption>Last FORSA poll before the 2013 election (September, 20th); n = 1995</caption>
 <thead>
  <tr>
   <th style="text-align:right;"> Union </th>
   <th style="text-align:right;"> SPD </th>
   <th style="text-align:right;"> Greens </th>
   <th style="text-align:right;"> FDP </th>
   <th style="text-align:right;"> The Left </th>
   <th style="text-align:right;"> Pirates </th>
   <th style="text-align:right;"> AfD </th>
   <th style="text-align:right;"> Others </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:right;"> 40% </td>
   <td style="text-align:right;"> 26% </td>
   <td style="text-align:right;"> 10% </td>
   <td style="text-align:right;"> 5% </td>
   <td style="text-align:right;"> 9% </td>
   <td style="text-align:right;"> 2% </td>
   <td style="text-align:right;"> 4% </td>
   <td style="text-align:right;"> 4% </td>
  </tr>
</tbody>
</table>

<table>
 <thead>
  <tr>
   <th style="text-align:right;"> Union </th>
   <th style="text-align:right;"> SPD </th>
   <th style="text-align:right;"> Greens </th>
   <th style="text-align:right;"> FDP </th>
   <th style="text-align:right;"> The Left </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:right;"> 44.44% </td>
   <td style="text-align:right;"> 28.89% </td>
   <td style="text-align:right;"> 11.11% </td>
   <td style="text-align:right;"> 5.56% </td>
   <td style="text-align:right;"> 10% </td>
  </tr>
</tbody>
</table>

```r
forsa_13$percent
round(gtools::rdirichlet(3, 1995*forsa_13$percent+0.5), 4)
```
```
##        [,1]   [,2]   [,3]   [,4]   [,5]   [,6]   [,7]   [,8]
## [1,] 0.4022 0.2634 0.1014 0.0439 0.0894 0.0157 0.0425 0.0414
## [2,] 0.4206 0.2526 0.0915 0.0446 0.0949 0.0210 0.0336 0.0413
## [3,] 0.4152 0.2629 0.0965 0.0518 0.0872 0.0154 0.0356 0.0353
```

```r
round(gtools::rdirichlet(3, 20*forsa_13$percent+0.5), 4)
```
```
##        [,1]   [,2]   [,3]   [,4]   [,5]   [,6]   [,7]   [,8]
## [1,] 0.4861 0.2415 0.0551 0.0123 0.0663 0.0148 0.0969 0.0270
## [2,] 0.2443 0.1557 0.2510 0.1100 0.0548 0.0205 0.0898 0.0738
## [3,] 0.4347 0.2295 0.0750 0.1335 0.0505 0.0016 0.0732 0.0021
```

- Coalition probabilities can be obtained by calculating the area underneath
the probability distribution for `$x> 50%$`
- Or simpler: `$P(\text{event}|\text{sample})=\frac{\# \text{simulations with event}}{\# \text{simulations}}$`

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
# KOALA: Coalitions Analysis
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>
In [our approach](http://www.stablab.stat.uni-muenchen.de/forschung/koala/methodik.pdf) we
- aggregate polls from different pollsters within a 14-day time-window
(pooled survey)

- Calculate the Posterior Dirichlet distribution (based on Multinomial Likelihood
and flat/uninformative Dirichlet Prior)

- Calculate "secondary" properties (e.g. probability that Union-FDP would have simple majority) via Monte-Carlo sampling

- Simulate election outcomes from known Posterior (based on current pooled survey)

- `$P(\text{event}|sample) = \frac{\# \text{event}}{\text{number of simulations}}$`

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
# Implementation (Backend)
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>
- Backend implemented in the R-package `coalitions`
	(see [Workflow vignette](https://adibender.github.io/coalitions/articles/workflow.html))

- scrapes wahlrecht.de for (new) polls

- (calculates *pooled* sample)

- calculate and sample from Posteriori

- Redistribute votes below 5% threshold and calculate Seats based on
	method by [Sainte-Lague-Scheppers](http://www.wahlrecht.de/verfahren/rangmasszahlen.html) (German Law)

- Calculate coalition probabilities

```r
devtools::install_github("adibender/coalitions")
```

- Surveys returned as *nested* tidy data set (`tibble`)

```r
surveys <- get_surveys()
surveys
```

```
## # A tibble: 7 x 2
##   pollster   surveys           
##   <chr>      <list>            
## 1 allensbach <tibble [41 × 5]> 
## 2 emnid      <tibble [222 × 5]>
## 3 forsa      <tibble [231 × 5]>
## 4 fgw        <tibble [82 × 5]> 
## 5 gms        <tibble [96 × 5]> 
## 6 infratest  <tibble [107 × 5]>
## 7 insa       <tibble [301 × 5]>
```

```r
surveys %>%
	unnest() %>%
	select(-start, -end)
```

```
## # A tibble: 1,080 x 4
##    pollster   date       respondents survey          
##    <chr>      <date>           <dbl> <list>          
##  1 allensbach 2018-01-25        1221 <tibble [7 × 3]>
##  2 allensbach 2017-12-21        1443 <tibble [7 × 3]>
##  3 allensbach 2017-11-30        1299 <tibble [7 × 3]>
##  4 allensbach 2017-10-25        1454 <tibble [7 × 3]>
##  5 allensbach 2017-09-22        1074 <tibble [7 × 3]>
##  6 allensbach 2017-09-19        1083 <tibble [7 × 3]>
##  7 allensbach 2017-09-06        1043 <tibble [7 × 3]>
##  8 allensbach 2017-08-22        1421 <tibble [7 × 3]>
##  9 allensbach 2017-07-18        1403 <tibble [7 × 3]>
## 10 allensbach 2017-06-20        1437 <tibble [7 × 3]>
## # ... with 1,070 more rows
```

```r
surveys %>%
	unnest() %>%
	slice(1) %>%
	unnest() %>%
	select(-start, -end)
```

```
## # A tibble: 7 x 6
##   pollster   date       respondents party  percent votes
##   <chr>      <date>           <dbl> <chr>    <dbl> <dbl>
## 1 allensbach 2018-01-25        1221 cdu      34.0  415  
## 2 allensbach 2018-01-25        1221 spd      21.0  256  
## 3 allensbach 2018-01-25        1221 greens   10.5  128  
## 4 allensbach 2018-01-25        1221 fdp      10.0  122  
## 5 allensbach 2018-01-25        1221 left      8.50 104  
## 6 allensbach 2018-01-25        1221 afd      12.0  147  
## 7 allensbach 2018-01-25        1221 others    4.00  48.8
```

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px

### pooled survey
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>

```r
pooled_survey <- surveys %>% pool_surveys()
pooled_survey %>% select(-start, -end)
```

```
## # A tibble: 7 x 6
##   pollster date       respondents party  percent votes
##   <chr>    <date>           <dbl> <chr>    <dbl> <dbl>
## 1 pooled   2018-02-13        2533 afd      13.9    352
## 2 pooled   2018-02-13        2533 cdu      31.4    796
## 3 pooled   2018-02-13        2533 fdp       9.44   239
## 4 pooled   2018-02-13        2533 greens   12.5    317
## 5 pooled   2018-02-13        2533 left     10.5    266
## 6 pooled   2018-02-13        2533 others    4.20   106
## 7 pooled   2018-02-13        2533 spd      18.0    455
```

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
### Draw from Posterior
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>

```r
draws <- pooled_survey  %>%
	draw_from_posterior(seed=123)
draws[1:6, ]
```

```
## # A tibble: 6 x 7
##     afd   cdu    fdp greens  left others   spd
##   <dbl> <dbl>  <dbl>  <dbl> <dbl>  <dbl> <dbl>
## 1 0.135 0.327 0.0841  0.126 0.116 0.0438 0.169
## 2 0.125 0.323 0.0951  0.126 0.104 0.0392 0.188
## 3 0.148 0.320 0.0934  0.126 0.101 0.0425 0.169
## 4 0.133 0.307 0.0940  0.125 0.106 0.0462 0.189
## 5 0.142 0.312 0.0957  0.116 0.103 0.0411 0.189
## 6 0.132 0.322 0.0900  0.126 0.103 0.0423 0.184
```

```r
## calculate probabilities to pass 5% threshold
draws %>% summarize_all(funs(mean(.>0.05)))
```

```
## # A tibble: 1 x 7
##     afd   cdu   fdp greens  left others   spd
##   <dbl> <dbl> <dbl>  <dbl> <dbl>  <dbl> <dbl>
## 1  1.00  1.00  1.00   1.00  1.00 0.0274  1.00
```

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
### Redistribution and seats calculation
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>

```r
seats <- get_seats(draws, pooled_survey, distrib.fun=sls, hurdle=0.05)
seats
```

```
## # A tibble: 60,000 x 3
##      sim party  seats
##    <int> <chr>  <int>
##  1     1 afd       84
##  2     1 cdu      204
##  3     1 fdp       53
##  4     1 greens    79
##  5     1 left      73
##  6     1 spd      105
##  7     2 afd       78
##  8     2 cdu      201
##  9     2 fdp       59
## 10     2 greens    78
## # ... with 59,990 more rows
```

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
### Calculate coalition probabilities
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>

```r
probs <- seats %>% have_majority() %>%
	calculate_probs(coalitions=list(c("cdu", "fdp"), c("cdu", "fdp", "greens")))
probs
```

```
## # A tibble: 2 x 2
##   coalition      probability
##   <chr>                <dbl>
## 1 cdu_fdp                  0
## 2 cdu_fdp_greens         100
```

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
### Wrapper
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>

```r
set.seed(123)
pooled_survey %>%
	nest(party:votes, .key="survey") %>%
	get_probabilities(seed=123, nsim=1e4) %>%
	unnest()
```

```
## # A tibble: 6 x 4
##   pollster date       coalition       probability
##   <chr>    <date>     <chr>                 <dbl>
## 1 pooled   2018-02-13 cdu                       0
## 2 pooled   2018-02-13 cdu_fdp                   0
## 3 pooled   2018-02-13 cdu_fdp_greens          100
## 4 pooled   2018-02-13 spd                       0
## 5 pooled   2018-02-13 left_spd                  0
## 6 pooled   2018-02-13 greens_left_spd           0
```

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
# Visualization
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>
- We visualize the posterior via "ridgeline plots" ([formerly Joy plots](http://serialmentor.com/blog/2017/9/15/goodbye-joyplots))

- Uses `ggplot`, `ggridges`, `gganimate`
- (click <a href="test-joy.gif">here</a> for animated version; also featured
at [Spektrum.de](http://www.spektrum.de/news/koennen-die-aktuellen-umfragen-noch-falschliegen/1503163))

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
# Joy/Ridges-Plot (Implementation)
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>

```r
gg_distrb <- ggplot(schw_gelb, aes(x = percent, y = date2,
	group=date2, frame=date, cumulative=TRUE, fill=..x..>50)) +
	geom_density_ridges_gradient(scale = 10, size = 0.25) +
	geom_vline(xintercept = 50, lty=1, lwd=1.2, col="grey90") +
	scale_fill_manual(
		name   = "Rechnerische Sitzemehrheit",
		breaks = c("FALSE", "TRUE"),
		labels = c("Nein", "Ja"),
		values = c("#d1d0ce", "steelblue") ) +
	theme(legend.position = "bottom")  +
	scale_x_continuous(labels = function(x) paste0(x, "%")) +
	scale_y_continuous(trans = rev_date) +
	guides(fill=guide_legend(override.aes=list(alpha=1))) +
	theme(
		axis.text    = element_text(size = rel(1.3)),
		axis.title.y = element_blank(),
		axis.title.x = element_blank(),
		legend.text  = element_text(size = rel(1.2)),
		legend.title = element_text(size = rel(1.3))) +
	labs(
		title = "Wahrscheinlichkeitsverteilung der Stimmanteile für Union - FDP",
		subtitle = "(seit Oktober 2016)")
```

```r
gg_distrb <- ggplot(schw_gelb, aes(x = percent, y = date,
	group=date, frame=date, cumulative=TRUE, fill=..x..>50)) +
   geom_density_ridges_gradient(scale = 10, size = 0.25) +
	# geom_vline(xintercept = 50, lty=1, lwd=1.2, col="grey90") +
	# scale_fill_manual(
	# 	name   = "Rechnerische Sitzemehrheit",
	# 	breaks = c("FALSE", "TRUE"),
	# 	labels = c("Nein", "Ja"),
	# 	values = c("#d1d0ce", "steelblue") ) +
	# theme(legend.position = "bottom")  +
	# scale_x_continuous(labels = function(x) paste0(x, "%")) +
	# scale_y_continuous(trans = rev_date) +
	# guides(fill=guide_legend(override.aes=list(alpha=1))) +
	# theme(
	# 	axis.text    = element_text(size = rel(1.3)),
	# 	axis.title.y = element_blank(),
	# 	axis.title.x = element_blank(),
	# 	legend.text  = element_text(size = rel(1.2)),
	# 	legend.title = element_text(size = rel(1.3))) +
	# labs(
	# 	title = "Wahrscheinlichkeitsverteilung der Stimmanteile für Union - FDP",
	# 	subtitle = "(seit Oktober 2016)")
```

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
# Animation
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>
- **If you're able to `ggplot`, you are able to `gganimate`!**
- The **`gganimate`** package currently not on CRAN, install via:

```r
devtools::install_github("drgtwo/gganimate")
```

```r
library(ggplot2)
library(ggridges)
library(gganimate)
gg_distrb <- ggplot(schw_gelb, aes(x = percent, y = date,
 		group=date, frame=date, cumulative=TRUE, fill=..x..>50)) +
	geom_density_ridges_gradient(scale = 10, size = 0.25)
gganimate(gg_distrb, "output.gif", interval=.2, ani.width=600)
```

- Set GIF parameters (width, height, etc. via `ani.options`)
- Control speed of the animation via `interval` argument </br>
(lower values `$\rightarrow$` higher speed)
- Don't forget to set the `frame` argument in the call to `ggplot` (this is
the variable over which the animation will iterate)
- By setting `cumulative=TRUE` current frame also contains previous frames
- Note: `alpha` argument does not work with `*_gradient` geoms

---
background-image: url("image/Koala_Logo_Schrift_hell.png")
background-position: 97% 2%
background-size: 150px
class: inverse
# Frontend implementation
<hr style='border:solid darkgray 1px; height:0px; margin-top:-20px; margin-bottom:50px'/>

1) Creating a homepage with Shiny

2) Setting up the server with Shiny Server

3) APIs and stuff: tweetR and googlesheets

4) Keep the website running

---
class: inverse
<iframe src='http://koala.stat.uni-muenchen.de' style='width:100%; height:96%'/>

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
# Frontend - Shiny
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>

Shiny in a nutshell:
- R package by RStudio

- Web application framework

- Creation of interactive dashboards, running R in the background

- Resources: <a href='https://www.meetup.com/de-DE/Applied-R-Munich/events/226912554/' target='_blank'>Applied R Shiny Meetup</a>,  <a href='http://shiny.rstudio.com/' target='_blank'>shiny.rstudio.com</a>

- Tip: Use <a href='https://rstudio.github.io/shinydashboard/' target='_blank'>shinydashboard</a> for a more appealing dashboard UI

<br/>
Why Shiny?

- Easy integration of interactive R output and calculations

- No need for learning another language, Shiny creates the HTML, CSS and JavaScript for you!

```R
library(shiny)

ui <- fluidPage(
  selectInput("plotType_picker", "Pick plot type",
              choices = c("boxplot","histogram")),
  plotOutput("my_plot")
)
server <- function(input, output) {
  x <- rnorm(100)
  output$my_plot <- renderPlot({
    if (input$plotType_picker == "boxplot") {
      boxplot(x)
    } else
      hist(x)
  })
}

shinyApp(ui = ui, server = server)
```

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
# Frontend - Shiny Server
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>

Shiny Server in a nutshell:

- Linux-based open source web server by RStudio

- Access to the homepage starts an R process with a Shiny app on the server

<br/><br/>
Resources:

- <a href='https://www.rstudio.com/products/shiny/shiny-server2/' target='_blank'>Official homepage</a>

Setting up Shiny Server (on 64-bit Ubuntu 12.04+): (<a href='http://docs.rstudio.com/shiny-server/' target='_blank'>administrator's guide</a>)

1) Install R and all needed R packages on the server

2) Install Shiny Server (<a href='https://www.rstudio.com/products/shiny/download-server/' target='_blank'>install guide</a>)
```bash
$ sudo apt-get install gdebi-core
$ sudo wget https://download3.rstudio.org/ubuntu-12.04/x86_64/shiny-server-1.5.3.838-amd64.deb
$ sudo gdebi shiny-server-1.5.3.838-amd64.deb
```

3) Customize the shiny-server.conf file to your needs
  <div style="margin-top:-20px">
  - &nbsp;Tip 1: <p style="font-family:Source Code Pro; display:inline-block">'sanitize_errors false;'</p> gets you clearer error messages
  </div>
  <div style="margin-top:-40px; margin-left:20px">
  - Tip 2: <p style="font-family:Source Code Pro; display:inline-block">'app_idle_timeout 0;'</p> saves startup time of the R process
  </div>

4) Put the Shiny files inside the path specified in shiny.server.conf

Capabilities of the free version (Shiny Server Open Source):

- For non-commercial projects

- Up to 20 users simultaneously

- No multiple R processes! <br/> &#8594; code efficiently and precalculate results where possible

<br/>
<a href='https://www.rstudio.com/products/shiny/shiny-server/' target='_blank'>(R-based) alternatives:</a>

- Shiny Server Pro: Commercial use, multiple R processes etc.

- <a href='https://www.shinyapps.io' target='_blank'>shinyapps.io</a>:
  - Deploy Shiny apps on RStudio servers
  - Free version limited to 25 hours of use per month!

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
# Frontend - tweetR <img src="image/twitter_dunkel.png" style='height:50px; margin-left:20px; margin-bottom:-5px'/>
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>

The tweetR package in a nutshell:

- Our use case: Send tweets with new results

- Resources: <a href='http://geoffjentry.hexdump.org/twitteR.pdf' target='_blank'>user vignette</a>, <a href='https://github.com/geoffjentry/twitteR' target='_blank'>tweetR on GitHub</a>

- Alternative (more modern) package: <a href='http://rtweet.info/' target='_blank'>rtweet</a>

Sending a Tweet with tweetR:

1) Register a new Twitter app on <a href='https://apps.twitter.com/' target='_blank'>apps.twitter.com</a>

2) Use the credentials to do the authorization with R
```R
setup_twitter_oauth(consumer_key = "your_consumer_key",
                    consumer_secret = "your_consumer_secret",
                    access_token = "your_access_token",
                    access_secret = "your_access_secret")
```

3) Start tweeting!
```R
tweet(message = "Tweet tweet", mediaPath = "my_picture.png")
```

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
# Frontend - googlesheets <img src="image/googlesheets.png" style='height:60px; margin-left:20px; margin-bottom:-10px'/>
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>

The googlesheets package in a nutshell:

- Our use case: Offering an API for our results

- Resources: <a href='https://github.com/jennybc/googlesheets' target='_blank'>googlesheets on GitHub</a>

Exporting a table to Google Sheets with googlesheets:

1) Extract your credentials:
```R
auth_info <- gs_auth()
saveRDS(auth_info, file = "auth_info.rds")
```

2) Use the credentials to do the authorization with R
```R
gs_auth(token = "auth_info.rds")
```

3) Start uploading!
```R
my_table <- data.frame("person"      = c("Sepp","Uli","Franz"),
                       "likes_koala" = c("yes", "yes", "yes"))
write.csv(my_table, file = "my_table.csv")
gs_upload("my_table.csv", sheet_title = "my_googleSheet",
          overwrite = TRUE)
```

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
# Frontend - Keep the website running
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>

Automation of the server:

- We check hourly if new surveys are available and update the results

- Implementation: see next slide

- Tip: Automatic error notification using <a href='https://www.pushbullet.com/' target='_blank'>Pushbullet</a> <img src="image/pushbullet.png" style="height:40px; margin-bottom:-13px; margin-left:15px"/>

- Supports notifications to all major (Desktop and mobile) systems

- R package: <a href='https://github.com/eddelbuettel/rpushbullet' target='_blank'>RPushbullet on GitHub</a>

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
<div style='margin-top:-25px'/> 
### Implementation of the automation process
1) Set up the R script
```R
while (1 < 2) { # do something eternally
  # Step 1: check for new surveys and perform calculations
  update_results()

# Step 2: Update services with new results
  if (new_results) { # if new results are available
    send_tweet()
    export_googleSheets()

# restart the server to fetch the new results (on Ubuntu 15.04+)
    system("sudo systemctl restart shiny-server")
  }

# Step 3: Rest for an hour
  Sys.sleep(60*10)
}
```

2) Start the R script on the server
```bash
R CMD BATCH update_results_everyHour.R &
```

---
background-image: url("image/Koala_Logo_Schrift_hell.png")
background-position: 97% 2%
background-size: 150px
class: inverse
# Outlook & sources
<hr style='border:solid darkgray 1px; height:0px; margin-top:-20px; margin-bottom:50px'/>

1) Future plans

2) Sources

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
# Outlook - Future plans
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>

- Make frontend R package (shiny app) open source

- Simplify data handling, Create online data base (with API), etc.

- Extend the framework to:
  - other elections (regional and international)
  - improve interactive experience
  - make @tagesschau (and others) use our methods!

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
# Sources
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>

General sources:

- Raw voting intention polls: <a href='http://www.wahlrecht.de' target='_blank'>www.wahlrecht.de</a>

- Our slides are powered by <a href='https://slides.yihui.name/xaringan/' target='_blank'>Xaringan</a>

- <a href='https://www.tidyverse.org' target='_blank'>tidyverse</a> (previously hadleyverse)

<br/>
--

How to reach us?

- Visit us on <a href='koala.stat.uni-muenchen.de' target='_blank'>koala.stat.uni-muenchen.de</a>

- Contact us directly: <a href='mailto:koala@stat.uni-muenchen.de'>koala@stat.uni-muenchen.de</a>

- Feel free to contribute on <a href='https://github.com/adibender/coalitions' target='_blank'>GitHub</a>

---
background-color: whitesmoke
background-image: url("image/Koala_Logo_Schrift.png")
background-position: 97% 2%
background-size: 150px
# Preparing for election day
<hr style='border:solid gray 1px; height:0px; margin-top:-20px; margin-bottom:25px'/>

Keep in mind:

- We do not make predictions!

- Many voters (~40%) still undecided

&nbsp;
&nbsp;
&nbsp;
<br/><br/>

--
###  So, stay tuned...
<br/>
... and <a href='https://twitter.com/KOALA_LMU' target='_blank'>follow us on Twitter</a> :-)