Stats dashboard

[nschneid]

It would be nice to have something along the lines of https://stats.aclrollingreview.org/

For example:

• Total number of papers, volumes, events, unique authors by year
• Distribution of number of authors per paper per year
• Pie chart of venues (by number of papers) per year
• Number, percentage of authorships with ORCIDs over time
• Number, percentage of person IDs with an ORCID over time
• Number, percentage of extended-slug person IDs that are semantic (degree institution) vs. automatic (ORCID suffix). If manual verification always introduces a semantic suffix, that reveals how many person entries are automatic.
• Number, percentage of (papers, volumes) with DOIs by year
• Number of unverified author pages
• Paper appearances on verified vs. unverified author pages

[nschneid]

@mbollmann Is the API stable such that work could begin on this (if we find a volunteer)?

[mbollmann]

Sure! Just the ORCID stuff might not be possible yet until we get the author transition done.

[nschneid]

OK the API is really easy to use. :) After importing/loading the anthology:

from acl_anthology import Anthology
anthology = Anthology.from_repo()

from collections import defaultdict, Counter
papers_by_year, doi_papers_by_year, volumes_by_year, doi_volumes_by_year, events_by_year = Counter(), Counter(), Counter(), Counter(), Counter()
uniq_authors_by_year = defaultdict(set)
for p in anthology.papers():
    papers_by_year[p.year] += 1
    if p.doi:
        doi_papers_by_year[p.year] += 1
    else:
        doi_papers_by_year[p.year] += 0
    for a in p.authors:
        person = anthology.resolve(a)
        uniq_authors_by_year[p.year].add(person)
for v in anthology.volumes():
    volumes_by_year[v.year] += 1
    if v.doi:
        doi_volumes_by_year[v.year] += 1
    else:
        doi_volumes_by_year[v.year] += 0
for e in anthology.events.keys():
    events_by_year[e.rsplit('-',1)[1]] += 1
num_uniq_authors_by_year = {year: len(auths) for year,auths in uniq_authors_by_year.items()}

import pandas
data = pandas.DataFrame({'papers': papers_by_year, 'doi_papers': doi_papers_by_year, 'uauthors': num_uniq_authors_by_year, 'evts': events_by_year, 'vols': volumes_by_year, 'doi_vols': doi_volumes_by_year})
data['pctdoi_papers'] = data['doi_papers']/data['papers']
data['pctdoi_vols'] = data['doi_vols']/data['vols']

>>> data
      papers  doi_papers  uauthors  evts  vols  doi_vols  pctdoi_papers  pctdoi_vols
1952      26           0        17     1     1         0           0.00          0.0
1956      11           0        11     1     1         0           0.00          0.0
1957      26           0        38     1     1         0           0.00          0.0
1960      81           0        53     1     2         0           0.00          0.0
1961      39           0        47     1     1         0           0.00          0.0
...      ...         ...       ...   ...   ...       ...            ...          ...
1973      64           0        88     1     2         0           0.00          0.0
1974      20           0        18     1     3         0           0.00          0.0
1975     115           0       114     2     6         0           0.00          0.0
1977      37           0        35     3     5         0           0.00          0.0
1979      50          28        54     3     2         0           0.56          0.0

[64 rows x 6 columns]

>>> data[data['doi_papers']>100]
      papers  doi_papers  uauthors  evts  vols  doi_vols  pctdoi_papers  pctdoi_vols
1984     231         116       248     6     7         0       0.502165     0.000000
1992     624         101       879    10    16         0       0.161859     0.000000
1996     619         204       862    14    19         2       0.329564     0.105263
1997     668         150       979    16    36         0       0.224551     0.000000
1998    1099         319      1232    22    37         0       0.290264     0.000000
1999     543         119       848    14    24         0       0.219153     0.000000
2000    1254         294      1904    24    46         0       0.234450     0.000000
2001     814         162      1379    18    43         0       0.199017     0.000000
2002    1216         245      2029    23    47         0       0.201480     0.000000
2003    1186         318      1886    24    65         0       0.268128     0.000000
2004    1924         137      3260    27    62         0       0.071206     0.000000
2005    1261         279      2131    22    54         1       0.221253     0.018519
2006    2174         257      3502    35    86         0       0.118215     0.000000
2007    1601         131      2624    30    63         1       0.081824     0.015873
2014    3639        1881      6423    74   119        71       0.516900     0.596639
2015    2958        1956      5283    75   105        54       0.661258     0.514286
2016    4220        1979      7642    79   115        52       0.468957     0.452174
2017    3424        2050      6382    75   122        55       0.598715     0.450820
2018    4755        2817      9206    71   118        28       0.592429     0.237288
2019    4994        4194     10146    82   146         4       0.839808     0.027397
2020    7171        4325     15589   140   169         9       0.603124     0.053254
2021    7033        5157     15174   142   166         1       0.733258     0.006024
2022    8649        5376     20008   171   206        12       0.621575     0.058252
2023    8982        7033     21160   137   174         1       0.783010     0.005747
2024   12031        7637     29880   168   205        39       0.634777     0.190244
2025    8332        6104     25608   111   126        63       0.732597     0.500000

Now to find someone to work on visualizing this....

[mbollmann]

Streamlit app deployed via https://streamlit.io/cloud ? I know that Streamlit itself is super easy to use, though I've never deployed an app with the cloud service.

[nschneid]

Mocked up some plots with Plotly: stats.ipynb

Could we integrate these directly into the site without a separate app?

Another thing that would be nice to have is a dashboard tracking DOI (and later ORCID) addition progress over time, but that would require saving the stats periodically as I assume "date DOI added" is not a piece of information stored in the XML.

[mbollmann]

Could we integrate these directly into the site without a separate app?

I was initially thinking this should be integrated, but now I’m not so sure.

Pros:

• Always in sync with the data on the website
• Unified layout

Cons:

• Might increase build times? (maybe negligible, not sure)
• Might increase maintenance burden if we add visualization libraries as dependencies just for this part

The last point assumes that we generate more than static images, which I really think we should, as toolkits that allow for interactive visualizations are so ubiquitous and easy to use these days. I think a simple, separate Streamlit app will add much more value over just statically generated plots.

Another thing that would be nice to have is a dashboard tracking DOI (and later ORCID) addition progress over time, but that would require saving the stats periodically as I assume "date DOI added" is not a piece of information stored in the XML.

I wonder how easily & reliably this could be inferred from a git-blame.