Where do we go from here? We will keep this short because, if the reader has made it this far, the answer really is: in whatever direction the reader’s research interests lie. The main purpose of this book was to introduce a general framework and workflow that enables us to enhance competence theories with fully specified performance/processing components. The resulting competence-performance theories can furthermore be embedded in Bayesian models, which enables us to fit them to data and quantitatively compare them in a systematic fashion. This Bayes+ACT-R+formal linguistics workflow of model development is in principle applicable to linguistic accounts of many syntactic and/or semantic phenomena—if suitable data can be obtained from properly designed experiments, which is far from trivial.

This being said, we think there are five specific directions worth pursuing in the near future:

  1. i.

    add more structure to the Bayesian models, for example, random effects for participants, grouping participants according to their strategies in self-paced reading tasks, etc.;

  2. ii.

    data-driven modeling: hand-coding models for specific experiments does not scale up well, and we should find ways to leverage syntactically and semantically annotated corpora to make the process of building ACT-R models for specific tasks and experiments more automatic and data-driven, and more easily comparable across tasks/experiments;

  3. iii.

    enrich the range of studied semantic phenomena—quantifiers, scope, binding, questions, attitude verbs, modals—and the range of semantic representations that are considered—(trees of) variable assignments in addition to or instead of DRSs, compositionally assembled higher-order terms in a suitable logical system etc.

    • relatively modest extensions of this framework could be used to build on the wealth of experimental results gathered in the last ten years or so and explicitly model and fit to data different theories of presupposition projection, scalar implicature computation etc., not to mention the large amount of experimental data about syntactic phenomena that is available in the literature;

  4. iv.

    provide a framework for integrating and comparing models and theories of language interpretation that have been developed in largely disparate traditions up to this point:

    • for example, the rise of distributional semantics and neural-network modeling work in formal semantics (Bowman 2016; McNally and Boleda 2017 among others), and linguistics more generally, raises a range of questions about what the appropriate division of labor is in natural language interpretation between symbolic and subsymbolic components; our Bayes+ACT-R+formal linguistics framework enables us to explore a range of hybrid models that would integrate both perspectives and that can be quantitatively compared, for example, models in which more of the cognitive heavy-lifting is performed either by symbolic components (chunks, rules) or subsymbolic components (base/spreading activation, rule utilities); see, for example, Marcus (2018) for a recent discussion of and arguments for hybrid (symbolic and subsymbolic) architectures;

    • a specific example would involve exploring hybrid representations for lexical items that would encode both structural information (like we have done throughout this book) and quantitative information, e.g., dense word embeddings of the kind proposed in Mikolov et al. (2013) or Pennington et al. (2014); these dense word embeddings could be used to modulate spreading activation for lexical items or for larger phrasal units;

    • incorporating drift-diffusion models (Ratcliff 1978; Ratcliff et al. 2017) into ACT-R (cf. (Van Maanen  et al. (2012))) and compare the resulting model(s) of language comprehension with other commonly used modeling choices;

    • yet another possibility is to systematically investigate rule learning for natural language interpretation: rules throughout this book were hand-coded, and no theory for how new rules are generated was put forth; this is a common feature of ACT-R modeling, but not a defining and necessary one: ACT-R does have a system for rule learning (production compilation) and we could go further by hypothesizing ‘rule-generating’ mechanisms;

    • similarly, ACT-R has a system for rule utility learning, but recent advances in reinforcement learning might contribute new insights to this component of the cognitive architecture.

  5. v.

    on the computational side, make improvements to enable faster estimation of posterior distributions for pyactr model parameters, e.g., by emulating pyactr models with neural networks, Gaussian Processes or other kinds of models; solutions along these lines could also enable us to do Approximate Bayesian Computation (ABC), that is, likelihood-free Bayesian inference for simulation-based models with intractable likelihoods, e.g., ACT-R models with various stochastic components turned on.

In addition, there are several ways in which ACT-R is showing its age for modeling natural language interpretation:

  • it has a rule-ordering architecture that effectively employs transformational models of the kind generative linguistics used in the ‘60s and ‘70s, and that we moved away from;

  • it has a fairly strict ban on hierarchical structures, rather than a softer one that would allow but penalize them, e.g., the way a probabilistic context free grammar penalizes deeper trees;

  • the underlying logic for facts/chunks is the logic of feature structures, basically a modal logic with features as modal operators and values as (atomic) non-modal sentence variables; in semantics, we have moved away from this type of theory construction with ‘local’-perspective logics, and more towards the ‘global’-perspective of classical (many-sorted) first-order or higher-order logic, which makes integrating ACT-R and formal semantics somewhat awkward.

However, ACT-R is a widely used hybrid (symbolic and subsymbolic) cognitive architecture and as such, it was the obvious choice for a framework in which to build mechanistic processing models and integrated competence-performance theories for natural language interpretation. As computational cognitive modeling for natural language phenomena develops further, we expect to see a critical reevaluation of a variety of architectural assumptions that we took for granted in the present work.