Castricato, Louis. Biderman, Stella. Thue, David. Cardona-Rivera, Rogelio. "Towards a Model-theoretic View of Narratives"
2021. Preprint. Available here.
In this paper, we propose the beginnings of a formal framework for modeling narrative qua narrative. Our framework affords the ability to discuss key qualities of stories and their communication, including the flow of information from a Narrator to a Reader, the evolution of a Reader's story model over time, and Reader uncertainty. We demonstrate its applicability to computational narratology by giving explicit algorithms for measuring the accuracy with which information was conveyed to the Reader and two novel measurements of story coherence.
Castricato, Louis. Frazier, Spencer. Balloch, Jonathan. Riedl, Mark. "Fabula Entropy Indexing: Objective Measures of Story Coherence."
2021. Preprint. Available here.
Automated story generation remains a difficult area of research because it lacks strong objective measures. Generated stories may be linguistically sound, but in many cases suffer poor narrative coherence required for a compelling, logically-sound story. To address this, we present Fabula Entropy Indexing (FEI), an evaluation method to assess story coherence by measuring the degree to which human participants agree with each other when answering true/false questions about stories. We devise two theoretically grounded measures of reader question-answering entropy, the entropy of world coherence (EWC), and the entropy of transitional coherence (ETC), focusing on global and local coherence, respectively. We evaluate these metrics by testing them on human-written stories and comparing against the same stories that have been corrupted to introduce incoherencies. We show that in these controlled studies, our entropy indices provide a reliable objective measure of story coherence.
Castricato, Louis. Fitz, Stephen. Shin, Gary. "Parameter-Efficient Neural Question Answering Models via Graph-Enriched Document Representations."
2020. Preprint. Available here.
As the computational footprint of modern NLP systems grows, it becomes in- creasingly important to arrive at more efficient models. We show that by employing graph convolutional document representation, we can arrive at a question answering system that performs comparably to, and in some cases exceeds the SOTA solutions, while using less than 5% of their resources in terms of trainable parameters. As it currently stands, a major issue in applying GCNs to NLP is document repre- sentation. In this paper, we show that a GCN enriched document representation greatly improves the results seen in HotPotQA, even when using a trivial topology. Our model (gQA), performs admirably when compared to the current SOTA, and requires little to no preprocessing. In "Is graph structure necessary for multi-hop reasoning?," the authors suggest that graph networks are not necessary for good performance in multi-hop QA. In this paper, we suggest that large language models are not necessary for good performance by showing a na ̈ıve implementation of a GCN performs comparably to SoTA models based on pretrained language models.
Orchard, Jeff. Castricato, Louis. "Combating Adversarial Inputs Using a Predictive-Estimator Network."
2017. ICONIPS. Best paper award. Available here.
Deep classification networks have shown great accuracy in classifying inputs. However, they fall prey to adversarial inputs, random inputs chosen to yield a classification with a high confidence. But percep- tion is a two-way process, involving the interplay between feedforward sensory input and feedback expectations. In this paper, we construct a predictive estimator (PE) network, incorporating generative (predictive) feedback, and show that the PE network is less susceptible to adversarial inputs. We also demonstrate some other properties of the PE network.