A prediction

You didn’t build that. – Barack Obama, July 13, 2012

Connectionism originates in psychology, but the “old connectionists” are mostly gone, having largely failed to pass on their ideology to their trainees, and there really aren’t many “young connectionists” to speak of. But, I predict that in the next few years we’ll see a bunch of psychologists of language—the ones who define themselves by their opposition to internalism, innateness, and generativism—become some of the biggest cheerleaders for large language models (LLMs). In fact, psychologists have not made substantial contributions to neural network modeling in many years. Virtually all the work on improving neural networks over the last few decades has been done by computer scientists who cared not a whit whether they had anything to do with human brains or cognitive plausibility.¹ (Sometimes they’ll put things like “…inspired by the human brain…” in the press releases, but we all know that’s just fluff.) At this point, psychology as a discipline has no more claim to neural networks than the Irish do to Gaul, and in the rather unlikely case that LLMs do end up furnishing deep truths about cognition, psychology as a discipline will have failed us by not following up on a promising lead. I think it will be particularly revealing if psychologists who previously worshipped at the Church of Bayes suddenly lose all interest in mathematical rigor and find themselves praying to the great Black Box. I want to say it now: if this happens—and I am starting to see signs that it will—those people will be cynics, haters, and trolls, and you shouldn’t pay them any mind.

Endnotes

I am also critical of machine learning pedagogy, and it is therefore interesting to see that those same computer scientists pushing things forward don’t seem to care much for machine learning as an academic discipline either.

Industry postdocs

I find the very idea of industry postdocs funny (funny-sad, though). Sure, it makes sense for the academy, with all of its scarcities, to make use of precarious, casualized post-graduate labor, but to extend this to the tech sector is vaguely monstrous. It’s extra funny (but funny-sad too) when you hear of a senior professor doing an industry postdoc at a company with a name like baz.ly during their sabbatical.

Neurolinguistic deprogramming

I venture to say most working linguists would reject—outright—strong versions of linguistic relativity and the Sapir-Whorf hypothesis, and would regard neuro-linguistic programming as pseudoscientific rubbish. This is of course in contrast to the general public: even the highly-educated take linguistic relativity as an obvious description of human life. Yet, it is not uncommon for the same linguists to endorse beliefs in the power of renaming that is hard to reconcile with the general disrepute of the vulgar Whorfian view the power of renaming assumes.

For instance, George Lakoff’s work on “framing” in politics argued that renaming social programs was the one weird trick needed to get Howard Dean into the White House. While this seems quaint in retrospect, his proposal was widely debated at the time. Pinker’s (sigh) takedown is necessary reading. The problem, of course, is that Lakoff ought to have provided, and ought to have been expected to provide, any evidence at all for a view of language widely regarded as untutored by his colleagues.

The case of renaming languages is a grayer one. I believe that one ought to call people what they want to be called, and that if stakeholders would prefer their language to be referred to as Tohono Oʼodham rather than Pápago, I am and will remain happy to oblige.¹ If African American Vernacular English is renamed to African American Language (as seems to be increasing common in scholarship), I will gladly follow suit. But I can’t imagine how it could be the case that the renaming represents a reconceptualization of either the language itself, or a change in how we study it. Indeed, it would be strange for the name of any language to reflect any interesting property of said language. French by any other name would still have V-to-T movement and liaison.

It may be that these acts of renaming have power. Indeed, I hope they do. But I have to suspect the opposite: they’re the sort of fiddling one does when one is out power, when one is struggling to believe that a better world is possible. And if I’m wrong, who is better suited to show that than the trained linguist?

Endnotes

Supposedly, the older name of the language comes from a pejorative used by a neighboring tribe, the Pima. Ba꞉bawĭkoʼa means, roughly ‘tepary bean eater’. The Spanish colonizers adapted this as Pápago. I feel like the gloss sounds like a cutting insult in English too, so I get why this exonym has fallen in disrepute.

Journal websites

It is now 2023, and virtually every journal I review for has a broken website, which further penalizes me for volunteer work I ought to be paid for. This is really unacceptable. Maybe some of the big publishers can take a tiny bite out of their massive revenues (Springer Nature apparently pulled down 1.72b USD in revenue in 2021) and invest it into actually testing their the CRUD apps.

Caffeine

I recently stopped consuming caffeine on a daily basis. For at least a dozen years, I’d had a cup of fully caffeinated coffee first thing pretty much every morning. And over the last few years, I also found myself getting a lot of pleasure out of a 3pm espresso shot. I quit because I hoped to improve my sleep. I understand from browsing the literature that caffeine actually has a reasonably long half- and quarter-life, and a morning cup really does negatively impact your sleep 14 hours later. I also understand that caffeine does not “give” you energy; it just temporarily causes your body to consume energy stores at a higher rate. This seems to have worked; I am certainly more refreshed in the morning than I used to be, and I am as active as ever. Only negative thinking and parties keep me up late now.

Having tried to quit caffeine before, I knew that I would have to titrate down gradually to avoid painful headaches. I therefore reduced my consumption gradually, over the course of two weeks, and didn’t experience much pain. I understood, of course, that there is a low-level addictive component to caffeine, the sort of thing that gives you transitory headaches if you don’t get your fix. What I didn’t understand, however, is the degree to which my addiction to caffeine (and that’s the right word here) had seeped into my higher-level consciousness. I found my mind coming up with elaborate justifications for why I needed caffeine. During the first few weeks, my mind was telling me that perhaps I’m just not as smart, handsome, clever, or strong without it. I recognize this as classic addict talk.

I have kept up my coffee ritual. As I have for many years, I start every morning by grinding 10g of fresh roasted beans, heating water to 205°, and using these to prepare about 12 oz of hot coffee. However, this coffee has no more than a tiny trace of caffeine thanks to the solvent-free “Swiss Water” diffusion process. My roaster provides a decent sample of different coffees prepared with this process (with no real markup over the caffeinated variety), including a nice fair trade Sumatran. I am also allowing myself to have one caffeinated cup (at least until I run out of caffeinated beans) a week on Friday morning just before I go the gym to lift weights.

I think I have to recommend going through this detox, if you’re in a state of mind where you can exert a bit of will power.

1-on-1 Zoom

If you’re just doing a “meeting” with one other person located in the same country, I don’t see the point of using Zoom. Ordinary phone lines are more reliable and have more familiar acoustic qualities (this is why VoIP sounds worse: unless you’re quite young, you’re probably far more familiar with the 8kHz sampling rate and whatever compression curve the phone system uses). Just call people on the phone!

Generalized capitalist realism

One of the most memorable books I’ve read over the last decade or so is Mark Fisher’s Capitalist Realism: Is There No Alternative? (2009). The book is a slim, 81-page pamphlet describing the feeling that “not only is capitalism the only viable political and economic system, but also that it is now impossible even to imagine a coherent alternative to it.” As Fisher explains, a lot of ideological work is done to prevent us from imagining alternatives, including the increasingly capitalist sheen of anti-capitalism, and there are a few areas—the overall non-response to climate change and biosphere-scale threats, for example—where capitalist realism ideology has failed to co-opt dissent, suggesting at least the possibility of an alternative on the horizon, even if Fisher himself does not imagine or present one.

A very clear example of capitalist realism can be found in the ethical altruism (EA) movement, which focuses on getting charity to the less well-off via existing capitalist structures. Singer (2015), the moment’s resident philosopher, justifies this by setting the probability of a viable alternative to capitalism surfacing in any reasonable time frame to be zero. Therefore the most good one can do is to ruthlessly accumulate wealth in the metropole and then give it away where it is most needed. Any synergies between the wealth of the first world and the dire economic conditions in the third world simply have to set aside.

Fisher’s term capitalist realism is a sort of pun on socialist realism, a term for idealized, realistic, literal art from 20th century socialist countries. His use of the term realism is (deliberately, I think) ironic, since both capitalist and socialist realism apply firm ideological filters to the real world. The continental philosophy stuff that this ultimately gets down to is a bit above my pay grade, but I think we can generalize the basic idea: X realism is an ideology that posits and enforces the hypothesis that there is no alternative to X.

If one is willing to go along with this, we can easily talk about, for instance, neural realism, which posits that there is simply no alternative to neural networks for machine learning. You can see this for instance in the debate between “deep learning fundamentalists” like LeCun and the rigor police like Rahimi (see Sproat 2022 for an entertaining discussion): LeCun does seem believe there to be no alternative to employing methods we do not understand with the scientific rigor that Rahimi demands, when it seems obvious that these technologies remain a small part of the overall productive economy. An even clearer example is the term foundation model, which has the fairly obvious connotation that they are crucial to the future of AI. Foundation model realism would also necesarily posit that there is no alternative and discard any disconfirming observation.

References

Fisher, M. 2009. Capitalist Realism: Is There No Alternative? Zero Books.
Singer, P. 2015. The Most Good You Can Do. Yale University Press.
Sproat, R. 2022. Boring problems are sometimes the most interesting. Computational Linguistics 48(2): 483-490.

Foundation models

It is widely admitted that the use of language in terms like formal language and language model tend to mislead neophytes, since they suggest the common-sense notion (roughly, e-language) rather than the narrow technical sense referring to a set of strings. Scholars at Stanford have been trying to push foundation model as an alternative to what were previously called large language models. But I don’t really like the implication—which I take to be quite salient—that such models ought to serve as the foundation for NLP, AI, whatever. I use large language models in my research, but not that often, and I actually don’t think they have to be part of every practitioner’s toolkit. I can’t help thinking that Stanford is trying to “make fetch happen”.

New-jack Elon haters

It’s on trend to hate on Elon Musk. This bugs me slightly, because I was doing it before it was cool. The thing any new-jack Elon hater should read is the 2015 (authorized!) biography by Ashlee Vance, entitled Elon Musk: Tesla, SpaceX, and the Quest for a Fantastic Future, which your local library almost certainly has. (There’s even an abridged YA edition.) Basically all the worst things you can imagine about Musk were already firmly documented by 2015. Repeated there is the suggestion that he abused his first wife and more or less purchased his second one. Occasionally, Vance allows Musk to interject things, particularly about who deserves credit for which rocket nozzle, but apparently Musk had “no notes” about these salacious personal life details.

Stop being weird about the Russian language

As you know, Russia is waging an unprovoked war on Ukraine. It should go without saying that my sympathies are with Ukraine, but of course both states are undemocratic, one-party kleptocracies and I have little hope for anything good coming from the conflict.

That’s all besides the point. Since the start of the war, I have had several conversations with linguists who suggested that the study of the Russian language—one of the most important languages in linguistic theorizing over the years—is now “cringe”. This is nonsense. First, official statistics show that a majority of Ukrainian citizens identify as ethnically Russian, and that a substantial minority speak Russian as a first language (and this is probably skewed by social-desirability bias). Secondly, it is wrong to identify a language with any one nation. (It is “cringe” to use flag emojis to label languages; just use the ISO codes.) Third, it is foolish to equate the state with the people who live underneath them, particularly after the end of the kind of mass political movements that in earlier times could stop this kind of state violence. It is a basic corollary of the i-language view that children learn whatever languages they’re sufficiently exposed to, regardless of their location or of their caretakers’ politics. The iniquity of war does not travel from nation to language to its speakers. Stop being weird about it.