The attention R package, describing how to implement from scratch the attention mechanism - which forms the basis of transformers - in the R language is now available on CRAN. A key example of the results that were achieved using (much larger and more complex forms of) transformers is the change from AlphaFold (1) (which relied primarily on LSTM) to AlphaFold2 (which is primarily based on transformers). This change pushed the results in the protein folding competition CASP-14 to a level of accuracy that made the protein structure prediction accurate enough for practical purposes. A major scientific breakthrough, the impact of which can barely overstated.

EDIT 2022-06-24: this code is now available (with helper functions) in the R package attention, which is on CRAN. You can install it simply using: install.packages('attention') See also my blog post attention on CRAN. The development takes place on GitHub. This post describes how to implement the attention mechanism - which forms the basis of transformers - in the R language. The code is translated from the Python original by Stefania Cristina (University of Malta) in her post The Attention Mechanism from Scratch

With over a year working from home and an end not immediately in sight, I felt it was time to think a bit structurally about how to work remotely as effectively as possible. The clear missing element is the watercooler conversations / coffees at the cafeteria. Universities have to some extend always have had to deal with faculty not having a default schedule for being at their desks, which normally makes dropping in easy. The way in which this is typically dealt with is by holding office hours at a set time every week (when not traveling). Normally for the person holding office hours, when nobody comes by, this is a good time to get done with some paperwork that otherwise gets forgotten.

For a job interview at WHO I was asked by build a numeric version of Noughts and Crosses (Tic-Tac-Toe to some), this is called Ron Graham’s Game (repo). Ron Graham’s Game Ron Graham’s Game is a numerical variant of Naughts and Crosses / Tic-Tac-Toe. In the general form, the board is a square matrix of length L >= 3, Player 1 has stones for all the Odd numbers in the range 1:L, Player 2 has stones for all the Even numbers in the range 2:(L-1), a player wins if it completes a row/column/diagonal and the sum equals (1:L)L/2.

Today I gave a presentation at the UN Innovation Network’s TechLearnTalks (archived, backup): The slides from my presentation are available here: https://docs.google.com/presentation/d/1qDtY8jrMnDz3tGpqg-AvgIBB5iiY54Jko6lMDZP7c5o/ The live demo spreadsheet is available here: https://docs.google.com/spreadsheets/d/1j1dXgZ_9RzvBdKFyA1Goii6IzPniNWSLs14D-kN_RNo/ EDIT (2020-06-11): due to popular demand I am turning the spreadsheet into a somewhat more formal product: http://spreadsheet.network/ a neural network in a spreadsheet With an FAQ. Paper to follow.