M-expression

John McCarthy

In computer programming, M-expressions (or meta-expressions) were an early proposed syntax for the Lisp programming language, inspired by contemporary languages such as Fortran and ALGOL. The notation was never implemented into the language and, as such, it was never finalized.

M-expressions are a syntax for LISP code and provide function notation, syntax for a cond form and for embedded literal data (via S-expressions) into programs. Thus M-Expressions used S-Expressions for literal data. The syntax for S-Expressions ("The Data Language") and M-Expressions ("The Meta Language") is defined on pages 8 and 9 of the Lisp 1.5 manual..

M-Expressions also had a corresponding S-Expression representation. Code was manually translated from M-Expressions to S-Expressions. The in M-expressions embedded literal data, then had to be quoted in S-Expressions.

The M-Expression form

append[listvar;(PARIS BERLIN NEWYORK TOKYO)]

then needs to be transformed to the S-Expression form

(APPEND LISTVAR (QUOTE (PARIS BERLIN NEWYORK TOKYO)))

Background

John McCarthy published the first paper on Lisp in 1960 while a research fellow at the Massachusetts Institute of Technology. In it he described a language of symbolic expressions (S-expressions) that could represent complex structures as lists. Then he defined a set of primitive operations on the S-expressions, and a language of meta-expressions (M-expressions) that could be used to define more complex operations. Finally, he showed how the meta-language itself could be represented with S-expressions, resulting in a system that was potentially self-hosting. The draft version of this paper is known as "AI Memo 8".

Example M-expressions (LISP 1.5, 1965) and their S-Expression representation

Expression type	LISP M-expression	corresponding LISP S-expression
Literal atomic symbol, written in uppercase	ALPHA123	(QUOTE ALPHA123)
Literal list in cons notation	(A . (B . NIL))	(QUOTE (A . (B . NIL)))
Literal list in list notation	(A B C 1 2 3)	(QUOTE (A B C 1 2 3))
Identifier for variables and functions, written in lowercase in M-Expressions	name123	NAME123
Function application with variables	f[x;y]	(F X Y)
Function application on an S-Expression	car[(A B C)]	(CAR (QUOTE (A B C)))
Function application on two S-Expressions	cons[A;(B C D)]	(CONS (QUOTE A) (QUOTE (B C D)))
Conditional expression	[lessp[x;0] → minus[x]; T → x]	(COND ((LESSP X 0) (MINUS X)) (T X))
Recursive function definition	label[ff;λ[[x];[atom[x] → x; T → ff[car[x]]]]]	(LABEL FF (LAMBDA (X) (COND ((ATOM X) X) (T (FF (CAR X))))))

McCarthy had planned to develop an automatic Lisp compiler (LISP 2) using M-expressions as the language syntax and S-expressions to describe the compiler's internal processes. Stephen B. Russell read the paper and suggested to him that S-expressions were a more convenient syntax. Although McCarthy disapproved of the idea, Russell and colleague Daniel J. Edwards hand-coded an interpreter program that could execute S-expressions. This program was adopted by McCarthy's research group, establishing S-expressions as the dominant form of Lisp.

McCarthy reflected on the fate of M-expressions in 1979:

The project of defining M-expressions precisely and compiling them or at least translating them into S-expressions was neither finalized nor explicitly abandoned. It just receded into the indefinite future, and a new generation of programmers appeared who preferred internal notation to any FORTRAN-like or ALGOL-like notation that could be devised.

— History of Lisp

The book Anatomy of LISP by John Allen explains the definition of M-Expressions and uses them throughout the book to explain Lisp and its implementation.

Examples

The definitions for the functions apply and eval from the Lisp 1.5 Manual, page 13.

apply[fn;x;a] =
   [atom[fn] →
       [eq[fn;CAR] → caar[x];
        eq[fn;CDR] → cdar[x];
        eq[fn;CONS] → cons[car[x];cadr[x]];
        eq[fn;ATOM] → atom[car[x]];
        eq[fn;EQ] → eq[car[x];cadr[x]];
        T → apply[eval[fn;a];x;a]];
    eq[car[fn];LAMBDA] → eval[caddr[fn];parlis[cadr[fn];x;a]];
    eq[car[fn];LABEL] → apply[caddr[fn];x;cons[cons[cadr[fn];caddr[fn]];a]]]

eval[e;a] =
   [atom[e] → cdr[assoc[e;a]];
    atom[car[e]] →
          [eq[car[e],QUOTE] → cadr[e];
           eq[car[e];COND] → evcon[cdr[e];a];
           T → apply[car[e];evlis[cdr[e];a];a]];
    T → apply[car[e];evlis[cdr[e];a];a]]

Using the function eval on an s-expression.

eval[(EQ (QUOTE A) (CAR (CONS (QUOTE A) (QUOTE (B C D)))));NIL]

Implementations

For LISP

MLisp was a contemporary (1968–1973) project to implement an M-expression-like frontend for Lisp. A few extra features like hygienic macros, pattern matching, and backtracking were incorporated. It eventually evolved into an abandoned LISP70 draft. M-LISP (MetaLISP) from 1989 was another attempt to blend M-expressions with Scheme.

A parser for the "AI Memo 8" M-expression is available in Common Lisp, but the author intends it as a case against M-expressions due to its perceived inability to cope with macros.

Further development

A CGOL (1977) was implemented in MacLisp and follows a similar goal of introducing Algol-like syntax with infix operators. It is known to work on Armed Bear Common Lisp.

A more recent (circa 2003) variant is the I-expression, which use indentation to indicate parentheses implicitly, and are thus in some ways intermediate between S-expressions and M-expressions. I-expressions were introduced in Scheme Request For Implementation 49 as an auxiliary syntax for Scheme, but they have not been widely adopted.

A further development is the "sweet" t-expression, which has infix operators without precedence. Like I-expressions, t-expressions are only a simple transformation away from S-expressions, so that theoretically they can be used on any Lisp dialect and not interfere with features like macros.

Additional syntax-related include Apple's Dylan (Algol-like tokens) and Clojure's addition of other literal syntaxes.