Lichen Restarted

Originally, lots of work was being put in to support various Python features that are arguably superfluous. The realisation was had that a lot of effort was being made for little practical benefit by trying to support things that are, in the larger picture, not that important. Consequently, Lichen was refocused on a smaller set of more useful Python features.

This document is of historical interest only, with the design and other documents attempting to communicate the results of this restarting effort. Some obsolete information is therefore preserved below. For example, attributes hold context information in the diagrams, but context information is now held in wrappers or is maintained separately within programs.

Objectives:

Names

Names are locals, globals or built-ins. (Special names exist internally to support certain operations.)

Locals inside functions are dynamic; locals outside functions are static, as are module globals. Built-ins are defined statically in the __builtins__ package.

Imports

Imports provide access to external references. The "leaf" module in a module path is the module returned by the statement.

Indicate that module "compiler" is accessed via compiler...

import compiler

Indicate that module "compiler" is accessed via comp...

import compiler as comp

Indicate that module "compiler.ast" is accessed via ast; module "compiler.transformer" is accessed via tr...

import compiler.ast as ast, compiler.transformer as tr

Import compiler.ast, access Function...

from compiler.ast import Function

Import compiler.ast, access Function as F...

from compiler.ast import Function as F

This causes some semantic differences with Python, with the most significant one being the following:

Python

Lichen

Import compiler, import compiler.ast, set ast on compiler...

import compiler.ast

...returning compiler

Import compiler.ast...

import compiler.ast

...returning compiler.ast as ast

Some statements can be rewritten to achieve the same effect:

Python

Lichen

Import compiler, access ast as submodule...

from compiler import ast

Import compiler.ast...

import compiler.ast

...returning compiler.ast as ast

Other kinds of import are not directly possible with Lichen. For example:

Import all names from compiler.ast...

from compiler.ast import *

Some notes:

Self

In Python:

Apart from tests for the nature of the context and the callable, the argument list is effectively variable.

With self as a ubiquitous, hidden parameter:

The argument list for each callable thus remains static, at a cost of allocating an extra argument that may not be used. (Various calling conventions for certain processor architectures employ potentially unused registers, anyway.) Note that a callable may support defaults, however, and thus any argument list may need extending to include default values for parameters without corresponding arguments.

Python

Without self

inst.method(x, y, z)
# -> inst.method(inst, x, y, z)

def method(self, a, b, c):
    # self = inst; a = x; b = y; c = z

inst.method(x, y, z)
# -> inst.method(inst, x, y, z)

def method(a, b, c):
    # self = inst; a = x; b = y; c = z

cls.method(self, x, y, z)

def method(self, a, b, c):
    # parameters = arguments

f = get_using(cls.method, self)
# context of f = self; value of f = cls.method
f(x, y, z)
# -> f(context of f, x, y, z)

def method(a, b, c):
    # self = context of f = self; a = x; b = y; c = z

To avoid usage of self in undefined ways, only methods are able to use self and are not allowed to redefine it. Consequently, when invoking a callable, the context is set (where the callable is unknown until run-time; it is not set if compile-time knowledge indicates that it is not needed), and in situations where self is not permitted, the context is therefore safely ignored. Meanwhile, methods are always supplied with a context compatible with self.

Callable self Remarks
Class context context discarded and replaced by allocated instance
Function null self not permitted, context ignored
Function (stored on class) class as context self not permitted, context ignored
Function (stored on instance) instance as context self not permitted, context ignored
Instance instance as context self set to instance
Method (via class) class as context method not called (see "unbound methods")
Method (via instance) instance as context self set to instance

Note that the treatment of functions stored on classes differs from Python. In Python, such functions would become unbound methods (see below) and would employ their first parameter as an effective self parameter (regardless of name).

Unbound Methods

Since methods acquired directly from classes ("unbound methods" in Python) are meant to be combined with an instance as context (using the get_using function), they must be uncallable until combined with the appropriate context, yet the same methods when acquired via instances ("bound methods" in Python) must be immediately callable.

To support the two different states of methods, the principal structure of a class has attributes referencing uncallable versions of its methods. Meanwhile, such uncallable methods reference callable versions and when instances are employed to access the class attributes, it is these callable versions that are retrieved. For example:

structuresinstance of Ccontext of avalue of acontext of bvalue of bclass Ccontext of mvalue of mcontext of nvalue of nmuncallablenuncallableC.mC.nmcallablencallableget_using(C.m, instance)get_using(C.n, instance)

The precise structure usage is as follows:

methodsclass Ccontext of muncallable for m...attrCuncallable for mC.muncallable for m__fn__bound method referenceunbound method routine__args__minimum #parametersparameter table referenceattrinstancecallable for mget_using(C.m, instance)callable for m__fn__0bound method routine__args__minimum #parametersparameter table referenceparameter table for m...

Callable methods provide a reference to a callable routine in its special callable member, just as functions and classes do. Uncallable methods populate the callable member with a reference to an error routine. Thus, any attempt to call an uncallable method would cause the error routine to be invoked. In addition, uncallable methods reference the corresponding callable methods so that the callable methods can be found and referenced.

Accessor Provider Attribute Context Summary
Instance Instance Function not used Preserve context
Bound method Original instance
Unbound method Providing class
Other Same as value
Instance Class Function not used
Bound method Original instance
Unbound method Accessing instance, if compatible Test and replace context
Other Same as value Preserve context
Class Class Function not used
Bound method Original instance
Unbound method Providing class
Other Same as value

When obtaining an unbound method from an instance attribute, the context of the method attribute is provided. Indeed, the context is always preserved when accessing instance attributes.

When obtaining an unbound method from a class attribute via an instance, the context of the method attribute is tested against the accessing instance. If compatible, an attribute is copied containing the instance as context and a callable method reference as value.

When obtaining an unbound method from a class attribute, the context of the method attribute is provided. Indeed, the context is always preserved when accessing class attributes directly.

When combining an unbound method obtained from a class with an instance using get_using, the context of the method attribute is tested against the supplied instance. If compatible, an attribute is copied containing the instance as context and a callable method reference as value.

Functions as Unbound Methods

Functions not defined within classes could be treated as unbound methods if they were to employ self (thus indicating that they are intended as methods). Such functions would then be recorded as uncallable in the module namespace, needing to be explicitly bound to a class using a special function. However, there appears to be limited utility in defining functions in this way, instead of defining them directly as methods, or instead of merely using such generic functions from existing methods.