Lexer and Interpreter

The Lexer and Interpreter are one of the most important parts of Mini.js that makes state re-renders work.

The Lexer identifies and replaces the identifiers, while the Interpreter executes it.

Last updated at March 2, 2024 by Joeylene

Lexer

The Lexer uses acorn, acorn-walk, and escodegen to replace the identifiers.

acorn - for parsing the Abstract Syntax Tree (AST)
acorn-walk - for traversing the AST and identifying and replacing the identifiers.
escodegen - for turning the AST back to a valid JavaScript code string.

The Lexer is mainly used for two things:

Finding referenced variables in dynamic attributes and events.
Replacing referenced identifiers to use proxyWindow to trigger re-renders.
- Turning firstName into proxyWindow.firstName.
- Turning el.firstName into proxyWindow['Entity123'].firstName
- Turning scope.firstName into proxyWindow['Entity123'].firstName

Lexer Variable Types

There are categorization to the variables that are found by the Lexer:

Declared - these are variables that were declared or created inside the code.
- Declared variables are ignored by the Lexer when it comes to identifying and replacing identifiers for re-renders. An exemption is the this keyword, which is the only declared variable that the Lexer can identify and replace.
- The following variables that are identified as declared are:
  - variables created using const and let.
```
const declaredVar = 'I am declared!'
```
  - functions created using the function keyword.
```
function declaredFunction() {}
```
  - function parameters are considered as declared:
```
function declaredFunction(param1, param2) {}
```
  - the this keyword
```
this.stuff
```
Referenced - these are the variables that were not declared and are just used inside the code.
- Referenced variables are the ones that the Lexer identifies and replaces for re-rendering.
- Native Functions such like eval, alert, setTimeout, etc, are ignored by the Lexer when identifying variables.
- An example of a referenced variable is:
```
const a = b + c
```
  b and c are both referenced variables because you will not find its declaration inside the given code, therefore it is referenced.

Lexer Usage

The lexer can be use to find and replace identifiers:

Getting the identifiers

To get the identifiers in a given code string, you can use the identifier property:

const expr = 'showSearch = false'
const lexer = new Lexer(expr)
lexer.identifiers // outputs: ['showSearch']

const expr = 'const arr = [1, 2, a, b]'
const lexer = new Lexer(expr)
lexer.identifiers // outputs: ['a', 'b']